CYCLONE

CYCLONE

CYCLONE is a replication tool which uses the Change Data Capture (CDC) method.

Overview

Database records the data changes which occur during the operation in the redo log file for the recovery. CDC performs the replication by analyzing the information of the recorded redo log file.
CYCLONE is driven being divided into master and slave. Master recognizes the changes of the redo log file in the original database and analyzes it, then transfers it to the slave. Slave analyzes the received data and performs the replication by using ODBC.

Operational Features

Operational Restrictions

The occurrence of give up and whether to allow DDL statement according to DDL category

DDL category

Occurrence of

give up

Whether to

allow

DDL statement

DDL statement

Table DDL

X

-

CREATE TABLE

O

X

DROP TABLE

O

X

TRUNCATE TABLE

O

X

ALTER TABLE .. RENAME

X

-

ALTER TABLE .. STORAGE

X

-

ALTER TABLE .. ADD SUPPLEMENTAL LOG

O

X

ALTER TABLE .. DROP SUPPLEMENTAL LOG

X

-

ALTER TABLE .. READ { ONLY | WRITE }

Column DDL

O

O

ALTER TABLE .. ADD COLUMN

O

X

ALTER TABLE .. SET UNUSED COLUMN

X

-

ALTER TABLE .. RENAME COLUMN

X

-

ALTER TABLE .. ALTER COLUMN .. SET DEFAULT

X

-

ALTER TABLE .. ALTER COLUMN .. DROP DEFAULT

O

X

ALTER TABLE .. ALTER COLUMN .. SET NOT NULL

O

X

ALTER TABLE .. ALTER COLUMN .. DROP NOT NULL

O

X

ALTER TABLE .. ALTER COLUMN .. ALTER IDENTITY

O

X

ALTER TABLE .. ALTER COLUMN .. DROP IDENTITY

O

O

ALTER TABLE .. ALTER COLUMN .. SET DATATYPE

Constraint DDL

O

X

ALTER TABLE .. ADD CONSTRAINT

O

X

ALTER TABLE .. DROP CONSTRAINT

O

X

ALTER TABLE .. ALTER CONSTRAINT

X

-

ALTER TABLE .. RENAME CONSTRAINT

Index DDL

O

X

CREATE UNIQUE INDEX

X

-

CREATE INDEX

O

X

DROP INDEX unique_index

X

-

DROP INDEX non_unique_index

X

-

ALTER INDEX .. STORAGE

X

-

ALTER INDEX .. RENAME

Superordinate object

of table

O

X

DROP USER

O

X

DROP SCHEMA

O

X

DROP TABLESPACE

The server property DISABLE_DDL_CDC_GIVEUP can disable the DDL statement which causes the replication give up to avoid user created errors.

DDL Processing during Replication

It should comply with the following procedure when the allowable DDL is performed for the table in which the replication is being performed.

  1. Terminate all operating CYCLONE in master and slave before executing DDL in master.

    • The process in progress does not need to be terminated.

    • e.g. cyclone --master --stop / cyclone --slave --stop

  2. Execute DDL in both master and slave. (It should be the allowable DDL.)

  3. Restart CYCLONE of master and slave.

    • cyclone --master --start ... / cyclone --slave --start ...

    • It does not require --reset option.

    • It performs the recovery from the termination of the first stage and process DDL when restarting. (Refer to trace log)

DDL application procedure

Item

MASTER CYCLONE

MASTER DB

SLAVE CYCLONE

SLAVE DB

1

CYCLONE STOP

-

-

-

2

-

-

CYCLONE STOP

-

3

-

Executing DDL

-

-

4

-

-

-

Executing DDL

5

CYCLONE START

-

-

-

6

-

-

CYCLONE START

-

If the table which executed DDL in CYCLONE master is given up after DDL application procedure is normally performed, then the following two should be checked.


  1. Check if DDL performed in the master DB and in the slave DB is same, and check if the table structure is same after performing the DDL. (The order of performing DDL does not matter.)

  2. Check if the performed DDL is allowable.


If the table is given up due to the reasons above, the given-up table should be replicated again from the current point by using reset in table unit and restarting it. (e.g. cyclone --start --master --reset TABLE_NAME).

Datatype Compatibility When Interworking with Other DBMS

Oracle

Datatype compatible with Oracle

GOLDILOCKS

Oracle

Remarks

Boolean

X

The corresponding datatype does not exist.

NATIVE_SMALLINT

NUMBER(5)

-

NATIVE_INTEGER

NUMBER(10)

-

NATIVE_BIGINT

NUMBER(19)

-

NATIVE_REAL

BINARY_FLOAT

-

NATIVE_DOUBLE

BINARY_DOUBLE

-

FLOAT

FLOAT

-

SMALLINT

NUMBER(5,0)

-

INTEGER

NUMBER(10,0)

-

BIGINT

NUMBER(19,0)

-

INT2

NUMBER(5,0)

-

INT4

NUMBER(10,0)

-

INT8

NUMBER(19,0)

-

REAL

FLOAT(24)

-

DOUBLE

FLOAT(53)

-

DOUBLE PRECISION

FLOAT(53)

-

FLOAT4

FLOAT(24)

-

FLOAT8

FLOAT(53)

-

DECIMAL

NUMERIC

-

NUMBER

NUMBER

-

NUMERIC

NUMERIC

-

CHAR

CHAR

-

VARCHAR

VARCHAR, VARCHAR2

-

BINARY

RAW

-

VARBINARY

RAW

-

DATE

DATE

-

TIME

X

The corresponding datatype does not exist.

TIMESTAMP

TIMESTAMP

-

TIMESTAMP WITH TIMEZONE

X

It does not support ODBC driver.

INTERVAL

X

It does not support ODBC driver.

LONG VARCHAR

LONG VARCHAR

-

LONG VARBINARY

LONG RAW

-

Others

The replication moments are as follows.

Requirements

Original GOLDILOCKS: It is required to perform GOLDILOCKS preparations, user registration and privilege setting all.
Remote GOLDILOCKS: It is required to perform user registration and privilege setting only.

GOLDILOCKS Requirements

The followings should be set in GOLDILOCKS before starting the replication using CYCLONE.

SUPPLEMENTAL LOGGING

SUPPLEMENTAL LOGGING stores additional information in the redo log file for the replication of CYCLONE. The database restart is required to change the settings of the database in operation, but the database restart is not required to set SUPPLEMENTAL LOGGING for a particular table.

Setting SUPPLEMENTAL LOGGING in Database

Setting SUPPLEMENTAL LOGGING in the Specific Table Participating in Replication

<add table supplemental log statement> ::=
    ALTER TABLE table_name 
        ADD SUPPLEMENTAL LOG DATA ( PRIMARY KEY ) COLUMNS
    ;

ARCHIVE LOG

GOLDILOCKS reuses the redo log files recursively. When GOLDILOCKS reuses the redo log file being processed by CYCLONE, then CYCLONE does not proceed and is terminated. GOLDILOCKS should be operated in ARCHIVE LOG mode to ensure the continuous replication operation.

Changing Database in Operation to ARCHIVE LOG Mode

gSQL> \startup mount

Startup success

gSQL> alter database archivelog;

Database altered.

Setting ARCHIVE LOG Mode when Creating Database

The path in which ARCHIVE LOG file is stored can be viewed and updated with 'ARCHIVELOG_DIR'.

Settings in Cluster Environment

When GOLDILOCKS in which slave is operated is operated in cluster, then either TRANSACTION_COMMIT_WRITE_MODE is '1' (Wait), or CLUSTER_ASYNC_COMMIT is 'NO'. Either of them should be set.

This prevents data confliction due to cluster's transaction policy and Cyclone's distributor policy.

Altering TRANSACTION_COMMIT_WRITE_MODE

If the value of TRANSACTION_COMMIT_WRITE_MODE is '0', then it is NO-WAIT, and if it is '1', then it is WAIT.

Altering CLUSTER_ASYNC_COMMIT

gSQL> alter system set CLUSTER_ASYNC_COMMIT=NO;

System altered.

DATA_STORE_MODE

CYCLONE performs the replication by reading the redo log files of GOLDILOCKS. Therefore, GOLDILOCKS should be operated in Transactional Data Store (TDS) mode.

Changing DATA_STORE_MODE

If the value of DATA_STORE_MODE is 1, it indicates Concurrent Data Store (CDS), and if it is 2, it indicates Transactional Data Store (TDS).

Registering User and Setting Privileges

CYCLONE retrieves and manipulates the required information during the operation. The user operating CYCLONE and the proper privileges for the user are required.

Creating Database User

A specific user should be added to operate CYCLONE, and the corresponding user should be added for all GOLDILOCKS of which CYCLONE is operated in master, slave mode.

<user definition> ::=
    CREATE USER user_identifier IDENTIFIED BY password
    [ DEFAULT TABLESPACE tablespace_name ]
    [ TEMPORARY TABLESPACE tablespace_name ]
    [ INDEX TABLESPACE {tablespace_name|NULL} ]
    [ <schema clause> ]
    ;

<schema clause> ::=
      WITH SCHEMA [schema_name]
    | WITHOUT SCHEMA

The following is an example of creating the user cdc_user with the password cdc_password.

gSQL> CREATE USER cdc_user IDENTIFIED BY cdc_password;

Database Privileges

Granting the User's Access Privilege

The following is an example for granting the access privilege to cdc_user. It should be set on all GOLDILOCKS which is operated in both master/ slave mode.

gSQL> GRANT CREATE SESSION ON DATABASE TO cdc_user;

Granting the Privilege for Altering Table

The following is an example of granting the table updating privilege to cdc_user. It is set on GOLDILOCKS which is operated in slave mode.

gSQL> GRANT INSERT ANY TABLE, DELETE ANY TABLE, UPDATE ANY TABLE ON
DATABASE TO cdc_user;

Tablespace Privileges

The privileges on the data tablespace and temporary tablespace should be set. The following is an example of granting the privilege for using the default tablespace to cdc_user. It is set on GOLDILOCKS which is operated in slave mode.

gSQL> GRANT CREATE OBJECT ON TABLESPACE mem_data_tbs TO cdc_user;
gSQL> GRANT CREATE OBJECT ON TABLESPACE mem_temp_tbs TO cdc_user;

Schema Privileges

The schema privileges for creating and managing meta managed in CYCLONE should be set. The following is an example granting the schema privilege to cdc_user. It is set on GOLDILOCKS which is operated in slave mode.

gSQL> GRANT CREATE TABLE, CREATE INDEX, CREATE SEQUENCE, CREATE VIEW,
ADD CONSTRAINT ON SCHEMA cdc_user TO cdc_user;

Configuration

Configuration File

When performing CYCLONE, the information and options required for operating are set by using the configuration file.

Configuration file options

Name

Description

Coverage

COMM_CHUNK_COUNT

It sets the size of BUFFER for communication.

Master/ slave

DSN

It sets Data Source Name.

Master/ slave

GROUP_NAME

It sets the group name.

Master/ slave

HOST_IP

It sets the host IP address of which GOLDILOCKS operates.

Master/ slave

HOST_EXTERNAL_IP

It is used when slave GOLDILOCKS IP to which the cyclone master is to be connected is different from HOST_IP. (When master and slave are on wan section.)

Slave

HOST_PORT

It sets the host port of which GOLDILOCKS operates.

Master/ slave

PORT

It sets the port for master/ slave communication.

Master/ slave

USER_ID

It sets the user name.

Master/ slave

USER_PW

It sets the user password.

Master/ slave

USER_ENCRYPT_PW

It sets the encrypted password for DB user.

Master/ slave

CAPTURE_TABLE

It sets the table to be replicated.

Master

LOG_PATH

It is used when interworking with LOGMIRROR and it sets the location of the redo log file.

Master

PROTOCOL

It sets the connection type which is to be connected to GOLDILOCKS. (DA or TCP)

Master/ slave

READ_LOG_BLOCK_COUNT

It sets the amount of data to be read at a time when operating CAPTURE.

Master

TRANS_SORT_AREA_SIZE

It sets the size of the BUFFER to be allocated to CAPTURE.

Master

TRANS_FILE_PATH

It sets the location in which the temporarily generated file is to be stored when operating CAPTURE.

Master

SYNCHER_COUNT

It is applied when using SYNC feature, and it sets the number of SYNCHER simultaneously performing the data insertion.

Master

SYNC_ARRAY_SIZE

It is applied when using SYNC feature, and it sets the array size which insert data at a time.

Master

GIVEUP_INTERVAL

If the replication performance speed is lower than GOLDILOCKS performance, it is set to stop the replication.

Master

APPLIER_COUNT

It sets the number of APPLIER simultaneously performed during the replication.

Slave

APPLY_COMMIT_SIZE

It sets the maximum size for COMMIT during the replication.

Slave

APPLY_TABLE

It sets the table to which the replicated table is to be applied.

Slave

MASTER_IP

It sets the IP address of the equipment of which CYCLONE master is operated.

Slave

PROPAGATE_MODE

It sets whether to propagate the data applied by CYCLONE.

Slave

UPDATE_APPLY_MODE

It distinguishes the operation when updating. (The default value is 0.)

  • 0: It is updated only when the primary keys are same.

  • 1: It is updated only when the primary key and the value before the update are same.

  • 2: It is updated only when the primary keys are same. It compares the value before and after the update, then leaves a log if the values are different.

Slave

TCP_NODELAY

It sets TCP_NODELAY option of a socket. (The default value is 1.)

  • 0: TCP_NODELAY off

  • 1: TCP_NODELAY on

Master

HEARTBEAT_TIMEOUT

It sets the maximum time (second) maintaining connection if the connection is not smooth due to network disconnection or system error after replication connection.

Master/ slave

LOG_CAPTURE_INTERVAL_1

It sets the execution cycle of capture. If the value is not changed after executing 10 times with that value, then it is converted to the value of LOG_CAPTURE_INTERVAL_2 and performs capture. (The default value is 0.2 seconds.)

Master

LOG_CAPTURE_INTERVAL_2

It sets the execution cycle of capture. If the value is not changed after executing with the value of LOG_CAPTURE_INTERVAL_1, then it sets the execution cycle of capture. (The default value is 1 second.)

Master

CLUSTER

It specifies the connection information of a master when the master is in cluster environment. A master consists of the following three information.

  • ID: It is a delimiter and sets to 1 or more value.

  • MASTER_IP

  • PORT

Slave

ORACLE_DRIVER

It specifies the file location of the oracle driver provided by Oracle.

Slave

PACKET_COMPRESSION_MODE

It sets whether to compress the data of communication of master and slave. (1: Enable, 0: Disable, Default: Enable)

Master

Configuration Option

COMM_CHUNK_COUNT

• Settings applied to all groups

COMM_CHUNK_COUNT = 10

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    COMM_CHUNK_COUNT=20
    ....
    ....
}

DSN

• Settings applied to all groups

DSN=GOLDILOCKS

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    DSN=GOLDILOCKS
    ....
    ....
}

GROUP_NAME

GROUP_NAME = testGROUP
{
    ....
    ....
}

HOST_IP

• Settings applied to all groups

HOST_IP = 127.0.0.1

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    HOST_IP = 127.0.0.1
    ....
    ....
}

HOST_EXTERNAL_IP

• Settings applied to all groups

HOST_EXTERNAL_IP = 192.168.0.120

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    HOST_EXTERNAL_IP = 192.168.0.120
    ....
    ....
}

HOST_PORT

• Settings applied to all groups

HOST_PORT = 22531

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    HOST_PORT = 22531
    ....
    ....
}

PORT

• It can be set only within a group.

GROUP_NAME = testGROUP
{
    PORT = 21102
    ....
    ....
}

USER_ID

• Settings applied to all groups

USER_ID = testID

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    USER_ID = testID
    ....
    ....
}

USER_PW

• Settings applied to all groups

USER_PW = testPW

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    USER_PW = testPW
    ....
    ....
}

USER_ENCRYPT_PW

• Settings applied to all groups

USER_ENCRYPT_PW = 't33KImiqvhqNyfN+uZmFrw=='

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    USER_ENCRYPT_PW = 't33KImiqvhqNyfN+uZmFrw=='
    ....
    ....
}

CAPTURE_TABLE

GROUP_NAME = testGROUP
{
    CAPTURE_TABLE = 
    (
        testSchema1.testTable1,
        testSchema1.testTable2,
        testSchema2.testTable1
    )
}

LOG_PATH

• Settings applied to all groups

LOG_PATH = '/data/wal/'

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    LOG_PATH = '/data/wal/'
    ....
    ....
}

PROTOCOL

• Settings applied to all groups

PROTOCOL = DA

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    PROTOCOL = DA
    ....
    ....
}

READ_LOG_BLOCK_COUNT

• Settings applied to all groups

READ_LOG_BLOCK_COUNT = 1024

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    READ_LOG_BLOCK_COUNT = 1024
    ....
    ....
}

TRANS_SORT_AREA_SIZE

• Settings applied to all groups

TRANS_SORT_AREA_SIZE = 300

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    TRANS_SORT_AREA_SIZE = 300
    ....
    ....
}

TRANS_FILE_PATH

• Settings applied to all groups

TRANS_FILE_PATH = '/data/TmpTrans'

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    TRANS_FILE_PATH = '/data/TmpTrans'
    ....
    ....
}

SYNCHER_COUNT

• Settings applied to all groups

SYNCHER_COUNT = 8

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    SYNCHER_COUNT = 8
    ....
    ....
}

SYNC_ARRAY_SIZE

• Settings applied to all groups

SYNC_ARRAY_SIZE = 1000

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    SYNC_ARRAY_SIZE = 1000
    ....
    ....
}

GIVEUP_INTERVAL

• Settings applied to all groups

GIVEUP_INTERVAL = 10000

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    GIVEUP_INTERVAL = 10000
    ....
    ....
}

APPLIER_COUNT

• Settings applied to all groups

APPLIER_COUNT = 16

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    APPLIER_COUNT = 16
    ....
    ....
}

APPLY_COMMIT_COUNT

• Settings applied to all groups

APPLY_COMMIT_COUNT = 1000

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    APPLY_COMMIT_COUNT = 1000
    ....
    ....
}

APPLY_TABLE

GROUP_NAME = testGROUP
{
    APPLY_TABLE =
    (
        testSchema1.testTable1 TO testSchema1.testTable1,
        testSchema1.testTable2 TO testSchema2.testTable3,
        testSchema2.testTable3 TO testSchema2.testTable4
    )
}

MASTER_IP

• Settings applied to all groups

MASTER_IP = 192.168.0.100

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    MASTER_IP = 192.168.0.100
    ....
    ....
}

PROPAGATE_MODE

• Settings applied to all groups

PROPAGATE_MODE = 1

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    PROPAGATE_MODE = 1
    ....
    ....
}

UPDATE_APPLY_MODE

• Settings applied to all groups

UPDATE_APPLY_MODE = 1

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    UPDATE_APPLY_MODE = 1
    ....
    ....
}

For a column whose data type is long varchar or long varbinary, then only the lengths are compared for the performance reason.

TCP_NODELAY

• Settings applied to all groups

TCP_NODELAY = 1

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    TCP_NODELAY = 1
    ....
    ....
}

HEARTBEAT_TIMEOUT

• Settings applied to all groups

HEARTBEAT_TIMEOUT = 40

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    HEARTBEAT_TIMEOUT = 40
    ....
    ....
}

LOG_CAPTURE_INTERVAL_1

• Settings applied to all groups

LOG_CAPTURE_INTERVAL_1 = 200

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    LOG_CAPTURE_INTERVAL_1 = 200
    ....
    ....
}

LOG_CAPTURE_INTERVAL_2

• Settings applied to all groups

LOG_CAPTURE_INTERVAL_2 = 1000

• Settings applied to a specific group

GROUP_NAME = testGROUP
{
    LOG_CAPTURE_INTERVAL_2 = 1000
    ....
    ....
}

CLUSTER

• Cluster can be set only within a group.

GROUP_NAME = testGROUP
{
    CLUSTER = (ID=1, MASTER_IP=192.0.0.100, PORT=21102),
              (ID=2, MASTER_IP=192.0.0.101, PORT=21103)
    ....
    ....
}

ORACLE_DRIVER

• It can be set in any group.

ORACLE_DRIVER = '/app/oracle/product/11.2.0/db_1/lib/libsqora.so.11.1'

GROUP_NAME = testGROUP
{
    ....
    ....
}

PACKET_COMPRESSION_MODE

• It can be set in a specific group.

GROUP_NAME = testGROUP
{
    PACKET_COMPRESSION_MODE = 1
    ....
    ....
}

Operating

CYCLONE can be operated in D/A or C/S environment of GOLDILOCKS.
CONFIG file or PROTOCOL configuration of ODBC.INI should be used according to the environments as follows. CONFIG configuration takes precedence over ODBC.INI configuration.
CONFIG, ODBC.INI

Configuration

Description

PROTOCOL=DA

It is used in D/A environment. (Default)

PROTOCOL=TCP

It is used in C/S environment.

The running contents during the operation can be viewed through trace log.

Item

File

Cyclone master

$GOLDILOCKS_DATA/trc/cyclone_master.trc

Cyclone slave

$GOLDILOCKS_DATA/trc/cyclone_slave.trc

For more information about error message and its handling method stored in trace log, refer to Troubleshooting of CYCLONE.

GOLDILOCKS Connection Policy

It is the policy for CYCLONE to connect to GOLDILOCKS.
There are two ways to connect to GOLDILOCKS. One is  that CYCLONE connects to local GOLDILOCKS, and the other is that CYCLONE master remotely connects to slave side GOLDILOCKS on the slave's side when performing SYNC. Both use  CYCLONE config value is preferentially used, then use odbc.ini. However, when performing SYNC, PROTOCOL value is ignored and  it is set to TCP no matter what.
The config properties relating to GOLDILOCKS connection are DSN, PROTOCOL, HOST_IP, HOST_EXTERNAL_IP, HOST_PORT, USER_ID, USER_PW.
(USER_ENCRYPT_PW is listed as USER_PW because it replaces USER_PW.)

Correct Examples

D/A로 접속하였기 때문에 USER_ID, USER_PW만으로도 가능하다.

Item

File

CONFIG

USER_ID=test, USER_PW=test

GOLDILOCKS configuration of odbc.ini

-

CONFIG나 odbc.ini 둘 중 한 군데에만 정보가 있으면 된다. D/A로 동작하기 때문에 HOST_IP, HOST_PORT는 무시된다.

Item

File

CONFIG

HOST_IP=127.0.0.1, HOST_PORT=22581,USER_ID=test

GOLDILOCKS configuration of odbc.ini

USER_PW=test

TCP로 접속하였고 CONFIG가 우선하므로 USER_ID는 test 계정으로 동작한다.

Item

File

CONFIG

PROTOCOL=TCP, USER_ID=test, USER_PW=test

GOLDILOCKS configuration of odbc.ini

HOST_IP=127.0.0.1, HOST_PORT=22581,

USER_ID=test2, USER_PW=test2

D/A로 접속하더라도 slave에는 HOST_EXTERANL_IP설정이 가능하다.

Item

File

CONFIG

PROTOCOL=DA, HOST_EXTERNAL_IP=192.168.0.10, USER_ID=test

GOLDILOCKS configuration of odbc.ini

HOST_IP=127.0.0.1, HOST_PORT=22581,USER_PW=test

Wrong Examples

D/A로 접속하는데 USER_PW가 없다.

Item

File

CONFIG

HOST_EXTERNAL_IP=192.168.0.10

GOLDILOCKS configuration of odbc.ini

PROTOCOL=DA,USER_ID=test

TCP로 접속하는데 HOST_IP가 없다.

Item

File

CONFIG

HOST_EXTERNAL_IP=192.168.0.10, USER_ID=test

GOLDILOCKS configuration of odbc.ini

PROTOCOL=TCP, HOST_PORT=22581,USER_PW=test

Executing Option

CYCLONE should be used with the following options at run-time.

Executing options

Option

Description

Remarks

--start | -s

It starts CYCLONE.

It should be used together with --master | --slave.

--stop | -t

It terminates CYCLONE.

It should be used together with --master | --slave.

--master | -m

It is performed in master mode.

It should be used together with --start | --stop.

--slave | -l

It is performed in slave mode.

It should be used together with --start | --stop.

--status | -u

It displays the operating status of CYCLONE.

It should be used together with --master | --slave.

--conf | -c

It sets the path of configuration file which is required when executing CYCLONE.

It is input in --conf CONFIG_FILE format.

It should be used together with --start.

If it is not explicitly set, master uses $GOLDILOCKS_DATA/cyclone.master.conf, and slave uses $GOLDILOCKS_DATA/cyclone.slave.conf.

--silent | -i

It sets not to output messages.

-

--reset | -r

It resets operational information of the replication.

It is input in --reset TABLE_NAME or --reset all format.

It should be described within a single quote (') when resetting multiple tables.

--group | -g

It sets a specific group.

It is input in --group GROUP_NAME format.

--help | -h

It outputs the help message.


--sync | -n

It performs data synchronization.

It should be used together with --master | --slave.

--encrypt | -e

It encrypts the user password with the given key.

-

--key | -k

It sets the encryption key when performing the --encrypt option.

If USER_ENCRYPT_PW is used in the config, it sets the decryption key.

-

--info | -o

It displays the status of the table which is currently being replicated.

It should be used together with --master, --group.

--recovery | -v

It is used when passing the replication being performed in a standalone mode of cluster environment to another cluster member.

It is used in --recovery GROUP_NAME form, and GROUP_NAME describes CYCLONE GROUP_NAME of a slave which was previously performed.

--stand-alone | -S

It is operated in a standalone mode of cluster environment. (It is operated in a cluster mode in cluster environment.)

It is valid only in master. (It is set in the configuration file in slave.)

--local | -a

It is used when synchronizing the table sharded in cluster environment. It synchronizes only the data in the corresponding cluster group.

It should be used together with --sync.

prompt> cyclone --master --start
prompt> cyclone --master --stop
prompt> cyclone --slave --start
prompt> cyclone --slave --stop
prompt> cyclone --master --start --group TEST_GROUP
prompt> cyclone --master --stop --group TEST_GROUP
prompt> cyclone --slave --start --conf TEST_CONFIG
prompt> cyclone --encrypt test --key 1234
Cyclone Encrypted Passwd : 'YFH+bpBPNvk='
prompt> cyclone --master --start --key 1234
prompt> cyclone --master --start --reset all
prompt> cyclone --slave --start --reset all
prompt> cyclone --master --start --reset 'T1, T2'
prompt> cyclone --slave --start
prompt> cyclone --master --info --group GROUP1
================================================
 GROUP NAME = GROUP1
================================================
 SCHEMA NAME         : PUBLIC
 TABLE NAME          : T1 (GIVE-UP (DDL-LSN:129541))
 PHYSICAL ID         : 5299989643264
================================================
 SCHEMA NAME         : PUBLIC
 TABLE NAME          : T2 (ACTIVE (CAPTURE-START-LSN:128922))
 PHYSICAL ID         : 35549444308992
================================================
 SCHEMA NAME         : PUBLIC
 TABLE NAME          : T3 (ACTIVE (CAPTURE-START-LSN:128922))
 PHYSICAL ID         : 35558034243584
================================================
 SCHEMA NAME         : PUBLIC
 TABLE NAME          : T4 (ACTIVE (CAPTURE-START-LSN:128922))
 PHYSICAL ID         : 35566624178176
================================================
   TOTAL   COUNT : 4
   GIVE-UP COUNT : 1
================================================

For more information about examples of adding/deleting node or group, refer to Adding and Deleting Nodes.

For more information about examples of initializing replication, refer to Initializing Replication.

Operating Examples

The structure of operating CYCLONE is as follows.

The structure of operating CYCLONE

The structure of operating CYCLONE

Operating Order

  1. Set the original GOLDILOCKS environment.

  2. Set the remote GOLDILOCKS environment.

  3. Set the CYCLONE MASTER environment.

  4. Set the CYCLONE SLAVE environment.

  5. Execute and operate it.

Original GOLDILOCKS Configuration

Execute all described Requirements.

Create tables T1, T2 for testing.
gSQL > CREATE TABLE T1( COL1 INTEGER PRIMARY KEY, COL2 VARCHAR(20) );
gSQL > CREATE TABLE T2( COL1 INTEGER PRIMARY KEY, COL2 VARCHAR(20) );
gSQL > COMMIT;

Remote GOLDILOCKS Configuration

Execute Registering User and Setting Privileges.

CYCLONE MASTER Configuration

• It is connected with D/A of the same device, so HOST information does not exist.

USER_ID = cdc_user
USER_PW = cdc_password

GROUP_NAME = GROUP1
{
    PORT = 21102
    CAPTURE_TABLE =
    ( 
        T1,
        T2
    )
}

CYCLONE SLAVE Configuration

USER_ID = cdc_user
USER_PW = cdc_password
HOST_IP = 192.168.0.20
HOST_PORT = 22581

GROUP_NAME = GROUP1
{
    PORT = 21102
    APPLY_TABLE =
    (
        T1 TO T1,
        T2 TO T2
    )
}

Executing and Operating

cyclone --master --start --conf $GOLDILOCKS_DATA/conf/cyclone.master.conf
cyclone --slave --start --conf $GOLDILOCKS_DATA/conf/cyclone.slave.conf

Synchronizing Data

cyclone --master --start --conf $GOLDILOCKS_DATA/conf/cyclone.master.conf --sync all
cyclone --slave --start --conf $GOLDILOCKS_DATA/conf/cyclone.slave.conf --sync --all
gSQL> CREATE TABLE T1( COL1 INTEGER PRIMARY KEY, COL2 VARCHAR(20) );
gSQL> INSERT INTO T1 VALUES( 1, 'HELLO' );
gSQL> CREATE TABLE T1( COL1 INTEGER PRIMARY KEY, COL2 VARCHAR(20) );
gSQL> INSERT INTO T1 VALUES( 1, 'BYE' );
cyclone --master --start --sync all
cyclone --slave  --start --sync all
SLAVE cyclone log
[RECEIVER] start the data synchronization.
[RECEIVER] Error Occurred.
STOP Done.

An error occurs and the sync does not operate due to an attempt to using the same value in the primary key.

Initializing Replication Information



Recovery

The process of resetting the replication information


1. Replicate with G1N1 member (standalone) in cluster environment.

prompt> cyclone --master --start --reset ALL --group g1n1_master --stand-alone

prompt> cyclone --slave --start --reset ALL --group g1n1_slave


2. Stop operating the existing cyclone to stop the cluster member or to hand over the service.

prompt> cyclone --master --stop

prompt> cyclone --slave --stop


3. Service the new replication (G1N2) by using the information of the existing replication. (G1N1)

prompt> cyclone --master --start --reset all --group g1n2_master --stand-alone --recovery g1n1_slave

prompt> cyclone --master --start --reset all --group g1n2_slave

Operating CYCLONE in Cluster

Requirements

  1. Master connecting to a slave should be a member in the same cluster group.

    • If it is a member of another cluster group, it should be operated in another slave.

  2. If master is cluster environment, CYCLONE is basically operated in cluster mode, and --stand-alone option should be used to operate CYCLONE in standalone mode.

    • --stand-alone option is used only when performing master.

    • If the cluster information of master is described by adding CLUSTER option to configuration file, then slave is operated in cluster mode.

Operating

Starting and Terminating

Node Type

In cluster environment, CYCLONE classifies master connecting to slave into a trust node and a non-trust node.
Node in cluster environment

Node

Description

Replication

Trust node

It is a node all of whose tables participating in replication are online.

O

Non-trust node

It is a node one or more of whose tables participating in replication are offline.

X

Operation According to Node Status

Executing

Configuration File

The configuration file of master is as same as that of standalone.
The following is an example of configuring G1N1 master environment, and IP of that device is assumed as 192.168.0.10.
USER_ID = cdc_user
USER_PW = cdc_password

GROUP_NAME = G1N1_Master
{
    PORT = 21011
    CAPTURE_TABLE =
    (
        T1,
        T2
    )
}
The following is an example of configuring G1N2 master environment, and IP of that device is assumed as 192.168.0.20.
USER_ID = cdc_user
USER_PW = cdc_password

GROUP_NAME = G1N2_Master
{
    PORT = 21012
    CAPTURE_TABLE =
    (
        T1,
        T2
    )
}
CLUSTER is added to the configuration file of slave to describe the information of master.

MASTER_IP, PORT which are the master information of CLUSTER are described in CLUSTER configuration.

USER_ID = cdc_user
USER_PW = cdc_password

GROUP_NAME = G1_Slave
{
    CLUSTER = ( ID = 1, MASTER_IP = 192.168.0.10, PORT = 21011 ),
              ( ID = 2, MASTER_IP = 192.168.0.20, PORT = 21012 )
    APPLY_TABLE =
    (
        T1 TO T1,
        T2 TO T2
    )
}

Executing Method

Executing in Master

• It is executed in G1N1.
cyclone --master --start --conf $GOLDILOCKS_DATA/conf/cyclone.master.conf --group G1N1_Master
cyclone --master --start --conf $GOLDILOCKS_DATA/conf/cyclone.master.conf --group G1N2_Master

Executing in Slave

cyclone --slave --start --conf $GOLDILOCKS_DATA/conf/cyclone.slave.conf --group G1_Slave

SYNC Method

Executing Master

It is executed in G1N1. (It is master of which SYNC is to be executed, and it uses --sync option.)
cyclone --master --start --conf $GOLDILOCKS_DATA/conf/cyclone.master.conf --sync all

Executing Slave

cyclone --slave --start --conf $GOLDILOCKS_DATA/conf/cyclone.slave.conf --sync all

--local option should be used when syncing if a sharded table participates in the replication. It is because the data of a sharded table exists only in each cluster group, and data in other cluster groups are synchronized when executing sync without --local option. However, if all tables participating in the replication in cluster environment are cloned tables, then --local option does not need to be used.

Others

--reset option can be used as same as it is used in standalone environment.

Monitoring (CYMON)

CYMON (CYclone MONitor) is a tool for monitoring CYCLONE which is a CDC replication tool. It periodically updates the monitoring information of CYCLONE to GOLDILOCKS operated as master.
Execution files

File name

Description

cymon

Monitoring cyclone

Configuration File

The configuration file of cymon uses the configuration file used when running cyclone as master. If a specific configuration file is not set by using the --conf option, the $GOLDILOCKS_DATA/conf/cyclone.master.conf file is read as the default when cyclone is run as master.

CYMON should be run on the device which is as same as the device of which CYCLONE runs as master.

Monitoring Contents

CYMON

CYMON periodically updates the operating information of CYCLONE to the CYCLONE_MONITOR_INFO table of GOLDILOCKS. The monitoring information is as follows.

CYCLONE_MONITOR_INFO

Column

Description

GROUP_NAME

It is the name of a group in which CYCLONE was executed.

TIME

It is the information of time at which the information was updated. (YYYY-MM-DD HH24:MI:SS)

MASTER_STATE

It is the state of CYCLONE which is operated as MASTER.

  • N/A: The state is unknown.

  • READY: It is waiting for connection of SLAVE

  • RUNNING: SLAVE is connected and the replication is running.

SLAVE_STATE

It is the state of CYCLONE which is operated as SLAVE.

  • N/A: The state is unknown.

  • RUNNING: The replication is running.

MASTER_PORT

It is the information of PORT on which CYCLONE operated in MASTER is waiting for slave.

SLAVE_IP

It is an IP address of the device connected with CYCLONE SLAVE.

REDO_LOG_FILESEQ

It is the sequence number of the redo log file of the running GOLDILOCKS.

REDO_LOG_BLOCKSEQ

It is the block sequence number of the redo log file of the running GOLDILOCKS.

CAPTURE_FILESEQ

It is the sequence number of the redo log file captured by CYCLONE master.

CAPTURE_BLOCKSEQ

It is the block sequence number of the redo log file captured by CYCLONE master.

CAPTURE_INTERVAL

It is the number of the remaining redo log blocks to be processed by CYCLONE master

  • 1 block = 512 bytes (Redo log block size = 512 bytes)

CAPTURE_INTERVAL_SIZE

It is the size of the remaining redo log files to be processed by CYCLONE master.

TOTAL_TX_COUNT

It is the number of all transactions captured by CYCLONE master.

CAPTURE_TX_COUNT

It is the number of all transactions in which the replication targets are included among the transactions captured by CYCLONE master.

CAPTURE_COMMIT_LSN

It is the commit log number of the last transaction which was captured in CYCLONE master. It is not updated if there is not a transaction to be captured any more.

INTERVAL information is the CAPTURE information for analyzing the redo log file of CYCLONE operated as master, and it is not the information reflected by APPLIER in CYCLONE SLAVE.

Executing and Monitoring

Executing Option

Executing option

Option

Description

Remarks

--conf | -c

It sets the configuration file path.

It is input in --conf CONFIG FILE format.

  • It should set the file as same as the configuration file used when running CYCLONE as master.

--start | -s

It executes CYMON.

-

--stop | -t

It terminates CYMON.

-

--status | -u

It displays the operating status of CYMON.

-

--cycle | -y

It sets the update cycle of monitoring information.

It is input in --cycle X format.

  • It should be entered in seconds.

--key | -k

It sets the decryption key when USER_ENCRYPT_PW is used in config.

-

--trace | -r

It simultaneously records the monitoring information of capture on the trace log.

The trace log is recorded in $GOLDILOCKS_DATA/trc/cymon.trc.

--silent | -i

It sets not to output the message.

-

--help | -h

It outputs the help message.

-

Examples

cymon --start --cycle 1
gSQL> \set vertical on
gSQL> select * from cyclone_monitor_info;

              GROUP_NAME # GROUP1
                    TIME # 2015-01-13 17:34:53
            MASTER_STATE # READY
             SLAVE_STATE # N/A
             MASTER_PORT # 21102
                SLAVE_IP # null
        REDO_LOG_FILESEQ # 0
       REDO_LOG_BLOCKSEQ # 52392
         CAPTURE_FILESEQ # 0
        CAPTURE_BLOCKSEQ # 0
        CAPTURE_INTERVAL # 0
   CAPTURE_INTERVAL_SIZE # 0
gSQL> \set vertical on
gSQL> select * from cyclone_monitor_info;

              GROUP_NAME # GROUP1
                    TIME # 2015-01-13 17:36:17
            MASTER_STATE # RUNNING
             SLAVE_STATE # RUNNING
             MASTER_PORT # 21102
                SLAVE_IP # 127.0.0.1
        REDO_LOG_FILESEQ # 0
       REDO_LOG_BLOCKSEQ # 52811
         CAPTURE_FILESEQ # 0
        CAPTURE_BLOCKSEQ # 52811
        CAPTURE_INTERVAL # 0
   CAPTURE_INTERVAL_SIZE # 0
cymon --start --trace
GROUP_NAME         TIME                MASTER_STATE SLAVE_STATE  MASTER_PORT  SLAVE_IP       CAPTURE_FILESEQ CAPTURE_BLOCKSEQ TOTAL_TX_COUNT CAPTURE_TX_COUNT  CAPTURE_COMMIT_LSN
------------------ ------------------- ------------ ------------ ----------- --------------- --------------- ---------------- -------------- ---------------- ----------------
GROUP1             2016-11-02 15:43:03 READY        N/A                21102 null                          0                0              0                0                0
GROUP2             2016-11-02 15:43:03 READY        N/A                21103 null                          0                0              0                0                0
GROUP_NAME         TIME                MASTER_STATE SLAVE_STATE  MASTER_PORT  SLAVE_IP       CAPTURE_FILESEQ CAPTURE_BLOCKSEQ TOTAL_TX_COUNT CAPTURE_TX_COUNT  CAPTURE_COMMIT_LSN
------------------ ------------------- ------------ ------------ ----------- --------------- --------------- ---------------- -------------- ----------------  ------------------
GROUP1             2016-11-02 15:43:11 RUNNING      RUNNING            21102 192.168.0.206                 7            52779              0                0               15346
GROUP2             2016-11-02 15:43:11 READY        N/A                21103 null                          0                0              0                0                   0
$ cymon --status
======================================
|             CYMON STATUS           |
======================================
|       Service is running...        |
--------------------------------------
$ cymon --stop
stop done.