个人网站实现与设计论文,软文推广服务,江北网站建设,装修设计图网站排名【Kingbase FlySync】命令模式:安装部署同步软件#xff0c;实现Oracle到KES实现同步 一.准备工作二.环境说明三.目标四.资源虚拟机 五.实操(1).准备安装环境Orcle服务器(Oracle40)1.上传所有工具包2.操作系统配置a.增加flysync 用户、设置密码b.配置环境变量c.调整limits.con… 【Kingbase FlySync】命令模式:安装部署同步软件实现Oracle到KES实现同步 一.准备工作二.环境说明三.目标四.资源虚拟机 五.实操(1).准备安装环境Orcle服务器(Oracle40)1.上传所有工具包2.操作系统配置a.增加flysync 用户、设置密码b.配置环境变量c.调整limits.confd.hosts文件e.配置hostnamef.授权flysync访问redog.安装ruby工具 3.数据库配置a.启动ORACLE监听b.启动ORACLE服务并将服务注册到监听程序c.创建flysync 用户并授权d.验证flysync 用户权限e.oracle 密码过期处理f.开启归档、开启补全日志 (建议在 MOUNT 模式下执行)。g.再次查看数据库归档情况、检查补全日志的状态。 4.环境检查 KingbaseES服务器(node2)1.上传所有工具包2.操作系统配置a.增加flysync 用户、设置密码b.调整limits.confc.hosts文件d.配置hostnamee.安装ruby工具 3.数据库配置a.创建flysync用户 4.环境检查 (2).部署同步程序Oracle端Oracle40部署抽取程序1.解压安装包并拷贝license2.静默安装a.创建Kingbase FlySync安装配置文件(flysync.ini)b.配置flysync.ini文件c.执行安装d.更换licensef.生效环境变量g.配置源端DML过滤策略h.启动同步程序并置于offline 状态i.查看同步软件的运行状态 KES端(node2)部署同步程序1.解压安装包并拷贝license2.静默安装a.配置驱动包b.创建Kingbase FlySync安装配置文件(flysync.ini)c.配置flysync.ini文件d.执行安装e.更换licensef.生效环境变量g.配置DML映射策略g1.查看映射功能是否开启g2.配置rename.csv h.启动同步程序并置于offline 状态h.查看同步软件的运行状态 (3).KFS存量数据搬迁1.使用ddlscan进行数据库结构搬迁2.在node2上创建oa11g0017模式3.用ddlscan极速模式/平滑模式目标端流水线迁移结构数据。4.用loader极速模式进行数据搬迁 (4).增量数据同步1.将两台服务器同步服务设置为ONLINE开始同步数据a.Oracle端(Oracle40)b.KES端(node2) 2.同步进程状态查看同步是否正常a.Oracle端(Oracle40)b.KES端(node2) 3.数据同步验证(验证DML)a.在源端(Oracle)中插入数据b.查看源端kufl 是否解析。c.在目标端验证kufl是否解析d.登录到KES数据中查询数据是否存在 配置文件服务名错误解决办法 一.准备工作
Oracle40虚拟机还原至【优化数据库】场景。node2虚拟机还原至【优化数据库】场景。
二.环境说明
两台虚拟机已安装好数据库
虚拟机IP数据库软件角色操作系统版本Oracle40192.168.40.40Oracle11g主库CenterOS7.2node2192.168.40.112KingbaseES V8R6备库CenterOS7.2
系统环境详细信息
类别源端目标端IP192.168.40.40192.168.40.112主机名Oracle40node2os用户1/密码root/kingbaseroot/kingbase.123os用户2/密码oracle/oraclekingbase/kingbase
数据库环境详细信息
类别源端目标端数据库软件Oracle11gKingbaseES V8R6数据库名orcltest数据库端口号152154321数据库用户sys/oraclesystem/kingbase
三.目标
类别源端目标端replicater安装路径/home/flysync/replicator/home/flysync/replicatorreplicator rmi-port端口1100011000replicator kufl-port端口31123112kfs服务名oracle_sourcekingbase8_targetKFS 同步所使用的用户/密码FLYSYNC/flysyncflysync/flysync
四.资源
虚拟机
下载地址百度网盘
五.实操
(1).准备安装环境
Orcle服务器(Oracle40)
1.上传所有工具包
[rootoracle40 ~]# mkdir /install
[rootoracle40 install]# ls -l
total 146552
-rw-r--r-- 1 root root 143069019 Nov 17 21:39 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
-rw-r--r-- 1 root root 1396 Nov 17 21:39 license.dat
-rw-r--r-- 1 root root 6993781 Nov 17 21:38 rbenv_2.2.2_linux_x64.zip
[rootoracle40 install]# chown -R flysync:flysync /install
[rootoracle40 install]# 2.操作系统配置
a.增加flysync 用户、设置密码
[rootoracle40 ~]# adduser flysync
[rootoracle40 ~]# usermod -G oinstall,dba flysync
[rootoracle40 ~]# echo flysync | passwd --stdin flysync
Changing password for user flysync.
passwd: all authentication tokens updated successfully.
[rootoracle40 ~]# b.配置环境变量 增加内容 export ORACLE_SIDorcl export ORACLE_HOME/home/oracle/app/oracle/product/11.2.0/dbhome_1 [rootoracle40 ~]# vi /home/flysync/.bash_profile
[rootoracle40 ~]# cat /home/flysync/.bash_profile
# .bash_profile# Get the aliases and functions
if [ -f ~/.bashrc ]; then. ~/.bashrc
fi# User specific environment and startup programsPATH$PATH:$HOME/.local/bin:$HOME/bin
export ORACLE_SIDorcl
export ORACLE_HOME/home/oracle/app/oracle/product/11.2.0/dbhome_1
export PATH
[rootoracle40 ~]# source /home/flysync/.bash_profile
[rootoracle40 ~]# c.调整limits.conf limits.conf 最大文件句柄数open files、允许创建的最大进程数量max user processes 新增内容 flysync - nofile 65535 flysync - nproc 8096 [rootoracle40 ~]# vi /etc/security/limits.conf
[rootoracle40 ~]# grep flysync /etc/security/limits.conf
flysync - nofile 65535
flysync - nproc 8096
[rootoracle40 ~]# d.hosts文件 新增内容 192.168.40.112 node2 [rootoracle40 ~]# vi /etc/hosts
[rootoracle40 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.40.40 oracle40
192.168.40.112 node2
[rootoracle40 ~]# e.配置hostname /etc/sysconfig/network内容改为 NETWORKING yes HOSTNAME oracle40 [rootoracle40 ~]# vi /etc/sysconfig/network
[rootoracle40 ~]# cat /etc/sysconfig/network
NETWORKING yes
HOSTNAME oracle40
[rootoracle40 ~]# f.授权flysync访问redo
[rootoracle40 ~]# chmod gr,gx /home/oracle
[rootoracle40 ~]#g.安装ruby工具
[rootoracle40 ~]# su - flysync
Last login: Fri Nov 17 21:45:45 CST 2023 on pts/0
[flysyncoracle40 ~]$ unzip /install/rbenv_2.2.2_linux_x64.zip -d ~/
[flysyncoracle40 ~]$ ls -l
total 4
-rwxr-xr-x 1 flysync flysync 340 Nov 4 2019 fspm
drwxrwxr-x 3 flysync flysync 17 Nov 4 2019 rbenv
[flysyncoracle40 ~]$ vi ~/.bash_profile
[flysyncoracle40 ~]$ cat ~/.bash_profile
# .bash_profile# Get the aliases and functions
if [ -f ~/.bashrc ]; then. ~/.bashrc
fi# User specific environment and startup programsPATH$PATH:$HOME/.local/bin:$HOME/bin
export ORACLE_SIDorcl
export ORACLE_HOME/home/oracle/app/oracle/product/11.2.0/dbhome_1
export RUBY_HOME/home/flysync/rbenv/ruby
export PATH$RUBY_HOME/bin:$PATH
export PATH
[flysyncoracle40 ~]$ source ~/.bash_profile
[flysyncoracle40 ~]$ ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysyncoracle40 ~]$ ~/.bash_profile文件增加内容 export RUBY_HOME/home/flysync/rbenv/ruby export PATH$RUBY_HOME/bin:$PATH 3.数据库配置
a.启动ORACLE监听
[flysyncoracle40 ~]$ exit
logout
[rootoracle40 ~]# su - oracle
Last login: Thu Apr 13 09:53:52 CST 2023 on pts/0
[oracleoracle40 ~]$ lsnrctl startLSNRCTL for Linux: Version 11.2.0.4.0 - Production on 17-NOV-2023 21:57:10Copyright (c) 1991, 2013, Oracle. All rights reserved.Starting /home/oracle/app/oracle/product/11.2.0/dbhome_1/bin/tnslsnr: please wait...TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Log messages written to /home/oracle/app/oracle/product/11.2.0/dbhome_1/log/diag/tnslsnr/oracle40/listener/alert/log.xml
Listening on: (DESCRIPTION(ADDRESS(PROTOCOLtcp)(HOSToracle40)(PORT1521)))Connecting to (ADDRESS(PROTOCOLtcp)(HOST)(PORT1521))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Start Date 17-NOV-2023 21:57:10
Uptime 0 days 0 hr. 0 min. 0 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Log File /home/oracle/app/oracle/product/11.2.0/dbhome_1/log/diag/tnslsnr/oracle40/listener/alert/log.xml
Listening Endpoints Summary...(DESCRIPTION(ADDRESS(PROTOCOLtcp)(HOSToracle40)(PORT1521)))
The listener supports no services
The command completed successfully
[oracleoracle40 ~]$ b.启动ORACLE服务并将服务注册到监听程序
[oracleoracle40 ~]$ sqlplus / as sysdbaSQL*Plus: Release 11.2.0.4.0 Production on Fri Nov 17 22:00:48 2023Copyright (c) 1982, 2013, Oracle. All rights reserved.Connected to an idle instance.SQL startup open
ORACLE instance started.Total System Global Area 789172224 bytes
Fixed Size 2257392 bytes
Variable Size 515902992 bytes
Database Buffers 264241152 bytes
Redo Buffers 6770688 bytes
Database mounted.
Database opened.
SQL alter system register;System altered.SQL c.创建flysync 用户并授权 SQL CREATE USER FLYSYNC IDENTIFIED BY flysync;
User created.
SQL GRANT CONNECT TO flysync;
Grant succeeded.
SQL GRANT CREATE SESSION TO flysync;
Grant succeeded.
SQL GRANT UNLIMITED TABLESPACE TO flysync;
Grant succeeded.
SQL GRANT CREATE TABLE TO flysync;
Grant succeeded.
SQL GRANT EXECUTE_CATALOG_ROLE TO flysync;
Grant succeeded.
SQL GRANT SELECT ANY DICTIONARY TO flysync;
Grant succeeded.
SQL GRANT SELECT ON V_$PARAMETER TO flysync;
Grant succeeded.
SQL GRANT SELECT ANY TABLE TO flysync;
Grant succeeded.
SQL GRANT SELECT ANY TRANSACTION TO flysync;
Grant succeeded.
SQL d.验证flysync 用户权限
SQL exit
[oracleoracle40 ~]$ cp /install/privs.sql ~/
[oracleoracle40 ~]$ sqlplus / as sysdbaSQL*Plus: Release 11.2.0.4.0 Production on Fri Nov 17 22:14:22 2023Copyright (c) 1982, 2013, Oracle. All rights reserved.Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing optionsSQL privs.sql FLYSYNCTYPE GRANTEE PRIVILEGE AD TABLE_NAME COLUMN_NAME OWNER
------ --------------- ------------------------------ --- ------------------------------ --------------- --------------------
ROLE FLYSYNC CONNECT NO / / /
ROLE FLYSYNC EXECUTE_CATALOG_ROLE NO / / /
SYSTEM FLYSYNC CREATE SESSION NO / / /
SYSTEM FLYSYNC CREATE TABLE NO / / /
SYSTEM FLYSYNC SELECT ANY DICTIONARY NO / / /
SYSTEM FLYSYNC SELECT ANY TABLE NO / / /
SYSTEM FLYSYNC SELECT ANY TRANSACTION NO / / /
SYSTEM FLYSYNC UNLIMITED TABLESPACE NO / / /
TABLE FLYSYNC SELECT NO V_$PARAMETER / SYS
9 rows selected.
SQL
e.oracle 密码过期处理
SQL ALTER USER OA11G identified by oa11g ACCOUNT UNLOCK;
User altered.f.开启归档、开启补全日志 (建议在 MOUNT 模式下执行)。
SQL SHUTDOWN immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL STARTUP MOUNT;
ORACLE instance started.
Total System Global Area 789172224 bytes
Fixed Size 2257392 bytes
Variable Size 515902992 bytes
Database Buffers 264241152 bytes
Redo Buffers 6770688 bytes
Database mounted.
SQL ALTER DATABASE archivelog;
Database altered.
SQL ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
Database altered.
SQL ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
Database altered.
SQL ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
Database altered.
SQL ALTER DATABASE OPEN;
Database altered.
SQL
g.再次查看数据库归档情况、检查补全日志的状态。
SQL SELECT log_mode FROM v$database;
LOG_MODE
------------
ARCHIVELOG
SQL SELECT supplemental_log_data_min, supplemental_log_data_pk,supplemental_log_data_all FROM v$database; 2
SUPPLEME SUP SUP
-------- --- ---
YES YES YES
SQL 4.环境检查
SQL exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
[oracleoracle40 ~]$ exit
logout
[rootoracle40 ~]# su - flysync
Last login: Fri Nov 17 21:49:13 CST 2023 on pts/0
[flysyncoracle40 ~]$ uname -n
oracle40
[flysyncoracle40 ~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 11210
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 8096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[flysyncoracle40 ~]$ hostname --ip-address
192.168.40.40
[flysyncoracle40 ~]$ ping -c 4 oracle40
PING oracle40 (192.168.40.40) 56(84) bytes of data.
64 bytes from oracle40 (192.168.40.40): icmp_seq1 ttl64 time0.035 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq2 ttl64 time0.046 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq3 ttl64 time0.037 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq4 ttl64 time0.044 ms--- oracle40 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev 0.035/0.040/0.046/0.007 ms
[flysyncoracle40 ~]$ ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq1 ttl64 time1.60 ms
64 bytes from node2 (192.168.40.112): icmp_seq2 ttl64 time0.320 ms
64 bytes from node2 (192.168.40.112): icmp_seq3 ttl64 time0.253 ms
64 bytes from node2 (192.168.40.112): icmp_seq4 ttl64 time0.227 ms--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev 0.227/0.602/1.609/0.582 ms
[rootoracle40 ~]# date
Fri Nov 17 22:24:39 CST 2023
[rootoracle40 ~]# ssh rootnode2 date
The authenticity of host node2 (192.168.40.112) cant be established.
ECDSA key fingerprint is 03:d4:89:aa:aa:7c:e5:b4:c2:ca:09:8f:c7:63:19:a3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added node2,192.168.40.112 (ECDSA) to the list of known hosts.
rootnode2s password:
Fri Nov 17 22:25:00 CST 2023
[rootoracle40 ~]# KingbaseES服务器(node2)
1.上传所有工具包
[rootnode2 install]# ls -l
总用量 998884
-rw-r--r-- 1 root root 872781824 4月 9 2023 KingbaseES_V008R006C005B0023_Lin64_single_install.iso
-rw-r--r-- 1 root root 143069019 11月 17 22:34 KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz
-rw-r--r-- 1 root root 3351 4月 9 2023 license_12350_0_dev.dat
-rw-r--r-- 1 root root 1396 11月 17 22:34 license.dat
-rw-r--r-- 1 root root 6993781 11月 17 22:34 rbenv_2.2.2_linux_x64.zip
drwxrwxr-x 2 kingbase kingbase 6 4月 9 2023 scripts
[rootnode2 install]# chown -R flysync:flysync /install
[rootnode2 install]# 2.操作系统配置
a.增加flysync 用户、设置密码
[rootnode2 install]# adduser flysync
[rootnode2 install]# usermod -G kingbase flysync
[rootnode2 install]# echo flysync |passwd --stdin flysync
更改用户 flysync 的密码 。
passwd所有的身份验证令牌已经成功更新。
[rootnode2 install]#
b.调整limits.conf limits.conf 最大文件句柄数open files、允许创建的最大进程数量max user processes 新增内容 flysync - nofile 65535 flysync - nproc 8096 [rootnode2 install]# vi /etc/security/limits.conf
[rootnode2 install]# grep flysync /etc/security/limits.conf
flysync - nofile 65535
flysync - nproc 8096
[rootnode2 install]# c.hosts文件 新增内容 192.168.40.40 oracle40 192.168.40.112 node2 [rootnode2 install]# vi /etc/hosts
[rootnode2 install]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6192.168.40.40 oracle40
192.168.40.112 node2
[rootnode2 install]#
d.配置hostname /etc/sysconfig/network内容改为 NETWORKING yes HOSTNAME node2 [rootnode2 install]# vi /etc/sysconfig/network
[rootnode2 install]# cat /etc/sysconfig/network
NETWORKING yes
HOSTNAME node2
[rootnode2 install]# e.安装ruby工具
[rootnode2 ~]# su - flysync
Last login: Fri Nov 17 21:45:45 CST 2023 on pts/0
[flysyncnode2 ~]$ unzip /install/rbenv_2.2.2_linux_x64.zip -d ~/
[flysyncnode2 ~]$ ls -l
总用量 4
-rwxr-xr-x 1 flysync flysync 340 11月 4 2019 fspm
drwxrwxr-x 3 flysync flysync 17 11月 4 2019 rbenv
[flysyncnode2 ~]$
[flysyncnode2 ~]$ vi ~/.bash_profile
[flysyncnode2 ~]$ cat ~/.bash_profile
# .bash_profile# Get the aliases and functions
if [ -f ~/.bashrc ]; then. ~/.bashrc
fi# User specific environment and startup programsPATH$PATH:$HOME/.local/bin:$HOME/bin
export RUBY_HOME/home/flysync/rbenv/ruby
export PATH$RUBY_HOME/bin:$PATH
export PATH
[flysyncnode2 ~]$ source ~/.bash_profile
[flysyncnode2 ~]$ ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
[flysyncnode2 ~]$ ~/.bash_profile文件增加内容 export RUBY_HOME/home/flysync/rbenv/ruby export PATH$RUBY_HOME/bin:$PATH 3.数据库配置
a.创建flysync用户
[flysyncnode2 ~]$ exit
登出
[rootnode2 install]# su - kingbase
上一次登录五 11月 17 21:19:58 CST 2023
[kingbasenode2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbasenode2 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 help 来获取帮助信息. test# CREATE USER flysync PASSWORD flysync;
CREATE ROLE
test# GRANT ALL PRIVILEGES ON DATABASE test TO flysync ;
GRANT
test# ALTER USER flysync SUPERUSER;
ALTER ROLE
test#
4.环境检查
test# \q
[kingbasenode2 bin]$ exit
登出
[rootnode2 install]# uname -n
node2
[rootnode2 install]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 9838
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 9838
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[rootnode2 install]# hostname --ip-address
192.168.40.112
[rootnode2 install]# ping -c 4 node2
PING node2 (192.168.40.112) 56(84) bytes of data.
64 bytes from node2 (192.168.40.112): icmp_seq1 ttl64 time0.031 ms
64 bytes from node2 (192.168.40.112): icmp_seq2 ttl64 time0.045 ms
64 bytes from node2 (192.168.40.112): icmp_seq3 ttl64 time0.037 ms
64 bytes from node2 (192.168.40.112): icmp_seq4 ttl64 time0.036 ms--- node2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev 0.031/0.037/0.045/0.006 ms
[rootnode2 install]# ping -c 4 oracle40
PING oracle40 (192.168.40.40) 56(84) bytes of data.
64 bytes from oracle40 (192.168.40.40): icmp_seq1 ttl64 time1.24 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq2 ttl64 time0.298 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq3 ttl64 time0.242 ms
64 bytes from oracle40 (192.168.40.40): icmp_seq4 ttl64 time0.281 ms--- oracle40 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev 0.242/0.517/1.248/0.422 ms
[rootnode2 install]# date
2023年 11月 18日 星期六 00:19:27 CST
[rootnode2 install]# (2).部署同步程序
Oracle端Oracle40部署抽取程序
1.解压安装包并拷贝license
[rootoracle40 ~]# su - flysync
Last login: Fri Nov 17 22:20:35 CST 2023 on pts/0
[flysyncoracle40 ~]$ mkdir /home/flysync/install
[flysyncoracle40 ~]$ cd /home/flysync/install/
[flysyncoracle40 install]$ tar -zxf /install/KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz -C .
[flysyncoracle40 install]$ ls -l
total 0
drwxrwxr-x 6 flysync flysync 116 Mar 20 2023 KingbaseFlySync-V002R002C004B20230320-replicator
[flysyncoracle40 install]$ cp /install/license.dat .
[flysyncoracle40 install]$ 2.静默安装
a.创建Kingbase FlySync安装配置文件(flysync.ini)
[flysyncoracle40 install]$ cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysyncoracle40 ini]$ cp flysync_oracle-logminer_source.ini ~/flysync.ini
[flysyncoracle40 ini]$ b.配置flysync.ini文件
[flysyncoracle40 ini]$ vi ~/flysync.ini
[flysyncoracle40 ini]$ cat ~/flysync.ini
[defaults]
# 安装目录
install-directory~/replicator
# 环境变量脚本位置
profile-script~/.bash_profile
# rmi端口,默认11000
rmi-port11000# 服务名
[oracle_source]
# 监听kufl获取的端口
kufl-port3112
# master主机名
masteroracle40
# 当前主机名
membersoracle40# 数据源类型
datasource-typeoracle
# 同步数据库信息
replication-hostoracle40
replication-userFLYSYNC
replication-passwordflysync
replication-port1521# 同步服务角色
rolemaster## oracle 指定连接的数据库名称
datasource-oracle-serviceorcl
# oracle数据库需指定此项解析增量方式logminer
#oracle-extractor-methodlogminer
oracle-extractor-methodredo## oracle-LOGMINER 同步表信息、scn号差值的阈值
propertyreplicator.extractor.dbms.scnSegmentationThreshold5000## 过滤器开关配置
## dropstatementdata 不开启DDL同步
# svc_extractor_filtersdropstatementdata,replicate,casetransform,dropcolumn,rename
svc_extractor_filtersreplicate
## 集群参数
# propertyreplicator.datasource.global.connectionSpec.use_clustertrue
# propertyreplicator.datasource.global.connectionSpec.slave_add192.168.28.176,192.168.28.176
# propertyreplicator.datasource.global.connectionSpec.slave_port1521,1521### 过滤器详细配置
## replicate 表过滤
propertyreplicator.extractor.dbms.tablePatternsFLYSYNC.*,OA11G.*
propertyreplicator.filter.replicate.ignoreOA11G.tmpbak*
# propertyreplicator.filter.replicate.doPUBLIC.*
# propertyreplicator.filter.replicate.doPUBLIC.*,OA11G.*
# propertyreplicator.filter.replicate.ignorePUBLIC.T1
## casetransform 大小写转换
# propertyreplicator.filter.casetransform.to_upper_casetrue## 优化参数
# 达到多少条记录后分事务默认 500
propertyreplicator.extractor.dbms.maxRowsByBlock500
# 若多少秒没有新的事务产生那么就直接返回该事务不再等待合并
propertyreplicator.extractor.dbms.maxWaitingTime2
# 大事务优化参数
propertyreplicator.extractor.dbms.minRowsPerBlock4000# 自动恢复尝试次数系统 OFFLINE 时尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts0
# 系统 OFFLINE 时尝试 ONLINE 之前的等待时间s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval30s
# 系统自动恢复次数重置时间s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval300s
[flysyncoracle40 ini]$
c.执行安装
[flysyncoracle40 ini]$ cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
.
WARN oracle40 ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN oracle40 Linux swappiness is currently set to 30, on restart it will be 0, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
.License : /home/flysync/replicator/license.dat#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the
cluster will be available./home/flysync/replicator/flysync/cluster-home/bin/startallWait a minute for the services to start up and configure themselves. After
that you may proceed.We have added FlySync environment variables to ~/.bash_profile.
Run source ~/.bash_profile to rebuild your environment.Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server./home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl servicesConfiguration is now complete. For further information, please consult
FlySync documentation.NOTE Command successfully completed
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ d.更换license
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]cp ~/install/license.dat ~/replicator/license.dat
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls ~/replicator/
backups filters-config flysync kufl license.dat metadata relay releases service_logs share
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$f.生效环境变量
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ env | grep flysync
OLDPWD/home/flysync/install/KingbaseFlySync-V002R002C004B20230320-replicator
USERflysync
KINGBASE_ROOT/home/flysync/replicator
MAIL/var/spool/mail/flysync
PATH/home/flysync/rbenv/ruby/bin:/home/flysync/rbenv/ruby/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/flysync/.local/bin:/home/flysync/bin:/home/flysync/.local/bin:/home/flysync/bin:/home/flysync/replicator/flysync/flysync-manager/bin:/home/flysync/replicator/flysync/flysync-replicator/bin:/home/flysync/replicator/flysync/cluster-home/bin:/home/flysync/replicator/flysync/flysync-connector/bin:/home/flysync/replicator/share:/home/flysync/replicator/flysync/tools:/home/flysync/replicator/flysync/console/bin
PWD/home/flysync/install/KingbaseFlySync-V002R002C004B20230320-replicator
RUBY_HOME/home/flysync/rbenv/ruby
HOME/home/flysync
LOGNAMEflysync
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ g.配置源端DML过滤策略
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/flysync.ini
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ cat ~/flysync.ini | grep -E svc_extractor_filters|replicate.do
# svc_extractor_filtersdropstatementdata,replicate,casetransform,dropcolumn,rename
svc_extractor_filtersreplicate
# propertyreplicator.filter.replicate.doPUBLIC.*
# propertyreplicator.filter.replicate.doPUBLIC.*,OA11G.*
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ h.启动同步程序并置于offline 状态
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:28852
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ i.查看同步软件的运行状态
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service oracle_source services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : master
serviceName : oracle_source
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysyncoracle40 KingbaseFlySync-V002R002C004B20230320-replicator]$KES端(node2)部署同步程序
1.解压安装包并拷贝license
[rootnode2 ~]# su - flysync
上一次登录五 11月 17 22:50:12 CST 2023pts/0 上
[flysyncnode2 ~]$ mkdir /home/flysync/install
[flysyncnode2 ~]$ cd /home/flysync/install/
[flysyncnode2 install]$ tar -zxf /install/KingbaseFlySync-V002R002C004B20230320-replicator.tar.gz -C .
[flysyncnode2 install]$ ls -l
总用量 0
drwxrwxr-x 6 flysync flysync 116 3月 20 2023 KingbaseFlySync-V002R002C004B20230320-replicator
[flysyncnode2 install]$ cp /install/license.dat .
[flysyncnode2 install]$ 2.静默安装
a.配置驱动包
[flysyncnode2 install]$ cd KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/lib
[flysyncnode2 lib]$ cp ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/extensions/jdbc/kingbase8-8.6.0.jar .
[flysyncnode2 lib]$ ls -l kingbase8-8.6.0.jar
-rw-rw-r-- 1 flysync flysync 1019069 11月 18 11:33 kingbase8-8.6.0.jar
[flysyncnode2 lib]$b.创建Kingbase FlySync安装配置文件(flysync.ini) [flysyncnode2 ini]$ cp flysync_kingbase8_target.ini ~/flysync.ini
[flysyncnode2 ini]$c.配置flysync.ini文件
[flysyncnode2 ini]$cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini/
[flysyncnode2 ini]$ vi ~/flysync.ini
[flysyncnode2 ini]$ cat ~/flysync.ini
[defaults]
# 安装目录
install-directory~/replicator
# 环境变量脚本位置
profile-script~/.bash_profile
# rmi端口,默认11000
rmi-port11000# 服务名
[kingbase8_target]
# 监听kufl获取的端口
kufl-port3112
# master主机名
masteroracle40
# 当前主机名
membersnode2##指定连接master监听的kufl端口
master-kufl-port3112# 数据源类型
datasource-typekingbase
# 同步数据库信息
replication-hostnode2
replication-userflysync
replication-passwordflysync
replication-port54321# 同步服务角色
roleslave## 指定连接的数据库名称
kingbase-dbnametest## kingbase 版本号
datasource-version8## 过滤器开关配置
svc-remote-filtersrename
# svc-remote-filterscasetransform,dropcolumn,rename## 集群参数
# propertyreplicator.datasource.global.connectionSpec.use_clustertrue
# propertyreplicator.datasource.global.connectionSpec.slave_add192.168.28.176,192.168.28.176
# propertyreplicator.datasource.global.connectionSpec.slave_port1521,1521### 过滤器详细配置
## replicate 表过滤
# propertyreplicator.filter.replicate.doPUBLIC.*
# propertyreplicator.filter.replicate.ignorePUBLIC.T1
## casetransform 大小写转换
# propertyreplicator.filter.casetransform.to_upper_casetrue## 优化参数
# 开启小事务合并入库
propertyreplicator.applier.dbms.optimizeRowEventstrue
# 指定单表一次入库的数据量
propertyreplicator.applier.dbms.maxRowBatchSize5000# 自动恢复尝试次数系统 OFFLINE 时尝试将系统重新置于ONLINE状态的次数
repl_auto_recovery_max_attempts0
# 系统 OFFLINE 时尝试 ONLINE 之前的等待时间s(秒)、m(分钟)、h(小时)
repl_auto_recovery_delay_interval30s
# 系统自动恢复次数重置时间s(秒)、m(分钟)、h(小时)
repl_auto_recovery_reset_interval300s
[flysyncnode2 ini]$
d.执行安装
[flysyncnode2 ini]$ cd ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ./tools/fspm install
..
WARN node2 ntpd is not running. It is important that configured hosts have time synchronised. (NtpdRunningCheck)
WARN node2 Linux swappiness is currently set to 30, on restart it will be 0, consider setting this to 10 or under to avoid swapping. (SwappinessCheck)
..License : /home/flysync/replicator/license.dat#####################################################################
# Next Steps
#####################################################################
Unless automatically started, you must start the FlySync services before the
cluster will be available./home/flysync/replicator/flysync/cluster-home/bin/startallWait a minute for the services to start up and configure themselves. After
that you may proceed.We have added FlySync environment variables to ~/.bash_profile.
Run source ~/.bash_profile to rebuild your environment.Once your services start successfully replication will begin.
To look at services and perform administration, run the following command
from any database server./home/flysync/replicator/flysync/flysync-replicator/bin/fsrepctl servicesConfiguration is now complete. For further information, please consult
FlySync documentation.NOTE Command successfully completed
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ e.更换license
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cp ~/install/license.dat ~/replicator/license.dat
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ls ~/replicator/
backups filters-config flysync kufl license.dat metadata relay releases service_logs share
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ f.生效环境变量
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ source ~/.bash_profile
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ env | grep flysync
OLDPWD/home/flysync/install/KingbaseFlySync-V002R002C004B20230320-replicator/flysync-replicator/samples/ini
USERflysync
KINGBASE_ROOT/home/flysync/replicator
MAIL/var/spool/mail/flysync
PATH/home/flysync/rbenv/ruby/bin:/home/flysync/rbenv/ruby/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/flysync/.local/bin:/home/flysync/bin:/home/flysync/.local/bin:/home/flysync/bin:/home/flysync/replicator/flysync/flysync-manager/bin:/home/flysync/replicator/flysync/flysync-replicator/bin:/home/flysync/replicator/flysync/cluster-home/bin:/home/flysync/replicator/flysync/flysync-connector/bin:/home/flysync/replicator/share:/home/flysync/replicator/flysync/tools:/home/flysync/replicator/flysync/console/bin
PWD/home/flysync/install/KingbaseFlySync-V002R002C004B20230320-replicator
RUBY_HOME/home/flysync/rbenv/ruby
HOME/home/flysync
LOGNAMEflysync
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$
g.配置DML映射策略
g1.查看映射功能是否开启
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ cat ~/flysync.ini | grep -E svc-remote-filters
svc-remote-filtersrename
# svc-remote-filterscasetransform,dropcolumn,rename
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ svc-remote-filtersrename 没被注释证明映射开启了 g2.配置rename.csv
将OA11G模式下的增量数据映射到oa11g0017下。将OA11G.EMP 表的增量数据映射到 oa11g0017.emp0017 下
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ vi ~/replicator/filters-config/rename.csv
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ tail -8 ~/replicator/filters-config/rename.csvOA11G,*,*,oa11g0017,-,-
OA11G,EMP,*,oa11g0017,emp0017,-
FLYSYNC,trep_commit_seqno,*,flysync_kingbase8_target,-,-
FLYSYNC,consistency,*,flysync_kingbase8_target,-,-
FLYSYNC,heartbeat,*,flysync_kingbase8_target,-,-
FLYSYNC,trep_shard,*,flysync_kingbase8_target,-,-
FLYSYNC,trep_shard_channel,*,flysync_kingbase8_target,-,-
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$
h.启动同步程序并置于offline 状态
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ replicator start offline
Starting FlySync Replicator Service...
Waiting for FlySync Replicator Service......
running: PID:7693
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$
h.查看同步软件的运行状态
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : slave
serviceName : kingbase_target
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ (3).KFS存量数据搬迁
1.使用ddlscan进行数据库结构搬迁 检查replicator状态。确保同步进程处于【OFFLINENORMAL】状态。 Oracle端(oracle40)
[flysyncoracle40 ~]$ fsrepctl -service oracle_soure services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : master
serviceName : oracle_source
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysyncoracle40 ~]$
KES端(node2)
[flysyncnode2 ~]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : slave
serviceName : kingbase_target
serviceType : local
started : true
state : OFFLINE:NORMAL
Finished services command...
[flysyncnode2 ~]$
2.在node2上创建oa11g0017模式
[rootnode2 ~]# su - kingbase
上一次登录六 11月 18 09:34:24 CST 2023
[kingbasenode2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbasenode2 bin]$ ./ksql -U flysync -d test
ksql (V8.0)
输入 help 来获取帮助信息.test# CREATE SCHEMA oa11g0017;
CREATE SCHEMA
test# 3.用ddlscan极速模式/平滑模式目标端流水线迁移结构数据。
使用命令
ddlscan -target.service kingbase8_target -source.user FLYSYNC -source.pass flysync -source.db orcl -source.dbtype oracle -source.host oracle40 -source.port 1521 -source.schema OA11G -target.db test -mgType 0结果如下
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ ddlscan -target.service kingbase8_target -source.user FLYSYNC -source.pass flysync -source.db orcl -source.dbtype oracle -source.host oracle40 -source.port 1521 -source.schema OA11G -target.db test -mgType 0| [ - main] INFO ddlscan.DDLScanWorker Tables or TableFile not specified - extracting everything!| [ - main] INFO ddlscan.DDLScanWorker Connect to source database with url: jdbc:oracle:thin://oracle40:1521/orcl| [ - main] INFO database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver| [ - main] INFO database.AbstractDatabase Loading database driver: com.kingbase8.Driver| [ - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [ - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [ - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [ - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [ - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [ - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [ - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [ - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [ - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [ - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [ - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [ - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [ - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [ - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [ - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [ - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [ - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [ - main] INFO filter.RenameDefinitions OA11G.*.* - oa11g0017.-.-| [ - main] INFO filter.RenameDefinitions OA11G.EMP.* - oa11g0017.emp0017.-| [ - main] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [ - main] INFO filter.RenameDefinitions FLYSYNC.consistency.* - flysync_kingbase8_target.-.-| [ - main] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* - flysync_kingbase8_target.-.-| [ - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* - flysync_kingbase8_target.-.-| [ - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* - flysync_kingbase8_target.-.-| [ - main] INFO ddlscan.DDLScanWorker There is 12 tables to migration.| [ - main] INFO ddlscan.TargetDDLScanWorker Connect to target database with url: jdbc:kingbase8://node2:54321/test| [ - main] INFO ddlscan.TargetDDLScanWorker Drop tables on target database.| [ - main] INFO ddlscan.DDLScanCtrl
--------------------------------------------------------------------------------
TOTAL: 12
SUCCESS: 12
FAILURE: 0
TOTAL TIME: 2s
FINISHED AT: 2023-11-18 12:50:06
OUT: /tmp/ddlscan_20231118_125004.sql
ERROR:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|NUM |SOURCE_TABLE |TARGET_TABLE |STATUS |MESSAGE |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|1 |OA11G.COUNTRIES |oa11g0017.COUNTRIES |Success | |
|2 |OA11G.DEPARTMENTS |oa11g0017.DEPARTMENTS |Success | |
|3 |OA11G.DEPT |oa11g0017.DEPT |Success | |
|4 |OA11G.EMP |oa11g0017.emp0017 |Success | |
|5 |OA11G.EMPLOYEES |oa11g0017.EMPLOYEES |Success | |
|6 |OA11G.JOBS |oa11g0017.JOBS |Success | |
|7 |OA11G.JOB_HISTORY |oa11g0017.JOB_HISTORY |Success | |
|8 |OA11G.LOCATIONS |oa11g0017.LOCATIONS |Success | |
|9 |OA11G.MLOG$_EMP |oa11g0017.MLOG$_EMP |Success | |
|10 |OA11G.REGIONS |oa11g0017.REGIONS |Success | |
|11 |OA11G.RUPD$_EMP |oa11g0017.RUPD$_EMP |Success | |
|12 |OA11G.SALGRADE |oa11g0017.SALGRADE |Success | |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$
4.用loader极速模式进行数据搬迁
使用命令 loader -source.service oracle_source -source.user FLYSYNC -source.pass flysync \
-source.db orcl -source.dbtype oracle -source.host oracle40 \
-source.port 1521 -source.schema OA11G \
-source.rmiHost oracle40 -source.rmiPort 11000 \
-target.rmiHost node2 -target.rmiPort 11000 \
-target.service kingbase8_target -mgType 0 -clean结果如下
[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$ loader -source.service oracle_source -source.user FLYSYNC -source.pass flysync \-source.db orcl -source.dbtype oracle -source.host oracle40 \-source.port 1521 -source.schema OA11G \-source.rmiHost oracle40 -source.rmiPort 11000 \-target.rmiHost node2 -target.rmiPort 11000 \-target.service kingbase8_target -mgType 0 -clean| [ - main] INFO database.AbstractDatabase Loading database driver: oracle.jdbc.driver.OracleDriver| [ - main] INFO database.AbstractDatabase Loading database driver: com.kingbase8.Driver| [ - main] INFO loader.LoaderWorker Tables not specified - extracting everything!| [ - main] INFO loader.LoaderWorker Import table OA11G.COUNTRIES| [ - main] INFO loader.LoaderWorker Import table OA11G.DEPARTMENTS| [ - main] INFO loader.LoaderWorker Import table OA11G.DEPT| [ - main] INFO loader.LoaderWorker Import table OA11G.EMP| [ - main] INFO loader.LoaderWorker Import table OA11G.EMPLOYEES| [ - main] INFO loader.LoaderWorker Import table OA11G.JOBS| [ - main] INFO loader.LoaderWorker Import table OA11G.JOB_HISTORY| [ - main] INFO loader.LoaderWorker Import table OA11G.LOCATIONS| [ - main] INFO loader.LoaderWorker Import table OA11G.MLOG$_EMP| [ - main] INFO loader.LoaderWorker Import table OA11G.REGIONS| [ - main] INFO loader.LoaderWorker Import table OA11G.RUPD$_EMP| [ - main] INFO loader.LoaderWorker Import table OA11G.SALGRADE| [ - main] INFO conf.ReplicatorRuntime Replicator role: slave| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [kingbase8_target - main] INFO pipeline.Pipeline Configuring pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameglobal classNamecom.kingbase.flysync.replicator.datasource.KingbaseDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameextractor| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameextractor classNamecom.kingbase.flysync.replicator.datasource.AliasDataSource| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: namefile_metadata| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: namefile_metadata classNamecom.kingbase.flysync.replicator.datasource.FileDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [kingbase8_target - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false| [kingbase8_target - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms| [kingbase8_target - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave| [kingbase8_target - main] INFO pipeline.Pipeline Preparing pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: namefile_metadata| [kingbase8_target - main] INFO datasource.FileDataSource Service directory does not exist, creating: /home/flysync/replicator/metadata/applier/kingbase8_target| [kingbase8_target - main] INFO datasource.FileDataSource Initializing data source files: servicekingbase8_target directory/home/flysync/replicator/metadata/applier| [kingbase8_target - main] INFO datasource.FileCommitSeqno Initializing file-based seqno tracking: directory/home/flysync/replicator/metadata/applier/kingbase8_target filecommitseqno.0| [kingbase8_target - pool-1-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave| [kingbase8_target - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files1 channels1| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: nameglobal| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.*.* - oa11g0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.EMP.* - oa11g0017.emp0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.Pipeline Starting pipeline: slave| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 13| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 13| [kingbase8_target - pool-1-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave| [kingbase8_target - main] INFO pipeline.Pipeline Shutting down pipeline: slave| [kingbase8_target - main] INFO pipeline.Pipeline Releasing pipeline: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO conf.ReplicatorRuntime Replicator role: slave| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [kingbase8_target - main] INFO pipeline.Pipeline Configuring pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameglobal classNamecom.kingbase.flysync.replicator.datasource.KingbaseDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameextractor| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameextractor classNamecom.kingbase.flysync.replicator.datasource.AliasDataSource| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: namefile_metadata| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: namefile_metadata classNamecom.kingbase.flysync.replicator.datasource.FileDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [kingbase8_target - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false| [kingbase8_target - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms| [kingbase8_target - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave| [kingbase8_target - main] INFO pipeline.Pipeline Preparing pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: namefile_metadata| [kingbase8_target - main] INFO datasource.FileDataSource Initializing data source files: servicekingbase8_target directory/home/flysync/replicator/metadata/applier| [kingbase8_target - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files1 channels1| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: nameglobal| [kingbase8_target - pool-5-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.*.* - oa11g0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.EMP.* - oa11g0017.emp0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.*.* - oa11g0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.EMP.* - oa11g0017.emp0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-1] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.*.* - oa11g0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.EMP.* - oa11g0017.emp0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-2] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.Pipeline Starting pipeline: slave| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO applier.JdbcApplier nologging: oa11g0017.COUNTRIES| [kingbase8_target - main-q-to-dbms-0] INFO applier.JdbcApplier nologging: oa11g0017.emp0017| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 13| [kingbase8_target - main-q-to-dbms-1] INFO applier.JdbcApplier nologging: oa11g0017.DEPARTMENTS| [kingbase8_target - main-q-to-dbms-1] INFO applier.JdbcApplier nologging: oa11g0017.EMPLOYEES| [kingbase8_target - main-q-to-dbms-1] INFO applier.JdbcApplier nologging: oa11g0017.LOCATIONS| [kingbase8_target - main-q-to-dbms-1] INFO applier.JdbcApplier nologging: oa11g0017.RUPD$_EMP| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno(unavailable)| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task event count: 5| [kingbase8_target - main-q-to-dbms-0] INFO applier.JdbcApplier nologging: oa11g0017.JOB_HISTORY| [kingbase8_target - main-q-to-dbms-0] INFO applier.JdbcApplier nologging: oa11g0017.REGIONS| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 5| [kingbase8_target - main-q-to-dbms-2] INFO applier.JdbcApplier nologging: oa11g0017.DEPT| [kingbase8_target - main-q-to-dbms-2] INFO applier.JdbcApplier nologging: oa11g0017.JOBS| [kingbase8_target - main-q-to-dbms-2] INFO applier.JdbcApplier nologging: oa11g0017.MLOG$_EMP| [kingbase8_target - main-q-to-dbms-2] INFO applier.JdbcApplier nologging: oa11g0017.SALGRADE| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno(unavailable)| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task event count: 5| [kingbase8_target - pool-5-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave| [kingbase8_target - main] INFO pipeline.Pipeline Shutting down pipeline: slave| [kingbase8_target - main] INFO pipeline.Pipeline Releasing pipeline: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Replicator role: slave| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Configuring pipeline: slave| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Configuring data source: nameglobal| [kingbase8_target - pipeline-0] INFO datasource.DataSourceManager Loading data source: nameglobal classNamecom.kingbase.flysync.replicator.datasource.KingbaseDataSource| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Configuring data source: nameextractor| [kingbase8_target - pipeline-0] INFO datasource.DataSourceManager Loading data source: nameextractor classNamecom.kingbase.flysync.replicator.datasource.AliasDataSource| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Configuring data source: namefile_metadata| [kingbase8_target - pipeline-0] INFO datasource.DataSourceManager Loading data source: namefile_metadata classNamecom.kingbase.flysync.replicator.datasource.FileDataSource| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - pipeline-0] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [kingbase8_target - pipeline-0] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms| [kingbase8_target - pipeline-0] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Preparing pipeline: slave| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: namefile_metadata| [kingbase8_target - pipeline-0] INFO datasource.FileDataSource Initializing data source files: servicekingbase8_target directory/home/flysync/replicator/metadata/applier| [kingbase8_target - pipeline-0] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files1 channels1| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Preparing and initializing data source: nameglobal| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: nameglobal| [kingbase8_target - pipeline-0] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: nameglobal| [kingbase8_target - pool-11-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions OA11G.*.* - oa11g0017.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions OA11G.EMP.* - oa11g0017.emp0017.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions FLYSYNC.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - pipeline-0] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - pipeline-0] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - pipeline-0] WARN pipeline.StageTaskGroup [pipeline-0-q-to-dbms-0] Last event data not available; Setting extractor to current position| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [kingbase8_target - pipeline-0] WARN pipeline.StageTaskGroup [pipeline-0-loader-to-q-0] Last event data not available; Setting extractor to current position| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Starting pipeline: slave| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.COUNTRIES:0-25| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.DEPARTMENTS:0-27| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.DEPT:0-4| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.EMP:0-14| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.EMPLOYEES:0-107| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.JOBS:0-19| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.JOB_HISTORY:0-10| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.LOCATIONS:0-23| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.MLOG$_EMP:0-0| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.REGIONS:0-4| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.RUPD$_EMP:0-0| [kingbase8_target - pipeline-0-loader] INFO loader.JdbcLoaderTask transfer data from OA11G.SALGRADE:0-5| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - pipeline-0-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 13| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - pipeline-0-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 13| [kingbase8_target - pool-11-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Shutting down pipeline: slave| [kingbase8_target - pipeline-0] INFO pipeline.Pipeline Releasing pipeline: slave| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [kingbase8_target - pipeline-0] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [kingbase8_target - pipeline-0] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO loader.TargetLoaderWorker Tables imported| [kingbase8_target - main] INFO loader.LoaderWorker | -------------------------打印执行结果开始-------------------------| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.COUNTRIES : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.JOB_HISTORY : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.MLOG$_EMP : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.DEPT : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.REGIONS : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.DEPARTMENTS : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.EMPLOYEES : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.SALGRADE : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.LOCATIONS : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.EMP : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.JOBS : success| [kingbase8_target - main] INFO loader.LoaderWorker | OA11G.RUPD$_EMP : success| [kingbase8_target - main] INFO loader.LoaderWorker | -------------------------打印执行结果结束-------------------------| [kingbase8_target - main] INFO conf.ReplicatorRuntime Replicator role: slave| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyFailureStop to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnNames to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting consistencyCheckColumnTypes to true| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting applierFailurePolicy to warn| [kingbase8_target - main] INFO conf.ReplicatorRuntime Setting replicator.applier.failOnZeroRowUpdate to warn| [kingbase8_target - main] INFO pipeline.Pipeline Configuring pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameglobal classNamecom.kingbase.flysync.replicator.datasource.KingbaseDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: nameextractor| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: nameextractor classNamecom.kingbase.flysync.replicator.datasource.AliasDataSource| [kingbase8_target - main] INFO datasource.DataSourceService Configuring data source: namefile_metadata| [kingbase8_target - main] INFO datasource.DataSourceManager Loading data source: namefile_metadata classNamecom.kingbase.flysync.replicator.datasource.FileDataSource| [kingbase8_target - main] INFO datasource.AbstractDataSource No cvsType provided; using default settings| [kingbase8_target - main] INFO datasource.AbstractDataSource Checking CSV formatter class: com.kingbase.flysync.replicator.csv.DefaultCsvDataFormat| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Configuring raw extractor and heartbeat filter| [kingbase8_target - main] INFO event.EventMetadataFilter Use default schema for unknown SQL statements: false| [kingbase8_target - main] INFO extractor.ExtractorWrapper Master auto-repositioning on source_id change is enabled; extractor will reposition current log position if last extracted source_id differs from current source_id| [kingbase8_target - main] INFO pipeline.StageTaskGroup Instantiating and configuring tasks for stage: q-to-dbms| [kingbase8_target - main] INFO conf.ReplicatorRuntime Preparing pipeline for use: slave| [kingbase8_target - main] INFO pipeline.Pipeline Preparing pipeline: slave| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: namefile_metadata| [kingbase8_target - main] INFO datasource.FileDataSource Initializing data source files: servicekingbase8_target directory/home/flysync/replicator/metadata/applier| [kingbase8_target - main] INFO datasource.FileCommitSeqno Validated that trep_commit_seqno file count matches channels: files1 channels1| [kingbase8_target - main] INFO datasource.DataSourceService Preparing and initializing data source: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Setting catalog handling for privileged slave/relay: nameglobal| [kingbase8_target - main] INFO datasource.DataSourceService Disabling logging of updates for slave/relay: nameglobal| [kingbase8_target - pool-15-thread-1] INFO pipeline.DeferredShutdownTask Waiting for pipeline to shut down: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.*.* - oa11g0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.EMP.* - oa11g0017.emp0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.*.* - oa11g0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.EMP.* - oa11g0017.emp0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-1] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO filter.RenameDefinitions Parsing /home/flysync/replicator/flysync/flysync-replicator/support/filters-config/rename.csv:| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.*.* - schemac1.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemac.tablew.* - schemac2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemav.tablee.* - schemav1.tableee.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemab.tabler.* - -.tabler1.-| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablet.* - -.tablet1.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemaz.tableq.cola - -.-.colaa| [kingbase8_target - main] INFO filter.RenameDefinitions *.tablew.colb - -.-.colbb| [kingbase8_target - main] INFO filter.RenameDefinitions *.*.colc - -.-.colcc| [kingbase8_target - main] INFO filter.RenameDefinitions schemax.*.cold - -.-.coldd| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.*.* - scheman2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.* - -.tabley2.-| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions scheman.tabley.colk - -.-.colk2| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.*.* - schemam2.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.* - schemam2.tableu2.-| [kingbase8_target - main] INFO filter.RenameDefinitions schemam.tableu.colj - -.-.colj2| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.*.* - oa11g0017.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions OA11G.EMP.* - oa11g0017.emp0017.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_commit_seqno.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.consistency.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.heartbeat.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO filter.RenameDefinitions FLYSYNC.trep_shard_channel.* - flysync_kingbase8_target.-.-| [kingbase8_target - main] INFO applier.JdbcApplier Connecting to data source| [kingbase8_target - main] INFO applier.JdbcApplier Suppressing logging on privileged slave| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-q-to-dbms-2] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.StageTaskGroup Preparing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Preparing raw extractor and heartbeat filter| [kingbase8_target - main] WARN pipeline.StageTaskGroup [main-loader-to-q-0] Last event data not available; Setting extractor to current position| [kingbase8_target - main] INFO pipeline.Pipeline Starting pipeline: slave| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Starting stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-loader-to-q-0] INFO pipeline.SingleThreadStageTask Task event count: 13| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-q-to-dbms-0] INFO pipeline.SingleThreadStageTask Task event count: 5| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno(unavailable)| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-q-to-dbms-2] INFO pipeline.SingleThreadStageTask Task event count: 5| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task has been cancelled| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Performing rollback of possible partial transaction: seqno(unavailable)| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Terminating processing for stage task thread| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Last successfully processed event prior to termination: seqno12 eventidNIL| [kingbase8_target - main-q-to-dbms-1] INFO pipeline.SingleThreadStageTask Task event count: 5| [kingbase8_target - pool-15-thread-1] INFO pipeline.DeferredShutdownTask Pipeline has shut down, dispatching offline event: slave| [kingbase8_target - main] INFO pipeline.Pipeline Shutting down pipeline: slave| [kingbase8_target - main] INFO pipeline.Pipeline Releasing pipeline: slave| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: loader-to-q| [kingbase8_target - main] INFO extractor.ExtractorWrapper Releasing raw extractor and heartbeat filter| [kingbase8_target - main] INFO pipeline.StageTaskGroup Releasing tasks for stage: q-to-dbms| [kingbase8_target - main] INFO loader.LoaderCtrl Total time:2| [kingbase8_target - main] INFO loader.LoaderCtrl
--------------------------------------------------------------------------------
TOTAL: 12
SUCCESS: 12
FAILURE: 0
TOTAL TIME: 2s
FINISHED AT: 2023-11-18 12:53:12
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|NUM |SOURCE_TABLE |TARGET_TABLE |STATUS |TOTAL |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|1 |OA11G.DEPT |oa11g0017.DEPT |Success |4 |
|2 |OA11G.REGIONS |oa11g0017.REGIONS |Success |4 |
|3 |OA11G.JOB_HISTORY |oa11g0017.JOB_HISTORY |Success |10 |
|4 |OA11G.JOBS |oa11g0017.JOBS |Success |19 |
|5 |OA11G.DEPARTMENTS |oa11g0017.DEPARTMENTS |Success |27 |
|6 |OA11G.LOCATIONS |oa11g0017.LOCATIONS |Success |23 |
|7 |OA11G.EMP |oa11g0017.emp0017 |Success |14 |
|8 |OA11G.SALGRADE |oa11g0017.SALGRADE |Success |5 |
|9 |OA11G.COUNTRIES |oa11g0017.COUNTRIES |Success |25 |
|10 |OA11G.EMPLOYEES |oa11g0017.EMPLOYEES |Success |107 |
|11 |OA11G.RUPD$_EMP | |Success |0 |
|12 |OA11G.MLOG$_EMP | |Success |0 |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------[flysyncnode2 KingbaseFlySync-V002R002C004B20230320-replicator]$
(4).增量数据同步
1.将两台服务器同步服务设置为ONLINE开始同步数据
a.Oracle端(Oracle40)
[flysyncoracle40 ~]$ fsrepctl -service oracle_source online
[flysyncoracle40 ~]$ fsrepctl -service oracle_source services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : master
serviceName : oracle_source
serviceType : local
started : true
state : ONLINE
Finished services command...
[flysyncoracle40 ~]$ b.KES端(node2)
[flysyncnode2 ~]$ fsrepctl -service kingbase8_target online
[flysyncnode2 ~]$ fsrepctl -service kingbase8_target services
Processing services command...
NAME VALUE
---- -----
appliedLastSeqno: -1
appliedLatency : -1.0
role : slave
serviceName : kingbase8_target
serviceType : local
started : true
state : ONLINE
Finished services command...
[flysyncnode2 ~]$ 2.同步进程状态查看同步是否正常
a.Oracle端(Oracle40)
[flysyncoracle40 ~]$ fsrepctl -service oracle_source status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : NONE
appliedLastSeqno : -1
appliedLatency : -1.0
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : oracle_source
currentEventId : NONE
currentTimeMillis : 1700285898857
dataServerHost : oracle40
extensions :
host : oracle40
latestEpochNumber : -1
masterConnectUri : kufl://localhost:/
masterListenUri : kufl://oracle40:3112/
maximumStoredSeqNo : -1
minimumStoredSeqNo : -1
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : UNKNOWN
relativeLatency : -1.0
resourcePrecedence : 99
rmiPort : 11000
role : master
seqnoType : java.lang.Long
serviceName : oracle_source
serviceType : local
simpleServiceName : oracle_source
siteName : default
sourceId : oracle40
state : ONLINE
timeInStateSeconds : 281.988
timezone : GMT08:00
transitioningTo :
uptimeSeconds : 311.791
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysyncoracle40 ~]$
b.KES端(node2)
[flysyncnode2 ~]$ fsrepctl -service kingbase8_target status
Processing status command...
NAME VALUE
---- -----
appliedLastEventId : NONE
appliedLastSeqno : -1
appliedLatency : -1.0
autoRecoveryEnabled : false
autoRecoveryTotal : 0
channels : 1
clusterName : kingbase8_target
currentEventId : NONE
currentTimeMillis : 1700285956812
dataServerHost : node2
extensions :
host : node2
latestEpochNumber : -1
masterConnectUri : kufl://oracle40:3112/
masterListenUri : kufl://node2:3112/
maximumStoredSeqNo : -1
minimumStoredSeqNo : -1
offlineRequests : NONE
pendingError : NONE
pendingErrorCode : NONE
pendingErrorEventId : NONE
pendingErrorSeqno : -1
pendingExceptionMessage: NONE
pipelineSource : kufl://oracle40:3112/
relativeLatency : -1.0
resourcePrecedence : 99
rmiPort : 11000
role : slave
seqnoType : java.lang.Long
serviceName : kingbase8_target
serviceType : local
simpleServiceName : kingbase8_target
siteName : default
sourceId : node2
state : ONLINE
timeInStateSeconds : 157.735
timezone : GMT08:00
transitioningTo :
uptimeSeconds : 2963.937
useSSLConnection : false
version : Kingbase FlySync V002R002C004B20230320
Finished status command...
[flysyncnode2 ~]$ 3.数据同步验证(验证DML)
a.在源端(Oracle)中插入数据
[flysyncoracle40 ~]$ exit
logout
[rootoracle40 ~]# su - oracle
Last login: Sat Nov 18 10:11:54 CST 2023 on pts/1
[oracleoracle40 ~]$ sqlplus OA11G/oa11g
SQL*Plus: Release 11.2.0.4.0 Production on Sat Nov 18 13:40:20 2023
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing optionsSQ DESC oa11g.emp;Name Null? Type----------------------------------------- -------- ----------------------------EMPNO NOT NULL NUMBER(4)ENAME VARCHAR2(10)JOB VARCHAR2(9)MGR NUMBER(4)HIREDATE DATESAL NUMBER(7,2)COMM NUMBER(7,2)DEPTNO NUMBER(2)
SQL DESC oa11g.dept;Name Null? Type----------------------------------------- -------- ----------------------------DEPTNO NOT NULL NUMBER(2)DNAME VARCHAR2(14)LOC VARCHAR2(13)
SQL INSERT INTO oa11g.emp(EMPNO, ENAME) VALUES(7935,KFS);
1 row created.
SQL INSERT INTO oa11g.dept VALUES(50,engineer,NEW YORK);
1 row created.
SQL COMMIT;
Commit complete.
SQL b.查看源端kufl 是否解析。
SQL exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
[oracleoracle40 ~]$ exit
logout
[rootoracle40 ~]# su - flysync
Last login: Sat Nov 18 11:57:55 CST 2023 on pts/0
[flysyncoracle40 ~]$ kufl list -last 1
SEQ# 0 / FRAG# 0 (last frag)
- TIME 2023-11-18 13:40:46.0
- EPOCH# 0
- EVENTID ora:0:1212379
- SOURCEID oracle40
- METADATA [dbms_typeoracle;tz_awaretrue;xid1688884220003487;tz_awaretrue;all_tables[OA11G.EMP, OA11G.MLOG$_EMP, OA11G.DEPT];serviceoracle_source;shardOA11G]
- TYPE com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS []- SQL(0) - ACTION INSERT- SCHEMA OA11G- TABLE EMP- ROW# 0- COL(1: EMPNO) 7935- COL(2: ENAME) KFS- COL(3: JOB) NULL- COL(4: MGR) NULL- COL(5: HIREDATE) NULL- COL(6: SAL) NULL- COL(7: COMM) NULL- COL(8: DEPTNO) NULL- SQL(1) - ACTION INSERT- SCHEMA OA11G- TABLE MLOG$_EMP- ROW# 0- COL(1: EMPNO) 7935- COL(2: SNAPTIME$$) 4000-01-01 00:00:00- COL(3: DMLTYPE$$) I- COL(4: OLD_NEW$$) N- COL(5: CHANGE_VECTOR$$) [B180bc464- COL(6: XID$$) 1688884220003487- SQL(2) - ACTION INSERT- SCHEMA OA11G- TABLE DEPT- ROW# 0- COL(1: DEPTNO) 50- COL(2: DNAME) engineer- COL(3: LOC) NEW YORK- SQL(3) - ACTION DELETE- SCHEMA OA11G- TABLE MLOG$_EMP- ROW# 0- KEY(1: EMPNO) 7935- KEY(2: SNAPTIME$$) 4000-01-01 00:00:00- KEY(3: DMLTYPE$$) I- KEY(4: OLD_NEW$$) N- KEY(5: CHANGE_VECTOR$$) [B1324409e- KEY(6: XID$$) 1688884220003487[flysyncoracle40 ~]$ c.在目标端验证kufl是否解析
[flysyncnode2 ~]$ kufl list -last 1
SEQ# 0 / FRAG# 0 (last frag)
- TIME 2023-11-18 13:40:46.0
- EPOCH# 0
- EVENTID ora:0:1212379
- SOURCEID oracle40
- METADATA [dbms_typeoracle;tz_awaretrue;xid1688884220003487;tz_awaretrue;all_tables[OA11G.EMP, OA11G.MLOG$_EMP, OA11G.DEPT];serviceoracle_source;shardOA11G]
- TYPE com.kingbase.flysync.replicator.event.ReplDBMSEvent
- OPTIONS []- SQL(0) - ACTION INSERT- SCHEMA oa11g0017- TABLE emp0017- ROW# 0- COL(1: EMPNO) 7935- COL(2: ENAME) KFS- COL(3: JOB) NULL- COL(4: MGR) NULL- COL(5: HIREDATE) NULL- COL(6: SAL) NULL- COL(7: COMM) NULL- COL(8: DEPTNO) NULL- SQL(1) - ACTION INSERT- SCHEMA oa11g0017- TABLE MLOG$_EMP- ROW# 0- COL(1: EMPNO) 7935- COL(2: SNAPTIME$$) 4000-01-01 00:00:00- COL(3: DMLTYPE$$) I- COL(4: OLD_NEW$$) N- COL(5: CHANGE_VECTOR$$) [B68837a77- COL(6: XID$$) 1688884220003487- SQL(2) - ACTION INSERT- SCHEMA oa11g0017- TABLE DEPT- ROW# 0- COL(1: DEPTNO) 50- COL(2: DNAME) engineer- COL(3: LOC) NEW YORK- SQL(3) - ACTION DELETE- SCHEMA oa11g0017- TABLE MLOG$_EMP- ROW# 0- KEY(1: EMPNO) 7935- KEY(2: SNAPTIME$$) 4000-01-01 00:00:00- KEY(3: DMLTYPE$$) I- KEY(4: OLD_NEW$$) N- KEY(5: CHANGE_VECTOR$$) [B6be46e8f- KEY(6: XID$$) 1688884220003487
[flysyncnode2 ~]$ d.登录到KES数据中查询数据是否存在
[flysyncnode2 ~]$ exit
登出
[rootnode2 ~]# su - kingbase
上一次登录六 11月 18 12:01:37 CST 2023pts/0 上
[kingbasenode2 ~]$ cd /KingbaseES/V8/Server/bin/
[kingbasenode2 bin]$ ./ksql -Usystem test
ksql (V8.0)
输入 help 来获取帮助信息.
test# SELECT * FROM oa11g0017.emp0017 WHERE empno7935;empno | ename | job | mgr | hiredate | sal | comm | deptno
-----------------------------------------------------7935 | KFS | | | | | |
(1 行记录)
test# SELECT * FROM oa11g0017.dept WHERE deptno50;deptno | dname | loc
----------------------------50 | engineer | NEW YORK
(1 行记录)
test# 配置文件服务名错误解决办法 比如 oracle_source写成了oracle_sources 这是最快的解决办法 如果只是配置更新 请先停止服务然后修改flysync.ini文件并用fspm update进行更新即可 例
停止同步
[flysyncoracle40 ~]$ replicator stop
Stopping FlySync Replicator Service...
Stopped FlySync Replicator Service.删除安装目录
[flysyncoracle40 ~]$ rm -rf ~/replicator/修改~/flysync.ini配置文件并重新安装
[flysyncoracle40 ~]$ ~/install/KingbaseFlySync-V002R002C004B20230320-replicator/tools/fspm install重新拷贝授权 cp ~/install/license.dat ~/replicator/license.dat重启服务
fsrepctl -service oracle_source offline
