TiDB

The basic requirements for data integration are exactly the same as data transfer, for more information please refer to Data Transfer Description For TiDB

Migration Content

Data integration currently only migrates Database, Table and their data, and does not migrate other content such as View.

Function Description

Execution Order

• The data sources are sorted according to the order they were added, with tasks at the front being executed first.
• When the task type of a certain data source is a full task, only after this full task is completed will the subsequent tasks start.

Keep Exist Data

The data source configuration provides an option to KeepExistData, which is set to True by default.

• If the default True option is retained, new data will only be continuously added to the target database. If a conflict arises, the set conflict resolution strategy will be executed.
• If False is selected, the corresponding database table in the target of the ETL settings will be deleted before the task starts.

Data Conflict Resolution Strategy

The data source configuration provides a choice for the DupAction, which is used to select the processing method when data conflicts occur.

• If Replace is selected, 'replace into' will be used to replace existing data.
• If ignore is selected, the original data will be retained and the new data will be ignored.

Extract, Transform, and Load

• When filling in ETL information, if a database or table is selected, a mapping name must be filled in. If there is no need to modify the database/table name, fill in the same database/table name as the data source by default.
• When filling in ETL information, if a database is selected but no table is chosen, all tables under this database will be migrated.

Function Limitations

• Data integration tasks temporarily do not support DDL statements, as DDL statements may change the table structure of the target database, damaging the ETL function.