Skip to content

Support target database in write-back sink#18058

Merged
jt2594838 merged 7 commits into
apache:masterfrom
Caideyipi:writeback-sink-target-database
Jul 1, 2026
Merged

Support target database in write-back sink#18058
jt2594838 merged 7 commits into
apache:masterfrom
Caideyipi:writeback-sink-target-database

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Changes

  • Add sink.database / connector.database support for write-back sink data writes.
  • Validate write-back sink database names on the sink side instead of relying on generic path validation.
  • Use table-model create-database validation for table-model database names.
  • Use tree-model create-database validation semantics for tree-model database names, including root. requirement, legal tree path parsing, database-name pattern check, and length limit.
  • Support multi-level tree-model target databases such as root.target.db.
  • Reject table-model writes when the event or target mapping produces an invalid table-model database, for example tree database root.a.b mapping to table database a.b.
  • Rewrite target database for both table-model and tree-model insertion statements.
  • Add UT coverage for target database validation boundaries, target database customization/mapping, raw tablet event rejection, and statement event rejection.
  • Update tree-model write-back sink IT coverage to use a multi-level target database.

Semantics

  • sink.database=target is treated as a table-model database and mapped to tree database root.target for tree writes.
  • sink.database=root.target and sink.database=root.target.db are treated as tree-model databases.
  • A multi-level tree-model database such as root.target.db is valid for tree-model writes.
  • The corresponding table-model database name for root.target.db is target.db, which is invalid in table model and must reject table-model events.
  • Table-model events always validate their event database name with table-model create-database rules before applying a configured target database, so source values like root.a.b / a.b are rejected.
  • Tree-model events always validate their event database name with tree-model create-database rules before applying a configured target database.

Validation

  • mvn spotless:apply -pl iotdb-core/datanode
  • mvn spotless:apply -pl integration-test -P with-integration-tests
  • git diff --check
  • mvn -pl iotdb-core/datanode -Dtest=PipeSinkTest test was attempted, but datanode compilation fails before test execution in existing generated Freemarker sources unrelated to this change, including missing IFill, IFillFilter, LinearFill, Accumulator, org.apache.commons.collections4.comparators, and IoTDBConfig#getModeMapSizeThreshold().
Comment on lines +665 to +667
if (!startsWith(deviceNodes, sourceDatabaseNodes)) {
return devicePath;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When will this happen?

@jt2594838 jt2594838 merged commit 1c8ed31 into apache:master Jul 1, 2026
40 checks passed
@jt2594838 jt2594838 deleted the writeback-sink-target-database branch July 1, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants