VDiff2
Compare the source and target in a workflow to ensure integrity
vttablets
as compared with VDiff1 which runs on vtctld
.For additional details, please see the RFC and the README.
Command #
VDiff2 takes different sub-commands or actions similar to how the MoveTables
/Reshard
commands work. The first argument
is the <keyspace.workflow> followed by
Start a new VDiff #
These take the same parameters as VDiff1 and schedule VDiff to run on the primary tablet of each target shard to verify
the subset of data that will live on the given shard. Please note that if you do not specify a sub-command or action
then create
is assumed (this eases the transition from VDiff1 to VDiff2). If you do not pass a specific UUID then one
will be generated.
VDiff -- --v2 [--source_cell=<cell>] [--target_cell=<cell>] [--tablet_types=in_order:RDONLY,REPLICA,PRIMARY]
[--limit=<max rows to diff>] [--tables=<table list>] [--format=json] [--max_extra_rows_to_compare=1000]
[--filtered_replication_wait_time=30s] [--debug_query] [--only_pks] <keyspace.workflow> create [<UUID>]
Each scheduled VDiff has an associated UUID which is returned by the create
action. You can use it
to monitor progress. Example:
$ vtctlclient --server=localhost:15999 VDiff -- --v2 customer.commerce2customer
VDiff bf9dfc5f-e5e6-11ec-823d-0aa62e50dd24 scheduled on target shards, use show to view progress
Resume a previous VDiff #
Allows you to resume an existing VDiff workflow, picking up where it left off and comparing the records where the Primary Key column(s) are greater than the last record processed — with the progress and other status information saved when the run ends. This allows you to:
- Resume a VDiff that may have encountered an ephemeral error
- Do approximate rolling or differential VDiffs (e.g. done after MoveTables finishes the initial copy phase and then again just before SwitchTraffic)
VDiff -- --v2 [--source_cell=<cell>] [--target_cell=<cell>] [--tablet_types=in_order:RDONLY,REPLICA,PRIMARY]
[--limit=<max rows to diff>] [--tables=<table list>] [--format=json] [--max_extra_rows_to_compare=1000]
[--filtered_replication_wait_time=30s] [--debug_query] [--only_pks] <keyspace.workflow> resume <UUID>
Example:
$ vtctlclient --server=localhost:15999 VDiff -- --v2 customer.commerce2customer resume 4c664dc2-eba9-11ec-9ef7-920702940ee0
VDiff 4c664dc2-eba9-11ec-9ef7-920702940ee0 resumed on target shards, use show to view progress
resume
when different collations are used for a table between the source and target keyspaces (more details can be seen here).Show progress/status of a VDiff #
VDiff -- --v2 <keyspace.workflow> show [<UUID> | last | all]
You can either show
a specific UUID or use the last
convenience shorthand to look at the most recently created VDiff. Example:
$ vtctlclient --server=localhost:15999 VDiff -- --v2 customer.commerce2customer show last
VDiff Summary for customer.commerce2customer (4c664dc2-eba9-11ec-9ef7-920702940ee0)
State: completed
RowsCompared: 196
HasMismatch: false
StartedAt: 2022-06-26 22:44:29
CompletedAt: 2022-06-26 22:44:31
Use "--format=json" for more detailed output.
$ vtctlclient --server=localhost:15999 VDiff -- --v2 --format=json customer.commerce2customer show last
{
"Workflow": "commerce2customer",
"Keyspace": "customer",
"State": "completed",
"UUID": "4c664dc2-eba9-11ec-9ef7-920702940ee0",
"RowsCompared": 196,
"HasMismatch": false,
"Shards": "0",
"StartedAt": "2022-06-26 22:44:29",
"CompletedAt": "2022-06-26 22:44:31"
}
show all
lists all vdiffs created for the specified keyspace and workflow.
Description #
VDiff does a row by row comparison of all tables associated with the workflow, diffing the source keyspace and the target keyspace and reporting counts of missing/extra/unmatched rows.
It is highly recommended that you do this before you finalize a workflow with SwitchTraffic
.
The actions supported
Parameters #
--source_cell #
optional
default all
--target_cell #
optional
default all
--tablet_types #
optional
default in_order:RDONLY,REPLICA,PRIMARY
--filtered_replication_wait_time #
optional
default 30s
--limit #
optional
default 9223372036854775807
--tables #
optional
default all tables in the workflow
--format #
optional
default unstructured text output
--max_extra_rows_to_compare #
optional
default 1000
--debug_query #
optional
--only_pks #
optional
keyspace.workflow #
mandatory