vtgate

VTGate is a stateless proxy responsible for accepting requests from applications and routing them to the appropriate tablet server(s) for query execution. It speaks both the MySQL Protocol and a gRPC protocol.

Example Usage #

Start a vtgate proxy:

export TOPOLOGY_FLAGS="--topo_implementation etcd2 --topo_global_server_address localhost:2379 --topo_global_root /vitess/global"
export VTDATAROOT="/tmp"

vtgate \
  $TOPOLOGY_FLAGS \
  --log_dir $VTDATAROOT/tmp \
  --port 15001 \
  --grpc_port 15991 \
  --mysql_server_port 15306 \
  --cell test \
  --cells_to_watch test \
  --tablet_types_to_wait PRIMARY,REPLICA \
  --service_map 'grpc-vtgateservice' \
  --pid_file $VTDATAROOT/tmp/vtgate.pid \
  --mysql_auth_server_impl none

Options #

The following global options apply to vtgate:

NameTypeDefinition
--allowed_tablet_typesStringsSpecifies the tablet types this vtgate is allowed to route queries to. Should be provided as a comma-separated set of tablet types.
--alsologtostderrBooleanlog to standard error as well as files
--buffer_drain_concurrencyIntMaximum number of requests retried simultaneously. More concurrency will increase the load on the PRIMARY vttablet when draining the buffer. (default 1)
--buffer_implementationStringAllowed values: healthcheck (legacy implementation), keyspace_events (default) (default "keyspace_events")
--buffer_keyspace_shardsStringIf not empty, limit buffering to these entries (comma separated). Entry format: keyspace or keyspace/shard. Requires --enable_buffer=true.
--buffer_max_failover_durationDurationStop buffering completely if a failover takes longer than this duration. (default 20s)
--buffer_min_time_between_failoversDurationMinimum time between the end of a failover and the start of the next one (tracked per shard). Faster consecutive failovers will not trigger buffering. (default 1m0s)
--buffer_sizeIntMaximum number of buffered requests in flight (across all ongoing failovers). (default 1000)
--buffer_windowDurationDuration for how long a request should be buffered at most. (default 10s)
--catch-sigpipeBooleancatch and ignore SIGPIPE on stdout and stderr if specified
--cellString
--cells_to_watchStringcomma-separated list of cells for watching tablets
--consul_auth_static_fileStringJSON File to read the topos/tokens from.
--datadog-agent-hostStringhost to send spans to. if empty, no tracing will be done
--datadog-agent-portStringport to send spans to. if empty, no tracing will be done
--dbddl_pluginStringcontrols how to handle CREATE/DROP DATABASE. use it if you are using your own database provisioning service (default "fail")
--ddl_strategyStringSet default strategy for DDL statements. Override with @@ddl_strategy session variable (default "direct")
--default_tablet_typeTopodatapb.TabletTypeThe default tablet type to set for queries, when one is not explicitly selected. (default PRIMARY)
--discovery_high_replication_lag_minimum_servingDurationThreshold above which replication lag is considered too high when applying the min_number_serving_vttablets flag. (default 2h0m0s)
--discovery_low_replication_lagDurationThreshold below which replication lag is considered low enough to be healthy. (default 30s)
--emit_statsBooleanIf set, emit stats to push-based monitoring and stats backends
--enable-partial-keyspace-migrationBoolean(Experimental) Follow shard routing rules: enable only while migrating a keyspace shard by shard. See documentation on Partial MoveTables for more. (default false)
--enable-viewsBooleanEnable views support in vtgate.
--enable_bufferBooleanEnable buffering (stalling) of primary traffic during failovers.
--enable_buffer_dry_runBooleanDetect and log failover events, but do not actually buffer requests.
--enable_direct_ddlBooleanAllow users to submit direct DDL statements (default true)
--enable_online_ddlBooleanAllow users to submit, review and control Online DDL (default true)
--enable_set_varBooleanThis will enable the use of MySQL's SET_VAR query hint for certain system variables instead of using reserved connections (default true)
--enable_system_settingsBooleanThis will enable the system settings to be changed per session at the database connection level (default true)
--foreign_key_modeStringThis is to provide how to handle foreign key constraint in create/alter table. Valid values are: allow, disallow (default "allow")
--gate_query_cache_lfuBooleangate server cache algorithm. when set to true, a new cache algorithm based on a TinyLFU admission policy will be used to improve cache behavior and prevent pollution from sparse queries (default true)
--gate_query_cache_memoryIntgate server query cache size in bytes, maximum amount of memory to be cached. vtgate analyzes every incoming query and generate a query plan, these plans are being cached in a lru cache. This config controls the capacity of the lru cache. (default 33554432)
--gate_query_cache_sizeIntgate server query cache size, maximum number of queries to be cached. vtgate analyzes every incoming query and generate a query plan, these plans are being cached in a cache. This config controls the expected amount of unique entries in the cache. (default 5000)
--gateway_initial_tablet_timeoutDurationAt startup, the tabletGateway will wait up to this duration to get at least one tablet per keyspace/shard/tablet type (default 30s)
--grpc-send-session-in-streamingBooleanIf set, will send the session as last packet in streaming api to support transactions in streaming
--grpc-use-effective-groupsBooleanIf set, and SSL is not used, will set the immediate caller's security groups from the effective caller id's groups.
--grpc_auth_modeStringWhich auth plugin implementation to use (eg: static)
--grpc_auth_mtls_allowed_substringsStringList of substrings of at least one of the client certificate names (separated by colon).
--grpc_auth_static_client_credsStringWhen using grpc_static_auth in the server, this file provides the credentials to use to authenticate with server.
--grpc_auth_static_password_fileStringJSON File to read the users/passwords from.
--grpc_caStringserver CA to use for gRPC connections, requires TLS, and enforces client certificate check
--grpc_certStringserver certificate to use for gRPC connections, requires grpc_key, enables TLS
--grpc_compressionStringWhich protocol to use for compressing gRPC. Default: nothing. Supported: snappy
--grpc_crlStringpath to a certificate revocation list in PEM format, client certificates will be further verified against this file during TLS handshake
--grpc_enable_optional_tlsBooleanenable optional TLS mode when a server accepts both TLS and plain-text connections on the same port
--grpc_enable_tracingBooleanEnable gRPC tracing.
--grpc_initial_conn_window_sizeIntgRPC initial connection window size
--grpc_initial_window_sizeIntgRPC initial window size
--grpc_keepalive_timeDurationAfter a duration of this time, if the client doesn't see any activity, it pings the server to see if the transport is still alive. (default 10s)
--grpc_keepalive_timeoutDurationAfter having pinged for keepalive check, the client waits for a duration of Timeout and if no activity is seen even after that the connection is closed. (default 10s)
--grpc_keyStringserver private key to use for gRPC connections, requires grpc_cert, enables TLS
--grpc_max_connection_ageDurationMaximum age of a client connection before GoAway is sent. (default 2562047h47m16.854775807s)
--grpc_max_connection_age_graceDurationAdditional grace period after grpc_max_connection_age, after which connections are forcibly closed. (default 2562047h47m16.854775807s)
--grpc_max_message_sizeIntMaximum allowed RPC message size. Larger messages will be rejected by gRPC with the error 'exceeding the max size'. (default 16777216)
--grpc_portIntPort to listen on for gRPC calls. If zero, do not listen.
--grpc_prometheusBooleanEnable gRPC monitoring with Prometheus.
--grpc_server_caStringpath to server CA in PEM format, which will be combine with server cert, return full certificate chain to clients
--grpc_server_initial_conn_window_sizeIntgRPC server initial connection window size
--grpc_server_initial_window_sizeIntgRPC server initial window size
--grpc_server_keepalive_enforcement_policy_min_timeDurationgRPC server minimum keepalive time (default 10s)
--grpc_server_keepalive_enforcement_policy_permit_without_streamBooleangRPC server permit client keepalive pings even when there are no active streams (RPCs)
--grpc_use_effective_calleridBooleanIf set, and SSL is not used, will set the immediate caller id from the effective caller id's principal.
--grpc-use-static-authentication-calleridbooleanIf set, will set the immediate caller id to the username authenticated by the static auth plugin.
--healthcheck_retry_delayDurationhealth check retry delay (default 2ms)
--healthcheck_timeoutDurationthe health check timeout period (default 1m0s)
-h, --helpBooleandisplay usage and exit
--jaeger-agent-hostStringhost and port to send spans to. if empty, no tracing will be done
--keep_logsDurationkeep logs for this long (using ctime) (zero to keep forever)
--keep_logs_by_mtimeDurationkeep logs for this long (using mtime) (zero to keep forever)
--keyspaces_to_watchStringsSpecifies which keyspaces this vtgate should have access to while routing queries or accessing the vschema.
--lameduck-periodDurationkeep running at least this long after SIGTERM before stopping (default 50ms)
--legacy_replication_lag_algorithmBooleanUse the legacy algorithm when selecting vttablets for serving. (default true)
--lock-timeoutDurationMaximum time for which a shard/keyspace lock can be acquired for (default 45s)
--lock_heartbeat_timeDurationIf there is lock function used. This will keep the lock connection active by using this heartbeat (default 5s)
--log_backtrace_atTraceLocationwhen logging hits line file:N, emit a stack trace (default :0)
--log_dirStringIf non-empty, write log files in this directory
--log_err_stacksBooleanlog stack traces for errors
--log_queries_to_fileStringEnable query logging to the specified file
--log_rotate_max_sizeUintsize in bytes at which logs are rotated (glog.MaxSize) (default 1887436800)
--logtostderrBooleanlog to standard error instead of files
--max-stack-sizeIntconfigure the maximum stack size in bytes (default 67108864)
--max_memory_rowsIntMaximum number of rows that will be held in memory for intermediate results as well as the final result. (default 300000)
--max_payload_sizeIntThe threshold for query payloads in bytes. A payload greater than this threshold will result in a failure to handle the query.
--message_stream_grace_periodDurationthe amount of time to give for a vttablet to resume if it ends a message stream, usually because of a reparent. (default 30s)
--min_number_serving_vttabletsIntThe minimum number of vttablets for each replicating tablet_type (e.g. replica, rdonly) that will be continue to be used even with replication lag above discovery_low_replication_lag, but still below discovery_high_replication_lag_minimum_serving. (default 2)
--mysql-server-pool-conn-read-buffersBooleanIf set, the server will pool incoming connection read buffers
--mysql_allow_clear_text_without_tlsBooleanIf set, the server will allow the use of a clear text password over non-SSL connections.
--mysql_auth_server_implStringWhich auth server implementation to use. Options: none, ldap, clientcert, static, vault. (default "static")
--mysql_auth_server_static_fileStringJSON File to read the users/passwords from.
--mysql_auth_server_static_stringStringJSON representation of the users/passwords config.
--mysql_auth_static_reload_intervalDurationTicker to reload credentials
--mysql_auth_vault_addrStringURL to Vault server
--mysql_auth_vault_pathStringVault path to vtgate credentials JSON blob, e.g.: secret/data/prod/vtgatecreds
--mysql_auth_vault_role_mountpointStringVault AppRole mountpoint; can also be passed using VAULT_MOUNTPOINT environment variable (default "approle")
--mysql_auth_vault_role_secretidfileStringPath to file containing Vault AppRole secret_id; can also be passed using VAULT_SECRETID environment variable
--mysql_auth_vault_roleidStringVault AppRole id; can also be passed using VAULT_ROLEID environment variable
--mysql_auth_vault_timeoutDurationTimeout for vault API operations (default 10s)
--mysql_auth_vault_tls_caStringPath to CA PEM for validating Vault server certificate
--mysql_auth_vault_tokenfileStringPath to file containing Vault auth token; token can also be passed using VAULT_TOKEN environment variable
--mysql_auth_vault_ttlDurationHow long to cache vtgate credentials from the Vault server (default 30m0s)
--mysql_clientcert_auth_methodStringclient-side authentication method to use. Supported values: mysql_clear_password, dialog. (default "mysql_clear_password")
--mysql_default_workloadStringDefault session workload (OLTP, OLAP, DBA) (default "OLTP")
--mysql_ldap_auth_config_fileStringJSON File from which to read LDAP server config.
--mysql_ldap_auth_config_stringStringJSON representation of LDAP server config.
--mysql_ldap_auth_methodStringclient-side authentication method to use. Supported values: mysql_clear_password, dialog. (default "mysql_clear_password")
--mysql_server_bind_addressStringBinds on this address when listening to MySQL binary protocol. Useful to restrict listening to 'localhost' only for instance.
--mysql_server_flush_delayDurationDelay after which buffered response will be flushed to the client. (default 100ms)
--mysql_server_portIntIf set, also listen for MySQL binary protocol connections on this port. (default -1)
--mysql_server_query_timeoutDurationmysql query timeout
--mysql_server_read_timeoutDurationconnection read timeout
--mysql_server_require_secure_transportBooleanReject insecure connections but only if mysql_server_ssl_cert and mysql_server_ssl_key are provided
--mysql_server_socket_pathStringThis option specifies the Unix socket file to use when listening for local connections. By default it will be empty and it won't listen to a unix socket
--mysql_server_ssl_caStringPath to ssl CA for mysql server plugin SSL. If specified, server will require and validate client certs.
--mysql_server_ssl_certStringPath to the ssl cert for mysql server plugin SSL
--mysql_server_ssl_crlStringPath to ssl CRL for mysql server plugin SSL
--mysql_server_ssl_keyStringPath to ssl key for mysql server plugin SSL
--mysql_server_ssl_server_caStringpath to server CA in PEM format, which will be combine with server cert, return full certificate chain to clients
--mysql_server_tls_min_versionStringConfigures the minimal TLS version negotiated when SSL is enabled. Defaults to TLSv1.2. Options: TLSv1.0, TLSv1.1, TLSv1.2, TLSv1.3.
--mysql_server_versionStringMySQL server version to advertise.
--mysql_server_write_timeoutDurationconnection write timeout
--mysql_slow_connect_warn_thresholdDurationWarn if it takes more than the given threshold for a mysql connection to establish
--mysql_tcp_versionStringSelect tcp, tcp4, or tcp6 to control the socket type. (default "tcp")
--no_scatterBooleanwhen set to true, the planner will fail instead of producing a plan that includes scatter queries
--normalize_queriesBooleanRewrite queries with bind vars. Turn this off if the app itself sends normalized queries with bind vars. (default true)
--onclose_timeoutDurationwait no more than this for OnClose handlers before stopping (default 10s)
--onterm_timeoutDurationwait no more than this for OnTermSync handlers before stopping (default 10s)
--opentsdb_uriStringURI of opentsdb /api/put method
--pid_fileStringIf set, the process will write its pid to the named file, and delete it on graceful shutdown.
--planner-versionStringSets the default planner to use when the session has not changed it. Valid values are: V3, Gen4, Gen4Greedy and Gen4Fallback. Gen4Fallback tries the gen4 planner and falls back to the V3 planner if the gen4 fails.
--portIntport for the server
--pprof stringsBooleanenable profiling
--proxy_protocolBooleanEnable HAProxy PROXY protocol on MySQL listener socket
--purge_logs_intervalDurationhow often try to remove old logs (default 1h0m0s)
--query-timeoutIntSets the default query timeout (in ms). Can be overridden by session variable (query_timeout) or comment directive (QUERY_TIMEOUT_MS)
--querylog-buffer-sizeIntMaximum number of buffered query logs before throttling log output (default 10)
--querylog-filter-tagStringstring that must be present in the query for it to be logged; if using a value as the tag, you need to disable query normalization
--querylog-formatStringformat for query logs ("text" or "json") (default "text")
--querylog-row-thresholdUintNumber of rows a query has to return or affect before being logged; not useful for streaming queries. 0 means all queries will be logged.
--redact-debug-ui-queriesBooleanredact full queries and bind variables from debug UI
--remote_operation_timeoutDurationtime to wait for a remote operation (default 15s)
--retry-countIntretry count (default 2)
--schema_change_signalBooleanEnable the schema tracker; requires queryserver-config-schema-change-signal to be enabled on the underlying vttablets for this to work (default true)
--schema_change_signal_userStringUser to be used to send down query to vttablet to retrieve schema changes
--security_policyStringthe name of a registered security policy to use for controlling access to URLs - empty means allow all for anyone (built-in policies: deny-all, read-only)
--service_mapStringscomma separated list of services to enable (or disable if prefixed with '-') Example: grpc-queryservice
--sql-max-length-errorsInttruncate queries in error logs to the given length (default unlimited)
--sql-max-length-uiInttruncate queries in debug UIs to the given length (default 512) (default 512)
--srv_topo_cache_refreshDurationhow frequently to refresh the topology for cached entries (default 1s)
--srv_topo_cache_ttlDurationhow long to use cached entries for topology (default 1s)
--srv_topo_timeoutDurationtopo server timeout (default 5s)
--stats_backendStringThe name of the registered push-based monitoring/stats backend to use
--stats_combine_dimensionsStringList of dimensions to be combined into a single "all" value in exported stats vars
--stats_common_tagsStringsComma-separated list of common tags for the stats backend. It provides both label and values. Example: label1:value1,label2:value2
--stats_drop_variablesStringVariables to be dropped from the list of exported variables.
--stats_emit_periodDurationInterval between emitting stats to all registered backends (default 1m0s)
--statsd_addressStringAddress for statsd client
--statsd_sample_rateFloatSample rate for statsd metrics (default 1)
--stderrthresholdSeveritylogs at or above this threshold go to stderr (default 1)
--stream_buffer_sizeIntthe number of bytes sent from vtgate for each stream call. It's recommended to keep this value in sync with vttablet's query-server-config-stream-buffer-size. (default 32768)
--tablet_filtersStringsSpecifies a comma-separated list of 'keyspace
--tablet_grpc_caStringthe server ca to use to validate servers when connecting
--tablet_grpc_certStringthe cert to use to connect
--tablet_grpc_crlStringthe server crl to use to validate server certificates when connecting
--tablet_grpc_keyStringthe key to use to connect
--tablet_grpc_server_nameStringthe server name to use to validate server certificate
--tablet_protocolStringProtocol to use to make queryservice RPCs to vttablets. (default "grpc")
--tablet_refresh_intervalDurationTablet refresh interval. (default 1m0s)
--tablet_refresh_known_tabletsBooleanWhether to reload the tablet's address/port map from topo in case they change. (default true)
--tablet_types_to_waitStringsWait till connected for specified tablet types during Gateway initialization. Should be provided as a comma-separated set of tablet types.
--tablet_url_templateStringFormat string describing debug tablet url formatting. See getTabletDebugURL() for how to customize this. (default "http://{{.GetTabletHostPort}}")
--topo_consul_lock_delayDurationLockDelay for consul session. (default 15s)
--topo_consul_lock_session_checksStringList of checks for consul session. (default "serfHealth")
--topo_consul_lock_session_ttlStringTTL for consul session.
--topo_consul_watch_poll_durationDurationtime of the long poll for watch queries. (default 30s)
--topo_etcd_lease_ttlIntLease TTL for locks and leader election. The client will use KeepAlive to keep the lease going. (default 30)
--topo_etcd_tls_caStringpath to the ca to use to validate the server cert when connecting to the etcd topo server
--topo_etcd_tls_certStringpath to the client cert to use to connect to the etcd topo server, requires topo_etcd_tls_key, enables TLS
--topo_etcd_tls_keyStringpath to the client key to use to connect to the etcd topo server, enables TLS
--topo_global_rootStringthe path of the global topology data in the global topology server
--topo_global_server_addressStringthe address of the global topology server
--topo_implementationStringthe topology implementation to use
--topo_k8s_contextStringThe kubeconfig context to use, overrides the 'current-context' from the config
--topo_k8s_kubeconfigStringPath to a valid kubeconfig file. When running as a k8s pod inside the same cluster you wish to use as the topo, you may omit this and the below arguments, and Vitess is capable of auto-discovering the correct values. https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/#accessing-the-api-from-a-pod
--topo_k8s_namespaceStringThe kubernetes namespace to use for all objects. Default comes from the context or in-cluster config
--topo_read_concurrencyIntConcurrency of topo reads. (default 32)
--topo_zk_auth_fileStringauth to use when connecting to the zk topo server, file contents should be :, e.g., digest:user:pass
--topo_zk_base_timeoutDurationzk base timeout (see zk.Connect) (default 30s)
--topo_zk_max_concurrencyIntmaximum number of pending requests to send to a Zookeeper server. (default 64)
--topo_zk_tls_caStringthe server ca to use to validate servers when connecting to the zk topo server
--topo_zk_tls_certStringthe cert to use to connect to the zk topo server, requires topo_zk_tls_key, enables TLS
--topo_zk_tls_keyStringthe key to use to connect to the zk topo server, enables TLS
--tracerStringtracing service to use (default "noop")
--tracing-enable-loggingBooleanwhether to enable logging in the tracing service
--tracing-sampling-rateFloatsampling rate for the probabilistic jaeger sampler (default 0.1)
--tracing-sampling-typeStringsampling strategy to use for jaeger. possible values are 'const', 'probabilistic', 'rateLimiting', or 'remote' (default "const")
--transaction_modeStringSINGLE: disallow multi-db transactions, MULTI: allow multi-db transactions with best effort commit, TWOPC: allow multi-db transactions with 2pc commit (default "MULTI")
--truncate-error-lenInttruncate errors sent to client if they are longer than this value (0 means do not truncate)
--vLevellog level for V logs
-v, --versionBooleanprint binary version
--vmoduleModuleSpeccomma-separated list of pattern=N settings for file-filtered logging
--vschema_ddl_authorized_usersStringList of users authorized to execute vschema ddl operations, or '%' to allow all users.
--vtgate-config-terse-errorsBooleanprevent bind vars from escaping in returned errors
--warn_memory_rowsIntWarning threshold for in-memory results. A row count higher than this amount will cause the VtGateWarnings.ResultsExceeded counter to be incremented. (default 30000)
--warn_payload_sizeIntThe warning threshold for query payloads in bytes. A payload greater than this threshold will cause the VtGateWarnings.WarnPayloadSizeExceeded counter to be incremented.
--warn_sharded_onlyBooleanIf any features that are only available in unsharded mode are used, query execution warnings will be added to the session

Key Options #

  • --srv_topo_cache_ttl: There may be instances where you will need to increase the cached TTL from the default of 1 second to a higher number:
    • You may want to increase this option if you see that your topo leader goes down and keeps your queries waiting for a few seconds