Update docs (#48)

Author: levkk

docs/architecture/comparison.md

@@ -28,7 +28,7 @@ PgDog aims to be the de facto PostgreSQL proxy and pooler. Below is a feature co
 | [Manual routing](../features/sharding/manual-routing.md) | Only using comments (regex), doesn't work with prepared statements | :material-check-circle-outline: |
 | [Automatic routing](../features/sharding/query-routing.md) | No | :material-check-circle-outline: |
 | [Primary key generation](../features/sharding/schema_management/primary_keys.md) | No | :material-check-circle-outline: |
-| [Cross-shard queries](../features/sharding/cross-shard.md) | No | Partial support |
-| [COPY](../features/sharding/copy.md) | No | :material-check-circle-outline: |
+| [Cross-shard queries](../features/sharding/cross-shard-queries/index.md) | No | Partial support |
+| [COPY](../features/sharding/cross-shard-queries/copy.md) | No | :material-check-circle-outline: |
 | [Postgres-compatible sharding functions](../features/sharding/sharding-functions.md) | No | Same functions as declarative partitioning |
 | Two-Phase Commit  | No | :material-check-circle-outline: |

docs/configuration/pgdog.toml/general.md

@@ -393,8 +393,23 @@ Default: **`50_000`**
 
 ### `query_parser_enabled`
 
+!!! warning "Deprecated setting"
+    This setting is deprecated. Use [`query_parser`](#query_parser) instead.
+
 Force-enable query parsing to take advantage of its features in non-sharded databases, like [advisory locks](../../features/transaction-mode.md#advisory-locks) or managing [session state](../../features/transaction-mode.md#session-state).
 
+### `query_parser`
+
+Toggle the query parser to enable/disable query parsing and all of its benefits. By default, the query parser is turned on automatically, so only disable it if you know what you're doing.
+
+Available options:
+
+- `on` (enabled)
+- `off` (disabled)
+- `auto` (automatically enabled or disabled, depending on database configuration)
+
+Default: **`auto`**
+
 ## Logging
 
 ### `log_connections`

docs/configuration/pgdog.toml/rewrite.md

@@ -2,11 +2,9 @@
 icon: material/alpha-r-box-outline
 ---
 
-# Rewrite
+# Rewrite engine
 
-The `rewrite` section controls PgDog's automatic SQL rewrites for sharded clusters. It affects shard-key updates and multi-row INSERT statements, and can be toggled globally or per-policy.
-
-## Options
+The `rewrite` section controls PgDog's automatic SQL rewrites for sharded databases. It affects sharding key updates and multi-tuple inserts. Either one can be toggled separately:
 
 ```toml
 [rewrite]
@@ -15,58 +13,29 @@ shard_key = "error"
 split_inserts = "error"
 ```
 
-| Field | Description | Default |
+| Setting | Description | Default |
 | --- | --- | --- |
-| `enabled` | Master toggle: when `false`, PgDog parses but never applies rewrite plans. | `false` |
+| `enabled` | Enables/disables the query rewrite engine. | `false` |
 | `shard_key` | Behavior when an `UPDATE` changes a sharding key: `error` rejects the statement,<br>`rewrite` migrates the row between shards,<br>`ignore` forwards it unchanged. | `"error"` |
 | `split_inserts` | Behavior when a sharded table receives a multi-row `INSERT`: `error` rejects the statement, `rewrite` fans the rows out to their shards, `ignore` forwards it unchanged. | `"error"` |
 
 !!! note "Two-phase commit"
-    PgDog recommends enabling [two-phase commit](../../features/sharding/2pc.md) when either policy is set to `rewrite`. Without it, rewrites are committed shard-by-shard and can leave partial changes if a shard fails.
+    Consider enabling [two-phase commit](../../features/sharding/2pc.md) when either feature is set to `rewrite`. Without it, rewrites are committed shard-by-shard and can leave partial changes if a transaction fails.
 
 ## Runtime overrides
 
 The admin database exposes these toggles via the `SET` command:
 
 ```postgresql
-SET rewrite_enabled TO true;                -- mirrors [rewrite].enabled
+SET rewrite_enabled TO true;                -- enable/disable rewrite engine
 SET rewrite_shard_key_updates TO rewrite;   -- error | rewrite | ignore
 SET rewrite_split_inserts TO rewrite;       -- error | rewrite | ignore
 ```
 
 The setting changes are applied immediately. These overrides allow canary testing before persisting them in `pgdog.toml`.
 
-## Limitations
-
-### Sharding key updates
-
-Sharding key rewrites in an `UPDATE` clause have to resolve to a single row. If the sharding key isn't unique or the `WHERE` clause has an incorrect `OR` condition, for example, PgDog will rollback the transaction and raise an error.
-
-For example:
-
-```postgresql
-UPDATE users SET id = 5 WHERE admin = true;
-```
-
-On a single-shard deployment, this would raise a unique index violation error. On a cross-shard deployment, the PgDog rewrite engine will block cross-shard updates that could potentially affect multiple rows.
-
-### Multi-tuple inserts
-
-`INSERT` statements with multiple tuples have to be executed outside of an explicit transaction. PgDog needs to start a cross-shard transaction to safely commit the rows to multiple shards, and an existing transaction will interfere with its internal state.
-
-For example:
-
-```postgresql
-BEGIN;
-INSERT INTO users VALUES ($1, $2), ($3, $4);
-```
-
-This scenario will raise an error (code `25001`).
-
-### Default behavior
-
-Both split inserts and sharding key updates fallback to raising an error if `enabled` is set to `false`.
 
 ### Read more
 
-- [Rewrite behavior](../../features/sharding/sharding-functions.md#rewrite-behavior)
+- [Cross-shard INSERT](../../features/sharding/cross-shard-queries/insert.md#multiple-tuples)
+- [Cross-shard UPDATE](../../features/sharding/cross-shard-queries/update.md#sharding-key-updates)

docs/features/sharding/.pages

@@ -1,11 +1,11 @@
 nav:
   - 'index.md'
   - 'basics.md'
+  - 'supported-queries.md'
   - 'query-routing.md'
   - 'manual-routing.md'
-  - 'cross-shard.md'
+  - 'cross-shard-queries'
   - 'sharding-functions.md'
-  - 'copy.md'
   - '...'
   - 'resharding'
   - 'internals'

docs/features/sharding/2pc.md

@@ -31,7 +31,7 @@ ALTER SYSTEM SET max_prepared_transactions TO 1000;
 Alternatively, if you're running on managed Postgres (e.g., AWS RDS), this parameter can usually be set through your cloud admin panel.
 
 !!! note
-     This parameter can only be enabled on server start. Once you change it, make sure to restart your Postgres servers.
+     Changes to this parameter require a server restart to take effect.
 
 Once prepared transactions are enabled in Postgres, two-phase commit can be enabled in [`pgdog.toml`](../../configuration/pgdog.toml/general.md):
 
@@ -104,4 +104,4 @@ Two-phase commit is used for writes only. Read transactions are finished using n
 ## Read more
 
 - [Omnisharded tables](omnishards.md)
-- [Cross-shard queries](cross-shard.md)
+- [Cross-shard queries](cross-shard-queries/index.md)

docs/features/sharding/cross-shard-queries/.pages

@@ -0,0 +1,8 @@
+title: Cross-shard queries
+nav:
+    - 'index.md'
+    - 'select.md'
+    - 'insert.md'
+    - 'update.md'
+    - 'ddl.md'
+    - '...'

docs/features/sharding/copy.md …res/sharding/cross-shard-queries/copy.mddocs/features/sharding/copy.md renamed to docs/features/sharding/cross-shard-queries/copy.md docs/features/sharding/copy.md renamed to docs/features/sharding/cross-shard-queries/copy.md

@@ -1,9 +1,9 @@
 ---
 icon: material/upload
 ---
-# COPY command
+# COPY
 
-`COPY` is a special PostgreSQL command that ingests a file directly into a specified database table. This allows for writing data faster than by using individual `INSERT` queries.
+`COPY` is a special PostgreSQL command that can ingest a file directly into a specified database table. This allows for writing data faster than by using individual `INSERT` queries.
 
 PgDog supports parsing the `COPY` command, splitting the input data stream automatically, and sending the rows to each shard in parallel.
 
@@ -22,7 +22,7 @@ PgDog supports sharding data sent via `COPY`, using any one of the following for
 | Text | PostgreSQL version of CSV, with `<tab>` (`\t`) as the delimiter. | `hello\tworld\t1\t2\t3` |
 | Binary | PostgreSQL-specific format that encodes data using the format used to store it on disk. | |
 
-Each row is extracted from the data stream, inspected for the sharding key, and sent to a data node. The sharding key should be specified in the [configuration](../../configuration/pgdog.toml/sharded_tables.md) and provided in the command statement, for example:
+Each row is extracted from the data stream, inspected for the sharding key, and sent to a data node. The sharding key should be specified in the [configuration](../../../configuration/pgdog.toml/sharded_tables.md) and provided in the command statement, for example:
 
 ```postgresql
 COPY users (id, email) FROM STDIN;
@@ -36,7 +36,19 @@ By using _N_ nodes in a sharded database cluster, the performance of `COPY` incr
 
 The cost of parsing and sharding the CSV stream in PgDog is negligibly small.
 
+## COPY out
+
+All `COPY [...] TO STDOUT` statements are treated as cross-shard and are executed on all shards concurrently. The rows are streamed directly, without buffering or sorting, which allows reading large amounts of data from all shards quickly.
+
+PgDog doesn't currently support routing `COPY` statements based on its query. For example, the following statement will be sent to all shards even if it contains a sharding key:
+
+```postgresql
+COPY (SELECT * FROM users WHERE id IN ($1, $2, $3)) TO STDOUT;
+```
+
+If the query fetches rows from more than one shard, PgDog will also ignore any `ORDER BY` predicates and return rows in whatever order they arrive from the shards.
+
 ## Read more
 
-- [Two-phase commit](2pc.md)
-- [Omnisharded tables](omnishards.md)
+- [Two-phase commit](../2pc.md)
+- [Omnisharded tables](../omnishards.md)

docs/features/sharding/cross-shard-queries/ddl.md

@@ -0,0 +1,30 @@
+---
+icon: material/table-cog
+---
+
+# CREATE, ALTER, DROP
+
+`CREATE`, `ALTER` and `DROP`, also known as **D**ata **D**efinition **L**anguage (DDL), are, by design, cross-shard statements. When a client sends over a DDL command, PgDog will send it to all shards in parallel, ensuring the table, index, view and sequence definitions are identical across the database cluster.
+
+## Atomicity
+
+DDL statements should be atomic across all shards. This is to protect against a single shard failing to create a table or index, which could result in an inconsistent schema. PgDog can use [two-phase commit](../2pc.md) to ensure this is the case, however that means that all DDL statements must be executed inside a transaction, for example:
+
+```postgresql
+BEGIN;
+CREATE TABLE users (
+    id BIGSERIAL PRIMARY KEY,
+    email VARCHAR NOT NULL,
+    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
+);
+COMMIT;
+```
+
+## Idempotency
+
+Some statements, like `CREATE INDEX CONCURRENTLY`, cannot run inside transactions. To make sure these are safely executed, you have two options: use [manual routing](../manual-routing.md) and execute it on each shard individually, or write idempotent schema migrations, for example:
+
+```postgresql
+DROP INDEX IF EXISTS user_id_idx;
+CREATE INDEX CONCURRENTLY user_id_idx ON users USING btree(user_id);
+```

docs/features/sharding/cross-shard-queries/index.md

@@ -0,0 +1,74 @@
+---
+icon: material/multicast
+---
+
+# Cross-shard queries overview
+
+If a client can't or doesn't specify a sharding key in the query, PgDog will send that query to all shards in parallel, and combine the results automatically. To the client, this looks like the query was executed by a single database.
+
+<center style="margin-top: 2rem;">
+    <img src="/images/cross-shard.png" width="95%" alt="Cross-shard queries" />
+</center>
+
+## How it works
+
+PgDog understands the Postgres protocol and query language. It can connect to multiple database servers, send the query to all of them, and collect [`DataRow`](#under-the-hood) messages as they are returned by each connection.
+
+Once all servers finish executing the request, PgDog processes the result, performs any requested sorting, aggregation or row disambiguation, and sends the complete result back to the client, as if all rows came from one database server.
+
+Just like with [direct-to-shard](../query-routing.md) queries, each SQL command is handled differently, as documented below:
+
+- [`SELECT`](select.md)
+- [`INSERT`](insert.md)
+- [`UPDATE`, `DELETE`](update.md)
+- [`CREATE`, `ALTER`, `DROP`](ddl.md) (and other DDL statements)
+- [`COPY`](copy.md)
+
+
+## Under the hood
+
+PgDog implements the PostgreSQL wire protocol, which is well documented and stable. The messages sent by Postgres clients and servers contain all the necessary information about data types, column names and executed statements, which PgDog can use to present multi-database results as a single stream of data.
+
+The following protocol messages are especially relevant:
+
+| Message | Description |
+|-|-|
+| `DataRow` | Each `DataRow` message contains one tuple, for each row returned by the query. |
+| `RowDescription` | This message has the column names and data types returned by the query. |
+| `CommandComplete` | Indicates that the query has finished returning results. PgDog uses it to start sorting and aggregation. |
+
+The protocol has two formats for encoding tuples: text and binary. Text format is equivalent to calling the `to_string()` method on native types, while binary encoding sends them in network-byte order. For example:
+
+=== "Data"
+    ```postgresql
+    SELECT 1::bigint, 2::integer, 'three'::VARCHAR;
+    ```
+=== "Encoding"
+    | Data type | Text | Binary |
+    |-|-|-|
+    | `BIGINT` | `"1"` | `00 00 00 00 00 00 00 01` |
+    | `INTEGER` | `"2"` | `00 00 00 02` |
+    | `VARCHAR` | `"three"` | `three` |
+
+Since PgDog needs to process rows before sending them to the client, we implemented parsing both formats for [most data types](select.md#supported-data-types).
+
+
+## Disabling cross-shard queries
+
+If you don't want PgDog to route cross-shard queries, e.g., because you have a [multitenant](../../multi-tenancy.md) system with no interdependencies, cross-shard queries can be disabled with a configuration setting:
+
+```toml
+[general]
+cross_shard_disabled = true
+```
+
+When this setting is enabled and a query doesn't have a sharding key, instead of executing the query, PgDog will return an error and abort the transaction.
+
+## Read more
+
+- [Sharding functions](../sharding-functions.md)
+- [Cross-shard `SELECT`](select.md)
+- [Cross-shard `INSERT`](insert.md)
+- [Cross-shard `UPDATE` and `DELETE`](update.md)
+- [DDL, e.g., `CREATE TABLE`](ddl.md)
+- [`COPY` command](copy.md)