Quack: Bringing Client-Server Capabilities to DuckDB

For years, DuckDB has championed the in-process architecture, positioning itself as the "SQLite for analytics." By eliminating the overhead of a client-server protocol, DuckDB provided unmatched performance for data scientists and developers working within a single process. However, this design created a significant hurdle: the inability for multiple processes to modify the same database file simultaneously without complex workarounds.

To solve this, the DuckDB team has introduced Quack, a dedicated client-server protocol. Quack allows DuckDB instances to communicate with one another, enabling a setup where one instance acts as a server (holding the mutable state) and others act as clients. This evolution transforms DuckDB from a strictly local tool into a versatile building block for distributed data architectures.

How Quack Works

Quack is designed to be simple to deploy while maintaining high performance. It is implemented as a DuckDB extension, meaning both the client and server are simply DuckDB instances with the quack extension loaded.

Setup and Usage

Setting up a Quack server is straightforward. A server instance initializes the service using quack_serve, while a client instance attaches to that server using a secret token for authentication:

Server Side:

INSTALL quack FROM core_nightly;
LOAD quack;
CALL quack_serve('quack:localhost', token = 'super_secret');

Client Side:

INSTALL quack FROM core_nightly;
LOAD quack;
CREATE SECRET (TYPE quack, TOKEN 'super_secret');
ATTACH 'quack:localhost' AS remote;

Once attached, clients can query remote tables as if they were local or use the remote.query() function to ship complex SQL verbatim to the server, ensuring that heavy computation happens where the data resides.

Protocol Design Decisions

The design of Quack reflects a modern approach to database communication, prioritizing interoperability and low latency.

Built on HTTP

Quack leverages HTTP as its transport layer. This decision ensures that the protocol is compatible with existing infrastructure, such as load balancers, firewalls, and reverse proxies (like Nginx). Crucially, this allows DuckDB-Wasm to speak Quack natively, enabling a browser-based DuckDB instance to connect directly to a remote server.

Optimized Serialization and Round-Trips

Rather than using a generic interchange format, Quack uses a custom MIME type (application/duckdb). This leverages DuckDB's internal serialization primitives—the same ones used in its Write-Ahead Log (WAL)—to minimize the cost of encoding and decoding data.

To reduce latency, Quack is optimized for minimal round-trips. Most queries can be completed in a single request-response pair, a significant improvement over protocols like Arrow Flight SQL, which require multiple round-trips for a single query.

Security and Extensibility

By default, Quack binds to localhost and generates a random authentication token on startup. For production environments, the team recommends using a reverse proxy to terminate SSL.

One of the most powerful aspects of Quack is its extensible authentication and authorization model. Instead of a rigid system, Quack allows these functions to be overridden by user-supplied code or even plain SQL macros, allowing developers to integrate with LDAP or custom identity providers.

Performance Benchmarks

DuckDB tested Quack against PostgreSQL and Arrow Flight SQL on AWS m8g.2xlarge instances. The results highlight Quack's efficiency in two distinct scenarios:

Bulk Transfer

In tests transferring the TPC-H lineitem table, Quack significantly outperformed both competitors. For 60 million rows (approximately 76 GB in CSV format), Quack completed the transfer in 4.94 seconds, compared to 17.40 seconds for Arrow Flight and a staggering 158.37 seconds for PostgreSQL.

Small Writes

For transactional workloads (single-row INSERT transactions), Quack surprisingly outperformed PostgreSQL in low-concurrency scenarios, reaching approximately 5,500 transactions per second across 8 parallel threads. While PostgreSQL scales better beyond 8 threads, Quack provides a high-performance alternative for centralizing observability data or telemetry.

Community Perspectives and Use Cases

The introduction of Quack has sparked significant interest among developers who previously had to build custom RPC layers to scale DuckDB.

• Horizontal Scaling: Developers are now exploring how to use Quack to scale internal application frameworks that previously struggled with DuckDB's single-process limitation.
• Observability: The ability to pipe sensor readings or logs into a central DuckDB instance without locking out other users (such as those using a UI for data exploration) is a primary win.
• The "SQLite of Analytics": As one community member noted, Quack doesn't change DuckDB's identity; it simply extends the "everywhere" philosophy to include remote servers.

However, not all feedback was glowing. Some critics argued that building a database protocol on HTTP is a "path of least resistance" that could introduce timeout issues or streaming limitations compared to raw TCP implementations.

Future Roadmap

Quack is currently available in the core_nightly repository and is slated for a production release with DuckDB v2.0 in Fall 2026. Key upcoming improvements include:

1. DuckLake Integration: Using a remote DuckDB server as a DuckLake catalog to improve performance and data inlining.
2. Scaling Transactions: Increasing the limit of concurrent insertions per second to scale beyond 8 parallel threads.
3. Replication: Developing a replication protocol on top of Quack to support clusters of read replicas.
4. Protocol Extensions: Allowing other DuckDB extensions to define their own protocol messages and handlers.