Global read replication

D1 read replication can lower latency for read queries and scale read throughput by adding read-only database copies, called read replicas, across regions globally closer to clients.

Your application can use read replicas with D1 Sessions API. A session encapsulates all the queries from one logical session for your application. For example, a session may correspond to all queries coming from a particular web browser session. All queries within a session read from a database instance which is as up-to-date as your query needs it to be. Sessions API ensures sequential consistency for all queries in a session.

To checkout D1 read replication, deploy the following Worker code using Sessions API, which will prompt you to create a D1 database and enable read replication on said database.

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);

    // A. Create the Session.
    // When we create a D1 Session, we can continue where we left off from a previous
    // Session if we have that Session's last bookmark or use a constraint.
    const bookmark =
      request.headers.get("x-d1-bookmark") ?? "first-unconstrained";
    const session = env.DB01.withSession(bookmark);

    try {
      // Use this Session for all our Workers' routes.
      const response = await withTablesInitialized(
        request,
        session,
        handleRequest,
      );

      // B. Return the bookmark so we can continue the Session in another request.
      response.headers.set("x-d1-bookmark", session.getBookmark() ?? "");

      return response;
    } catch (e) {
      console.error({
        message: "Failed to handle request",
        error: String(e),
        errorProps: e,
        url,
        bookmark,
      });
      return Response.json(
        { error: String(e), errorDetails: e },
        { status: 500 },
      );
    }
  },
};

export default {
  async fetch(request, env, ctx): Promise<Response> {
    const url = new URL(request.url);

    // A. Create the Session.
    // When we create a D1 Session, we can continue where we left off from a previous
    // Session if we have that Session's last bookmark or use a constraint.
    const bookmark =
      request.headers.get("x-d1-bookmark") ?? "first-unconstrained";
    const session = env.DB01.withSession(bookmark);

    try {
      // Use this Session for all our Workers' routes.
      const response = await withTablesInitialized(
        request,
        session,
        handleRequest,
      );

      // B. Return the bookmark so we can continue the Session in another request.
      response.headers.set("x-d1-bookmark", session.getBookmark() ?? "");

      return response;
    } catch (e) {
      console.error({
        message: "Failed to handle request",
        error: String(e),
        errorProps: e,
        url,
        bookmark,
      });
      return Response.json(
        { error: String(e), errorDetails: e },
        { status: 500 },
      );
    }
  },
} satisfies ExportedHandler<Env>;

When using D1 without read replication, D1 routes all queries (both read and write) to a specific database instance in one location in the world, known as the primary database instance . D1 request latency is dependent on the physical proximity of a user to the primary database instance. Users located further away from the primary database instance experience longer request latency due to network round-trip time ↗.

When using read replication, D1 creates multiple asynchronously replicated copies of the primary database instance, which only serve read requests, called read replicas . D1 creates the read replicas in multiple regions throughout the world across Cloudflare's network.

Even though a user may be located far away from the primary database instance, they could be close to a read replica. When D1 routes read requests to the read replica instead of the primary database instance, the user enjoys faster responses for their read queries.

D1 asynchronously replicates changes from the primary database instance to all read replicas. This means that at any given time, a read replica may be arbitrarily out of date. The time it takes for the latest committed data in the primary database instance to be replicated to the read replica is known as the replica lag . Replica lag and non-deterministic routing to individual replicas can lead to application data consistency issues. The D1 Sessions API solves this by ensuring sequential consistency. For more information, refer to replica lag and consistency model.

A system with multiple read replicas located around the world improves the performance of databases:

• The query latency decreases for users located close to the read replicas. By shortening the physical distance between a the database instance and the user, read query latency decreases, resulting in a faster application.
• The read throughput increases by distributing load across multiple replicas. Since multiple database instances are able to serve read-only requests, your application can serve a larger number of queries at any given time.

By using Sessions API for read replication, all of your queries from a single session read from a version of the database which ensures sequential consistency. This ensures that the version of the database you are reading is logically consistent even if the queries are handled by different read replicas.

D1 read replication achieves this by attaching a bookmark to each query within a session. For more information, refer to Bookmarks.

Read replication can be enabled at the database level in the Cloudflare dashboard. Check Settings for your D1 database to view if read replication is enabled.

1. Go to Workers & Pages > D1 ↗.
2. Select an existing database > Settings > Enable Read Replication.

To create a session from any available database version, use withSession() without any parameters, which will route the first query to any database instance, either the primary database instance or a read replica.

const session = env.DB.withSession() // synchronous
// query executes on either primary database or a read replica
const result = await session
  .prepare(`SELECT * FROM Customers WHERE CompanyName = 'Bs Beverages'`)
  .run()

• withSession() is the same as withSession("first-unconstrained")
• This approach is best when your application does not require the latest database version. All queries in a session ensure sequential consistency.
• Refer to the D1 Workers Binding API documentation.

To create a session from the latest database version, use withSession("first-primary"), which will route the first query to the primary database instance.

const session = env.DB.withSession(`first-primary`) // synchronous
// query executes on primary database
const result = await session
  .prepare(`SELECT * FROM Customers WHERE CompanyName = 'Bs Beverages'`)
  .run()

• This approach is best when your application requires the latest database version. All queries in a session ensure sequential consistency.
• Refer to the D1 Workers Binding API documentation.

To create a new session from the context of a previous session, pass a bookmark parameter to guarantee that the session starts with a database version that is at least as up-to-date as the provided bookmark.

// retrieve bookmark from previous session stored in HTTP header
const bookmark = request.headers.get('x-d1-bookmark') ?? 'first-unconstrained';

const session = env.DB.withSession(bookmark)
const result = await session
  .prepare(`SELECT * FROM Customers WHERE CompanyName = 'Bs Beverages'`)
  .run()
// store bookmark for a future session
response.headers.set('x-d1-bookmark', session.getBookmark() ?? "")

• Starting a session with a bookmark ensures the new session will be at least as up-to-date as the previous session that generated the given bookmark.
• Refer to the D1 Workers Binding API documentation.

To see how D1 requests are processed by the addition of read replicas, served_by_region and served_by_primary fields are returned in the meta object of D1 Result.

const result = await env.DB.withSession()
  .prepare(`SELECT * FROM Customers WHERE CompanyName = 'Bs Beverages'`)
  .run();
console.log({
  servedByRegion: result.meta.served_by_region ?? "",
  servedByPrimary: result.meta.served_by_primary ?? "",
});

• served_by_region and served_by_primary fields are present for all D1 remote requests, regardless of whether read replication is enabled or if the Sessions API is used. On local development, npx wrangler dev, these fields are undefined.

With the REST API, set read_replication.mode: auto to enable read replication on a D1 database.

For this REST endpoint, you need to have an API token with D1:Edit permission. If you do not have an API token, follow the guide: Create API token.

curl -X PUT "https://api.cloudflare.com/client/v4/accounts/{account_id}/d1/database/{database_id}" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"read_replication": {"mode": "auto"}}'

const headers = new Headers({
  "Authorization": `Bearer ${TOKEN}`
});

await fetch ("/v4/accounts/{account_id}/d1/database/{database_id}", {
  method: "PUT",
  headers: headers,
  body: JSON.stringify(
    { "read_replication": { "mode": "auto" } }
  )
 }
)

With the REST API, set read_replication.mode: disabled to disable read replication on a D1 database.

For this REST endpoint, you need to have an API token with D1:Edit permission. If you do not have an API token, follow the guide: Create API token.

curl -X PUT "https://api.cloudflare.com/client/v4/accounts/{account_id}/d1/database/{database_id}" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"read_replication": {"mode": "disabled"}}'

const headers = new Headers({
  "Authorization": `Bearer ${TOKEN}`
});

await fetch ("/v4/accounts/{account_id}/d1/database/{database_id}", {
  method: "PUT",
  headers: headers,
  body: JSON.stringify(
    { "read_replication": { "mode": "disabled" } }
  )
 }
)

On the Cloudflare dashboard, check Settings for your D1 database to view if read replication is enabled.

Alternatively, GET D1 database REST endpoint returns if read replication is enabled or disabled.

For this REST endpoint, you need to have an API token with D1:Read permission. If you do not have an API token, follow the guide: Create API token.

curl -X GET "https://api.cloudflare.com/client/v4/accounts/{account_id}/d1/database/{database_id}" \
  -H "Authorization: Bearer $TOKEN"

const headers = new Headers({
  "Authorization": `Bearer ${TOKEN}`
});

const response = await fetch("/v4/accounts/{account_id}/d1/database/{database_id}", {
  method: "GET",
  headers: headers
});

const data = await response.json();
console.log(data.read_replication.mode);

• Check the read_replication property of the result object
  • "mode": "auto" indicates read replication is enabled
  • "mode": "disabled" indicates read replication is disabled

Currently, D1 automatically creates a read replica in every supported region, including the region where the primary database instance is located. These regions are:

• ENAM
• WNAM
• WEUR
• EEUR
• APAC
• OC

To see the impact of read replication and check the how D1 requests are processed by additional database instances, you can use:

• The meta object within the D1Result return object, which includes new fields:
  • served_by_region
  • served_by_primary
• The Cloudflare dashboard ↗, where you can view your database metrics breakdown by region that processed D1 requests.

There are some known limitations for D1 read replication.

• Sessions API is only available via the D1 Worker Binding and not yet available via the REST API.

To account for replica lag, it is important to consider the consistency model for D1. A consistency model is a logical framework that governs how a database system serves user queries (how the data is updated and accessed) when there are multiple database instances. Different models can be useful in different use cases. Most database systems provide read committed ↗, snapshot isolation ↗, or serializable ↗ consistency models, depending on their configuration.

Consider what could happen in a distributed database system.

1. Your SQL write query is processed by the primary database instance.
2. You obtain a response acknowledging the write query.
3. Your subsequent SQL read query goes to a read replica.
4. The read replica has not yet been updated, so does not contain changes from your SQL write query. The returned results are inconsistent from your perspective.

When using D1 Sessions API, your queries obtain bookmarks which allows the read replica to only serve sequentially consistent data.

1. SQL write query is processed by the primary database instance.
2. You obtain a response acknowledging the write query. You also obtain a bookmark (100) which identifies the state of the database after the write query.
3. Your subsequent SQL read query goes to a read replica, and also provides the bookmark (100).
4. The read replica will wait until it has been updated to be at least as up-to-date as the provided bookmark (100).
5. Once the read replica has been updated (bookmark 104), it serves your read query, which is now sequentially consistent.

In the diagram, the returned bookmark is bookmark 104, which is different from the one provided in your read query (bookmark 100). This can happen if there were other writes from other client requests that also got replicated to the read replica in between the two write/read queries you executed.

D1 read replication offers sequential consistency ↗. D1 creates a global order of all operations which have taken place on the database, and can identify the latest version of the database that a query has seen, using bookmarks. It then serves the query with a database instance that is at least as up-to-date as the bookmark passed along with the query to execute.

Sequential consistency has properties such as:

• Monotonic reads: If you perform two reads one after the other (read-1, then read-2), read-2 cannot read a version of the database prior to read-1.
• Monotonic writes: If you perform write-1 then write-2, all processes observe write-1 before write-2.
• Writes follow reads: If you read a value, then perform a write, the subsequent write must be based on the value that was just read.
• Read my own writes: If you write to the database, all subsequent reads will see the write.

You may wish to refer to the following resources:

• Blog: Sequential consistency without borders: How D1 implements global read replication ↗
• Blog: Building D1: a Global Database ↗
• D1 Sessions API documentation
• Starter code for D1 Sessions API demo ↗
• E-commerce store read replication tutorial