Google Cloud Firestore Transactions

I’ve been seeing a lot of misinformation from LLMs about the transaction model that Google Cloud Firestore exposes. I’m not sure if that’s because it’s tied to the Firebase Realtime Database or Google Cloud Datastore, which historically have had limited transactions in one way or another. If you read the Firestore whitepaper you can see that it’s built upon Spanner. This means that Firestore transactions under the hood are essentially just Spanner transactions and they inherit full serializability and external consistency from Spanner. Serializability is a property of transactions which means that the database executes the transactions in a manner as if they were done sequentially. However, in reality it would be super slow to do this in practice so transactions that don’t touch the same rows/documents end up executing concurrently.

Database isolation levels can feel like a very academic subject for a full stack developer, but if you aren’t using serializable transactions and your app is any more complex than a trivial CRUD app, you probably have some bugs unless you’re always very careful.

A motivating example

Let’s draw out the power of serializable transactions with an example derived from a question that spawned this blog. Let’s say a user in our application can have at most 100 projects. The naive version of this in most databases looks like:

1. SELECT COUNT(*) their projects
2. If under the limit, INSERT a new one

Under weaker isolation levels (which most databases default to), two concurrent requests can both read a count of 99, both insert, and now the user has 101 projects. Neither transaction touched the same row, so nothing conflicts. This anomaly is called write skew, and it can be a super gnarly bug to track down.

In Firestore and other databases that support serializable isolation levels, you can just write the obvious code:

await db.runTransaction(async (tx) => {
  const projects = await tx.get(
    db.collection("projects").where("owner", "==", uid)
  );
  if (projects.size >= 100) {
    throw new Error("too many projects");
  }
  tx.create(db.collection("projects").doc(), { owner: uid, name });
});

No advisory locks, no denormalized counter document that you have to keep in sync - the obvious code is also the correct code.

Watch it happen

We can actually write a small example that interleaves two transactions in a single process to trigger the conflict. We inject a synthetic barrier so that the first transaction stalls after its query, and the second one reads the same range before the first commits. Here’s a complete script you can run against the Firestore emulator:

import { Firestore, Transaction, Query } from "@google-cloud/firestore";

const db = new Firestore({ projectId: "demo" });
const LIMIT = 3; // small limit so the script is quick
const projects = (): Query =>
  db.collection("projects").where("owner", "==", "tyler");

async function addProject(
  name: string,
  holdUntil?: Promise<void>,
  onRead?: () => void
) {
  let attempts = 0;
  await db.runTransaction(async (tx: Transaction) => {
    attempts++;
    const snapshot = await tx.get(projects());
    console.log(`[${name}] attempt ${attempts}: sees ${snapshot.size} projects`);
    onRead?.();
    await holdUntil; // hold the transaction open after reading
    if (snapshot.size >= LIMIT) throw new Error(`[${name}] limit reached`);
    tx.create(db.collection("projects").doc(), { owner: "tyler", name });
  });
  console.log(`[${name}] committed after ${attempts} attempt(s)`);
}

async function main() {
  // Seed LIMIT - 1 projects, so exactly one slot remains.
  for (let i = 0; i < LIMIT - 1; i++) {
    await db.collection("projects").add({ owner: "tyler", name: `seed-${i}` });
  }

  // The barrier: tx1 reads first, then holds its transaction open
  // until tx2 has read the same range. Both see one slot left.
  let tx1HasRead!: () => void;
  let tx2HasRead!: () => void;
  const tx1Read = new Promise<void>((r) => (tx1HasRead = r));
  const tx2Read = new Promise<void>((r) => (tx2HasRead = r));

  const tx1 = addProject("tx1", tx2Read, tx1HasRead);
  await tx1Read;
  await addProject("tx2", undefined, tx2HasRead);
  await tx1;
}

main().catch((e) => console.error(`${e.message}`));

Here’s the output:

[tx1] attempt 1: sees 2 projects
[tx2] attempt 1: sees 2 projects
[tx1] attempt 2: sees 2 projects
[tx1] committed after 2 attempt(s)
[tx2] attempt 2: sees 3 projects
[tx2] limit reached

Both transactions read a count of 2 - one under the limit. If the database let both commit, we’d end up with 4 projects. Instead, each transaction’s commit conflicts with the range the other one scanned, so the database picks a loser and the SDK automatically re-runs its callback once the winner is out of the way. On its retry the loser sees 3 projects and throws. (Which transaction wins can vary between runs - the invariant holds either way.) This is also why the docs tell you to keep side effects out of the transaction callback - it can be re-executed when there’s a conflict.

But how?

What kind of magic enables the database to detect these conflicts? Range locks (also known as gap locks)!

Every database query is a scan over an index or table. where("owner", "==", uid) is a scan of a contiguous range of the owner index: all the entries with key prefix uid. When a transaction runs that query, the underlying storage takes a lock on that key range, not just on the entries that happened to exist at the time.

Now when another transaction inserts a new project for the same user, that insert has to write a new entry into the locked range of the index. That write conflicts with the range lock, so one of the two transactions loses and gets retried. This is the classic solution to the phantom read problem: you can’t lock a document that doesn’t exist yet, but you can lock the place in the index where it would appear.

A nice consequence of this is that it works for any query shape - as well as aggregation queries. If you swap the query in the transaction for tx.get(projects().count()) you don’t transfer any documents over the wire, but you keep the exact same guarantee - the count still executes as a scan over the same index range, and the scan is what takes the lock.

There is a trade off, however: it is more expensive in terms of the memory used to track these ranges, and also the compute to check all the overlapping ranges at commit time. This is why most databases don’t default to serializable transactions, or impose other limits like Firestore’s transaction timeouts. So in some aspects you’re trading off performance for safety and ergonomics.

Wrapping up

It’s actually incredibly freeing as a developer to be able to work in purely serializable transactions. In terms of consistency, you only have to worry about things outside of transactions. Within a transaction, you can always feel confident that your database will protect the invariants of your logic based on the reads and writes you perform. So next time you choose a database, I strongly encourage you to research the isolation it provides. As TigerBeetle says: “give me strict serializability or give me death”!