/Tracks

Two-week programme · Day 5 of 14

Leaking or replaced?

Study with AI

Part 1Basics

When two people can both work offline, both can change the same thing without knowing about each other. When they reconnect, the system has two answers and must do something.

This isn't only a technical problem. Whether to keep the newest value, keep both, or ask a person depends on what the data means. So you decide it field by field, with your product team.

Words you need today

Conflict
Two edits made from the same starting version, both wanting to be next.
Version
A number that goes up each time a value changes, used to detect that something changed.
Last-write-wins
Keep the newest value and discard the other. Simple, but it silently loses data.
Field-level merge
Comparing each field separately, so edits to different fields never conflict.
HLC
Hybrid logical clock, a timestamp that stays near real time and never runs backwards, unlike a phone's clock.
CRDT
A data type designed so that copies always merge to the same result. Useful for shared text, and overkill for forms.

The story

At 3:30 pm, while Priya was underground, her supervisor Dev took a call from a contractor who said valve 3 on j-84 had been swapped last week. Dev set it to replaced, which took the job to version 5. Priya's op says base version 4.

Who is right? Priya has seen water on the floor. Dev has a contractor's word. A newest-timestamp rule would pick Priya because her sync came later, or Dev because his edit did, and either way it silently deletes a safety fact.

On job j-81 the same thing happened with no drama: Dev changed the instructions, Priya changed an answer, and the server merged both because they touched different fields.

So at 4:14 pm Priya opens the amber badge and sees both values with their authors. She keeps leaking, adds a note, and the app writes a new op with base version 5. Version 6 reaches the server. Nothing vanished.

Part 2Today's question

When two people change the same thing offline, who wins?

By the end of this lesson, you should be able to:

  • Build a policy table for any entity
  • Explain why timestamps fail
  • Describe what the user sees during a conflict

Part 3In detail

Step by step

  1. The server gets Priya's edit. It says: set valve 3 to leaking, based on version 4.
  2. The server compares versions. Valve 3 is at version 5 now, because Dev changed it. Her base is older, so this field changed behind her back.
  3. The server looks up the policy. Valve 3 is a safety field, and its policy says ask a person.
  4. The server replies with both facts. It doesn't apply her edit. It returns conflict, plus Dev's value and version.
  5. The phone shows the choice. Job j-84 gets an amber badge. Tapping it shows both values and who set them.
  6. Priya decides. Her choice becomes a new edit based on version 5, which goes through normally.

Conflict policy is a product decision

The question isn't which algorithm is best. It is what a reasonable person expects. Two edits to different fields should both survive. Two edits to a safety field need a human. A read marker can use the latest value and nobody will mind.

Pick a policy per field

Write one table, field by field, and get product to sign it. Engineering owns the mechanism: per-field versions, so that unrelated edits never collide, and a 409-style reply that carries the current value.

Never trust the phone's clock

People change clocks and batteries die. Compare device timestamps and a phone set a day ahead wins every conflict. Where last-write-wins does fit, use a hybrid logical clock, which never runs backwards and stays near real time.

The code, explained

This is the real code, and our tests run it. Read each block, then the notes under it.

The decision for each field

This small function decides what happens when two people edit offline. The product team's table becomes code you can test.

FieldMerge.ktKotlin
1enum class Policy { LAST_WRITE_WINS, ADD_ONLY, ASK_USER }2 3/** A field on the server, and the job version at which it last changed. */4data class Stored(val value: String, val version: Int, val by: String)5 6sealed interface Merge {7  data class Apply(val value: String) : Merge8  data class Refuse(val current: Stored) : Merge9}10 11/** The product team's table, written down as code. */12fun policyFor(field: String): Policy = when {13  field.startsWith("photos") -> Policy.ADD_ONLY14  field == "instructions" || field == "read_marker" -> Policy.LAST_WRITE_WINS15  else -> Policy.ASK_USER                 // the safe default for anything new16}17 18fun merge(field: String, incoming: String, baseVersion: Int, current: Stored?): Merge {19  // Nobody touched this field since the user read it: j-81 takes this path.20  if (current == null || current.version <= baseVersion) return Merge.Apply(incoming)21 22  return when (policyFor(field)) {23    Policy.LAST_WRITE_WINS -> Merge.Apply(incoming)24    Policy.ADD_ONLY -> Merge.Apply("${current.value},$incoming")25    Policy.ASK_USER -> Merge.Refuse(current)            // j-84 takes this path26  }27}
27 linesUTF-8Kotlin
policyFor(field)
The policy table. Anything not listed asks the user, the safe choice for a new field.
current.version <= baseVersion
Nobody changed this field since the user read it, so apply it. That's why j-81 merged with no conflict.
Policy.ADD_ONLY
Photos: both people's additions are kept, because they never disagree.
Merge.Refuse(current)
Valve 3 on j-84 lands here. The server applies nothing, and returns what it holds.

Compiled and tested by npm run test:mobile.

Applying a pulled page

Changes and the new cursor commit together. That one choice removes a whole family of sync bugs.

SyncEngine.ktKotlin
1/** One transaction: the data and the cursor can never disagree. */2private fun applyPage(page: PullPage) = db.transaction {3  for (c in page.changes) {4    val job = jobs[c.jobId] ?: Job(c.jobId)5    val unsent = outbox.any { it.jobId == c.jobId && it.field == c.field }6    val inConflict = job.theirs?.field == c.field7    jobs = jobs + (c.jobId to job.copy(8      // Never overwrite something the user changed and the server has not settled.9      answers = if (unsent || inConflict) job.answers else job.answers + (c.field to c.value),10      version = maxOf(job.version, c.version),11    ))12  }13  cursor = page.next14}
14 linesUTF-8Kotlin
unsent || inConflict
Never overwrite something the user changed and the server hasn't settled. This is what keeps leaking on Priya's screen.
maxOf(job.version, c.version)
The version still moves forward, so her eventual choice is based on version 5.
cursor = page.next
Saved in the same transaction. If the phone dies, both the changes and the bookmark roll back, and the pull simply repeats.

Compiled and tested by npm run test:mobile.

What can go wrong

A phone's clock is a day ahead
Never compare device clocks. Use versions, or an HLC where last-write-wins fits.
Too many fields ask the user
People start tapping without reading. Reserve asking for fields where a wrong value does harm.
The user ignores the badge for days
Keep the job flagged, and show the supervisor it needs review. Don't resolve it automatically.

Decisions to defend

  • answers.valve_3: Ask the user, not last write wins. A safety fact must not disappear silently.
  • instructions: Last write wins, with an HLC, not ask the user. Only supervisors edit it, and a prompt would only annoy technicians.
  • photos: Add-only set, not replace. Two people adding photos never disagree.

Part 4Questions and answers

Answer out loud first, then open each one.

When would you need a CRDT rather than a field merge?
When several people edit the same text or list at once and every keystroke must merge. For a form with fields, per-field versions are far cheaper and easier to explain.
Who should choose the policy for a new field?
Product chooses the rule, because it decides what users experience. Engineering builds the mechanism and points out where a rule can't be met.

Part 5Going further

The three questions, for today

Where does it live?
Both values, side by side, on the conflict row and on the server.
Who runs the next step?
A person. Priya opens the badge and decides.
What if it runs twice?
Her choice is a new op with its own ID and base version 5.

Practise

Optional reference pages

You don’t need these for the lesson. They hold more detail if you want it later.

You have seen the server's answer from the phone's side. Tomorrow you go inside the server, and meet the 5 million phones that come with it.

← Day 4: 2.1 seconds at 4:12 pm

AI Engineer · Day 5 of 14