Clocks lie. Ordering isn't about time. Causality is what actually matters. And sometimes, even that isn't enough.

So, if we can’t rely on clocks, how do systems keep track of order at all?

They invent their own time.

Not physical time. Logical time.

We stop looking at the sun and start counting.

Logical Clocks

A logical clock doesn’t try to tell you what time it is.

It doesn’t know about seconds. It doesn’t care about time zones. It doesn’t try to match the real world.

A logical clock answers one question: What is the order of events?

Instead of measuring time, it tracks relationships. Instead of counting seconds, it counts causality. Instead of measuring the universe, it measures influence.

Every process keeps a counter. When something happens locally, it increases. When a message is sent, the current value goes with it. When a message is received, the counter updates to reflect what it just learned.

That’s it.

Logical clocks track how information flows through the system. They are a map of who talked to whom and in what order.

Why this works

Remember happens-before?

If A → B, we want B’s timestamp to reflect that it came after A.

Logical clocks guarantee exactly that.

No GPS. No quartz crystals. No atomic time. Just counters and message exchange.

They don't solve every problem. They won't tell you what time to meet for lunch, and they won't give you a global "now."

But they solve the right problem: detecting order when it exists.

And there isn’t just one kind.

Lamport clocks use a simple counter to preserve causal order. Vector clocks detect concurrency. Hybrid clocks blend physical and logical time.

Each exists because different systems need different trade-offs.

The simplest one, the one that started it all, is just a counter.

Lamport Clocks: The Simple Counter

Leslie Lamport's insight was that if we can't trust seconds, we can at least trust counting.

Lamport clocks are simple. Each process keeps a single integer counter. That’s it.

With that one number, we can preserve causal ordering across a distributed system.

Lamport clocks follow three simple rules.

1. The Local Rule: Every time a machine does something (writes to a disk, processes a request), it increments its own counter by 1.

2. The Sending Rule: When a machine sends a message, it attaches its current counter value to that message.

3. The Receiving Rule: When a machine receives a message, it takes the higher of its own counter and the received one, then adds 1. That's just: max(local_counter, received_counter) + 1.

That’s the whole algorithm.

Lamport clock update on message receive: Process B advances its counter to preserve causality after receiving a message from Process A.

Every event gets a logical timestamp.

Lamport clocks guarantee that if A → B, then timestamp(A) < timestamp(B). If I am at "Logical Time 5" and I send you a message, whatever you do next must be at least "Logical Time 6."

Causality is preserved.

No clocks drifting. No NTP. No leap seconds. Just integers increasing in response to information flow.

And for many distributed systems, that’s enough.

The catch

Lamport clocks preserve causality. But they do not capture it completely.

If you see that timestamp(A) < timestamp(B), that does not necessarily mean: A → B.

Because Lamport clocks collapse everything into a single number, and a single number forces a comparison even when one doesn't exist.

Two concurrent events will still receive different timestamps. One smaller, one larger. But that ordering might be arbitrary.

Lamport clocks give us: If A → B, then LC(A) < LC(B)

But they do not give us: If LC(A) < LC(B), then A → B

The arrow only goes one way.

Causality implies ordering. Ordering does not imply causality.

Lamport clocks are lossy. They tell you what couldn’t have happened. But they can't prove what did.

That asymmetry matters.

Turning Lamport clocks into total order

If we take Lamport timestamps and add a tie-breaker. For example, process ID, we can force a total order.

For any two events:

• Compare their timestamps.

• If equal, compare process IDs.

Now every pair of events is comparable. We’ve flattened the causal web into a single sequence.

But that total order is artificial. It respects causality where it exists, but it also imposes order where none existed.

When it’s enough

Lamport clocks are simple, lightweight, cheap to maintain, and easy to implement.

They’re often enough when:

• You need to preserve causality.

• You want to impose a deterministic order.

• You don’t need to detect concurrency explicitly.

But sometimes you don’t just want to know which event has the smaller number. You want to know whether two events were independent.

Did two users edit the same file without seeing each other’s changes? Did two replicas diverge without knowing it?

A single counter can’t answer that.

Because to detect independence, you need more than order.

You need ancestry.

Vector Clocks: Tracking the Ancestry

Lamport clocks gave us order. But they did it by collapsing everything into a single number.

That number preserved causality. But it erased independence.

If two events were concurrent, Lamport would still force one to look “earlier.”

Sometimes that’s fine. But sometimes, you don’t just want order. You want to know: “Were these two events independent?”

To answer that, you need more than a single counter.

You need memory.

Instead of one number per process, each process keeps a vector: one slot for every process in the system. If there are three processes- A, B, and C, each process maintains: [A_count, B_count, C_count]

Each position tells you how many events from that process have been seen. Not just locally. Globally, from their perspective.

The rules follow the same shape as Lamport, but trade simplicity for information.

1. The Local Rule: Increment only your own position in the vector. If Machine A does something, it only increments its own slot in the vector: [A: 1, B: 0, C: 0].

2. The Sending Rule: Attach your entire vector to the message. You're not just sending the time. You're sending your worldview.

3. The Receiving Rule: For every position, take the maximum of your current value and the received value. Then increment your own slot. When you receive a message, you merge histories.

Process B merges A’s history and advances, while Process C evolves independently. Revealing causality and concurrency.

We can now compare events. That gives us something Lamport couldn't.

Event A happened-before B if every element in A’s vector is ≤ B’s vector, and at least one is strictly less.

If neither vector is less than the other, they are concurrent. Not maybe. Provably independent.

Example:

A: [2,1]
B: [1,2]

Each is greater in one dimension. Neither contains the other. That is concurrency.

A vector clock is not a timestamp. It’s a summary of ancestry.

When I receive your vector, I don’t just learn when something happened.

I learn who influenced you.

If my vector is: [A:5, B:2, C:8], I am saying: I have seen 5 events from A, 2 from B, 8 from C.

It’s a compact map of causal history.

Lamport gave us a line. Vector clocks give us the tree.

The "Shopping Cart" Problem

Imagine you are shopping:

1. You add a book to your cart on your phone.

2. Your phone loses signal or goes offline.

3. You add a shirt on your laptop.

These two updates happened on different machines. They never saw each other.

With Lamport clocks, the system would assign different numbers and pick a “winner” using an arbitrary tie-breaker.

Oops. Your book disappears.

The system “resolved” a conflict you didn't even know you had by deleting data.

But with Vector Clocks, the database looks at the ancestry and sees the truth: these updates are concurrent.

It doesn’t guess. It keeps both. It says: “I have two independent versions of this cart. I’m going to keep both, and the next time the user is online, I’ll let them merge the two.”

Two replicas diverge without communication; on merge, their concurrent updates are preserved rather than forced into an order.

This is exactly how systems like Amazon Dynamo or Riak detect conflicts. They don't try to force a "fake" order on independent events. They detect concurrency first. Resolution comes later.

The Trade-off

Vector clocks are expressive. But they’re also expensive.

They require:

• O(n) metadata per event. The size of the timestamp grows with the number of machines. If you have 1,000 nodes, every single message has to carry 1,000 integers.

• Larger messages. The full vector travels with each message.

• Knowledge of all participants.

• Complexity of membership management when nodes join/leave.

That does not scale casually.

Vector clocks are the “don’t lose data” tool. You pay in metadata to preserve independence.

When they make sense

Vector clocks are the right tool when:

• You need to detect concurrent updates.

• You want to resolve conflicts intelligently.

• You care about causality and independence.

They show up in systems like version control, distributed databases, and conflict-resolution protocols.

But they are heavy. And sometimes,  we still want something close to real-world time.

Just not blindly.

The Takeaway

We started this journey by saying: Clocks lie.

They do. But understanding how they lie is what lets us design around them.

Physical clocks approximate reality. Logical clocks preserve causality. Vector clocks detect independence.

We can now track causality and detect independence without ever looking at a wall clock.

But in doing so, we've lost the real world.

Causality tells you what came before what. It tells you nothing about when.

So what happens when the real world matters? When “what happened before what” isn’t enough?

Can we bring the clock back, without breaking everything we just fixed?

We took the most fundamental tool in our toolkit, the timestamp, and discovered it’s just a local guess wearing a fancy suit.

But this realisation leads us to a more interesting question: If we can't trust the clock to tell us the order of events, what can we trust?

Before we answer that, here’s the twist: Ordering was never about time in the first place.

The "WhatsApp" revelation

Think about the group chat on your phone.

If I send a message asking, "Who wants pizza?"
And you reply, "I do!"

The order matters.

If your reply arrives "before" my question, the conversation makes no sense.

But notice something important. We don’t care what time those messages were sent.

We care that:

• your reply came after my question,

• and that it depended on it.

When we say we want to order events, we don’t actually care about wall-clock time. We care about something much more specific: “Did one event have the chance to influence another?”

We are not asking:

• “Which happened at 09:00:01?”

• “Which machine had the smaller number?”

But:

• Did event B know about event A?

• Could B have seen A’s effects?

• Was there a dependency between them?

This is a question about causality, not clocks.

A distributed example

Imagine two machines. Machine A writes a value. Machine B later reads that value.

Even if both clocks are wrong, one thing must be true: The write happened before the read, in a way that mattered.

Not before in universal time. Before in causal reality.

The read depends on the write. That dependency is the ordering.

A write on Machine A causally precedes a read on Machine B via message passing.

Now imagine something different:

• Machine A updates User 1’s profile.

• Machine B updates User 2’s profile at roughly the same time.

These two events may happen milliseconds apart. Or at the exact same time.

It doesn’t matter.

They are independent.

Neither could possibly have influenced the other.

Trying to force them into a single timeline doesn’t add clarity. It adds constraints.

Time answers the wrong question

Physical time tries to answer: “When did this happen relative to the universe?”. Distributed systems don’t need that. They need to answer: “Which events could have affected which other events?”

Two events can:

• have perfectly ordered timestamps,

• look clean and sequential,

• and still be completely unrelated.

And two events can:

• look simultaneous,

• have misleading timestamps,

• and be tightly linked by cause and effect.

Time can’t tell the difference. But the relationship between the events can.

The shift

This is the part where the ground finally stops moving. Soon, you will understand that ordering isn’t about placing everything on a single global timeline. It’s about identifying dependencies.

Some events are connected. Others are independent.

• If event A caused event B, that relationship matters.

• If A and B are unrelated, forcing an order between them is artificial. And, sometimes expensive.

This is why distributed systems don’t start with clocks.

They start with a rule: If A could have influenced B, then A must come before B.

That’s the entire foundation.

And this rule has a name.

Happens-Before: A language for causality

It’s not a timestamp. It’s not a clock. It’s a relationship.

Back in 1978, Leslie Lamport made a simple observation that would fundamentally change how we reason about distributed systems.

We don’t actually need clocks to determine order.
We need to track how information flows.

He introduced a concept called happens-before. The idea is almost embarrassingly simple: If Event A could have influenced Event B, then A happened before B.

Lamport wrote it like this: A → B . Which simply means: A happens-before B.

But forget the notation for a moment. The intuition is more important. There are only three ways we can say, with certainty, that one event came before another, without looking at a clock.

Order inside a single process

If a program executes:

• Step 1

• Then, Step 2

Step 1 happened before Step 2.

Not because of timestamps. Because programs execute instructions in sequence.

Within a single process or thread, order is clear.

If a and b are events in the same process, and a occurs before b, then a → b

Message passing

Now, imagine I send you an email.

You read it. You reply.

Your reply could not exist without my original email.

Even if my server thinks it’s 09:00 and yours thinks it’s 08:00, the reply depends on the message.

The data flow proves the order.

Cause precedes effect.

If a is the sending of a message and b is the receipt of that same message, then a → b

A message sent from Machine A travels through the network and is received by Machine B, establishing a causal relationship where the receive depends on the send.

Transitivity

This is the subtle part that actually makes the whole system work. It’s the "friend of a friend" rule for data.

If:

• A happened before B,

• and B happened before C,

Then A happened before C.

Even if A and C never directly communicated.

If a → b and b → c , then a → c .

This chaining of influence is what lets distributed systems reason about entire histories without ever asking, “What time was it?” If we can trace the path of messages, we don't need to guess about the timing. The relationship is set in stone.

What happens-before does not say

Happens-before does not force every event into a single timeline.

If two events:

• occur on different machines,

• never exchange messages,

• and never influence each other,

Then neither happened-before the other.

They are concurrent.

Our instinct is to ask: which one came first? Which one “won”? Which one happened at 09:00:00.001?

But in a distributed system, concurrency isn’t an error or a calculation we haven’t finished yet.

It’s a fact. It’s the system admitting: "These two events are strangers. Their order doesn't exist."

Causality

We can stop dancing around it. What we’ve really been talking about this whole time is called Causality.

If event A could have influenced event B, then A is a cause and B is an effect.

Happens-before is simply the formal way of capturing causal relationships. It’s a record of potential influence.

If there is no causal relationship between two events, then neither happened-before the other. The two events are independent and therefore concurrent.

And that distinction, causal vs independent, is the key to everything that comes next.

Why this changes everything

Here’s what changed. We replaced “What time is it?” with “What could this event have known?”

We traded timestamps with relationships.

And suddenly, ordering doesn’t depend on synchronised clocks, atomic hardware, or GPS satellites.

It depends on causality.

That’s the foundation.

Partial Ordering

Once you accept causality as the foundation, a trade-off appears: Not every pair of events can be compared.

In a world governed by happens-before, some events have a clear relationship:

• A caused B

• B depends on A

• Information flowed from A to B

But other events have no relationship at all.

They happened on different machines. They never exchanged messages. They never influenced each other.

In this world, trying to say which one “came first” is meaningless.

Not everything needs to be ordered

In everyday life, we’re used to total order. We assume everything in the universe is waiting in a single line.

• Monday comes before Tuesday.

• Page 3 comes before Page 4.

• 09:00 comes before 09:01.

Everything fits neatly on a single axis.

However, distributed systems don’t work that way.

Instead of a single straight timeline, you get something more like a web. Some events are connected. Others happen independently.

Think of it like lineage. You can say your grandfather came before your father. But can you say your friend came before your cousin?

Of course not. They are independent.

If A → B, then A must come before B.

But if A and C are unrelated, there is no rule that says one must come before the other.

That’s a partial order.

A partial order of events: arrows show causal relationships, while isolated nodes represent independent events with no defined ordering.

What “partial” really means

Partial doesn’t mean “incomplete” or “approximate.” It means: Only order what must be ordered.

If causality demands that A precede B, respect it. If causality says nothing about A and C, leave them alone.

This is not a limitation. It’s precision. It’s freedom.

Concurrency

When two events have no happens-before relationship, we call them concurrent.

Contrary to popular belief, concurrency does not mean “at the same time.” It simply means there is no causal link between them. They could have happened seconds apart. They could have happened on opposite sides of the planet. They could even have identical timestamps.

If neither could have influenced the other, they are concurrent.

That’s not ambiguity. That’s independence.

Why partial ordering is powerful

Partial ordering is the "cheat code" of distributed systems. It preserves exactly what matters:

• It respects causality.

• It avoids inventing artificial order.

• It allows independent events to proceed freely without waiting for other events.

This is why distributed systems prefer partial order whenever possible.

It maximises concurrency and minimises coordination. You don't need a "meeting" between two servers to decide who goes first if their work doesn't overlap.

But sometimes… we don't get that luxury.

Sometimes, we need a single line.

We need to decide who gets the last ticket to the concert or who spent the last dollar in a bank account.

In those moments, independence becomes a liability.

We need a way to take our causal web and flatten it into a single, reliable order.

Total ordering

Partial order is elegant. It respects causality. It avoids unnecessary constraints. It lets independent work stay independent. It stays close to reality.

But sometimes reality isn’t enough.

Sometimes we don’t just want to respect causality; we want everyone to agree on the exact same sequence of events. Even when causality doesn’t force one.

That’s total ordering.

What total order really means

A total order removes the "strangers" from the equation.

For any two events A and B, one must come before the other.

There are no unrelated events. No concurrency left undecided. No floating nodes in the graph.

Every event, no matter where it happened on the planet, gets a seat in a single, global line.

Why would we ever want that?

Because agreement is powerful.

Imagine a distributed database with three replicas. Two clients try to buy the last pair of sneakers at the same time.

• Replica 1 processes Buyer A, then Buyer B.

• Replica 2 processes Buyer B, then Buyer A.

Both replicas followed the rules of causality. Both are internally consistent.

And yet, they now disagree on who owns the shoes.

Same inputs. Different histories. Different state.

That’s a nightmare for a bank account, an inventory system, a file system, or a blockchain.

Nothing here is broken. Causality told us how the events were related, but it didn't give us a tie-breaker.

The system did exactly what we asked.

Which is the problem. It just wasn’t enough.

Total ordering forces every replica to say: “We will all process events in this exact order.”

That shared order becomes the system’s truth.

The cost of forcing order

Here’s the caveat: You don’t get total order for free.

If two events are concurrent, meaning causality says nothing about their order, the system has to invent one.

And inventing order requires:

• communication (talking to other machines).

• coordination (agreeing on a winner).

• waiting (latency).

• failure handling.

You pay for that agreement with latency and complexity.

This is why high-performance systems try to stay in the world of partial order for as long as possible, reaching for total order only when the business logic demands it.

Partial order is natural. Total order is negotiated.

From web to line

Partial order looks like a web. Total order flattens that web into a line.

Flattening a partial order into a total order: causal relationships are preserved, but independent events are placed arbitrarily so every machine can agree on a single sequence.

Some edges in that line are required by causality.

If A caused B, A must stay in front. But for independent events, the order is arbitrary. It’s chosen so that every machine can look at its history and see the same truth.

And that arbitrariness is fine as long as everyone chooses the same one.

Why this matters

You don’t choose one because it’s prettier. You choose based on what your system must guarantee.

Partial order maximises concurrency.

Total order maximises agreement.

If you need:

• strong consistency

• deterministic replay

• identical replicas

You need total order. And total order doesn’t emerge naturally.

It has to be negotiated.

That negotiation is where coordination protocols live. That’s where consensus begins.

Takeaway

Clocks can’t be trusted.

And ordering was never about time.

But systems still need to agree.

So how do you build a single global sequence… without relying on time?

That’s where the real work begins.

Right?

Wrong.

In distributed systems, this intuition is wrong in ways that are subtle, surprising, and dangerous.

When I first started learning about distributed systems and the concept of "ordering", I had no idea how shaky the ground beneath us actually was. I assumed a timestamp was a fact.

I was wrong.

It turns out, time is less of a universal constant and more of a local, unreliable opinion.

Today, we’re going to learn the lesson that every distributed system eventually learns the hard way: Clocks lie. Sometimes they drift apart like two childhood friends losing touch. Sometimes they jump backwards without warning. And sometimes, they agree just long enough to trick you into trusting them, right before they break assumptions you didn’t even know you were making.

This series is about why clocks, as we know them, lie and why distributed systems had to invent entirely new ways to reason about order.

The lie of physical time

So what does it actually mean to say that clocks lie?

After all, your laptop has a clock. Your phone has a clock. Every machine has a clock. We have protocols like NTP (Network Time Protocol) to keep them in sync, and we even have atomic clocks in GPS satellites keeping time with absurd, terrifying precision.

Surely, "09:00:00" is "09:00:00" everywhere.

The problem isn’t that clocks are useless. The problem is that we quietly ask them to do more than they can safely promise.

We ask them to be a source of truth for the order of the universe, but they were only ever designed for human-facing timestamps, for example, to tell us when to go to lunch.

To understand where things start to break, we need to look at what “time” even means in a distributed system. In particular, we need to understand why physical time is a shaky foundation for any system that cares about the truth and about order.

That’s where we’ll start.

What we expect from physical time

When we look at a timestamp, we’re making three huge, silent assumptions:

1. Precision: The clock is accurate down to the millisecond.

2. Monotonicity: The clock only moves forward.

3. Universality: Other machines agree with us.

In a distributed system, all three are routinely violated.

Assumption #1: Precision implies Accuracy

We tend to think that precision and accuracy are the same thing.

They aren't.

Precision is how many digits the clock gives you.
Accuracy is whether those digits are telling the truth.

When we see a timestamp like 09:00:00.123, it feels definitive. Three decimal places. Millisecond resolution. That number looks confident. Scientific. Trustworthy.

But here’s an uncomfortable truth: precision is not the same thing as accuracy.

Precision creates the illusion of certainty. Accuracy determines how close you are to the truth.

Your system can happily print timestamps with nanosecond resolution while being wrong by tens of milliseconds - or more.

Precision is just the ability to say a number with a straight face; accuracy is the ability to actually be right.

Precision is easy. Accuracy is hard.

Most machines measure time by counting ticks from a local hardware clock (usually a tiny vibrating quartz crystal). Those ticks are frequent, so it’s easy to divide them up and label events with fine-grained timestamps.

That’s where the illusion comes from.

The clock says: 09:00:00.123 . What it really means is closer to: “As far as I can tell, it’s somewhere around 09:00, give or take.” The extra digits don’t make the clock more accurate. They just make it look more confident while lying to your face.

A precise timestamp is just a single point hiding a range of possible truths.

The hidden gap between clocks

Now imagine two machines sitting next to each other in the same data centre.

Both report millisecond-precise timestamps.
Both are “synchronised”.
Both look fine in dashboards and logs.

And yet, one might be 15ms ahead of the other.

Nothing is broken. No alarms. This is normal.

From the system’s point of view, both clocks are working as designed. From your point of view, the timestamps are already lying to you, just quietly.

Why this breaks ordering

If two events happen close together on different machines, precision gives you a false sense of certainty. You might look at this and think the order is obvious:

Machine A: 09:00:00.120
Machine B: 09:00:00.118

Two timestamps can suggest an order, even when reality makes that order unknowable.

You would assume the event on Machine B happened first. But if Machine B’s clock is lagging by just 5ms, then in reality, A actually happened first.

The problem isn’t that the clocks are "sloppy." The problem is that “accurate to the millisecond” was never a promise they could safely keep. When we rely on these numbers for ordering, we implicitly assume: “If two times differ by a few milliseconds, the order is real.”

In a distributed system, that assumption is a gamble, and the house always wins.

Precision gives us numbers. Accuracy gives us truth. Physical clocks mostly just give us the numbers.

Assumption #2: Monotonicity

In the real world, time is relentless. It is a one-way trip. You can’t unspill milk, and you can’t un-tick a second. We build our software logic on this basic law of the universe: Forward-ever, backwards-never!

But there’s a quieter, more dangerous assumption hiding behind precision: the belief that once time moves forward, it stays there.

We assume that if an event happened at 09:00:01, then nothing in the future can ever happen at 09:00:00. This feels so fundamental that we rarely question it. After all, time doesn’t go backwards.

We build systems on that assumption.

On a single machine, under ideal conditions, this mostly holds. In real systems, it absolutely does not.

Time can go backwards

Let’s start with the most unsettling case.

Clocks can, and do, move backwards. This happens whenever the system decides its idea of "now" is wrong and tries to fix it. For example:

• a clock synchronisation correction

• a virtual machine pause and resume

• a leap second adjustment

• a manual clock change by an administrator

The system doesn’t rewind history. It simply updates its opinion of “now”.

From the clock’s point of view, that looks like this:

09:00:01.500
09:00:01.200   <- time went backwards!

Nothing crashed. Nothing is “broken”. But any code assuming monotonic time just lost its footing.

Wall-clock time can jump backwards, breaking the assumption that time always moves forward.

When time doesn’t go backwards, it “slews”

To avoid sudden jumps, many systems try to be clever. Instead of stepping the clock backwards, they slow it down until reality catches up. Time still moves forward, just not at the rate you expect.

This is called slewing.

From the outside, everything looks fine. Timestamps are increasing. But durations stretch. Timeouts fire late. Delays feel inconsistent. Imagine doing a plank for 1 minute, and it feels longer!

From the system’s point of view:

• One second is no longer one second

• “Now” is a moving target

Time didn’t jump. It quietly changed speed. It just changed the rules of physics while you weren't looking.

When clocks don’t jump, they drift - silently stretching time instead.

The Great Divide: Wall-Clock Time vs. Monotonic Time

At this point, most operating systems quietly admit defeat.

They expose two clocks:

1. Wall-clock time: Meant for humans. Can jump. Can slew. Can go backwards. It’s for knowing when to send that ”Happy New Year” text.

2. Monotonic time: Meant for measuring durations. Never goes backwards. Has no date. No timezone. No meaning outside the process. It’s just a counter that goes up. It doesn’t know if it is Tuesday or what year it is. It just knows it’s been ticking since the machine woke up.

Monotonic time answers: “How much time has passed?” “How long did that take?”

Wall-clock time answers: “What time do humans think it is?”

Confusing the two is one of the easiest ways to break a system.

Wall-clock time tells you what time it is. Monotonic time tells you how time has passed.

Why this breaks ordering

Ordering assumes a simple rule: Later events have later timestamps. But once time can jump or slow down, that rule collapses. An event can:

1. Suddenly appear to happen before the thing that caused it

2. Get a timestamp earlier than a previous event

3. Violate “read after write” assumptions

The problem isn’t bad engineering. It’s that monotonicity was never a guarantee that wall-clock time could make.

Once you notice this, it’s hard to un-see it. Precision lied quietly. Monotonicity lies loudly.

If time can’t even be trusted to move forward on one machine, the idea that multiple machines could ever agree on a global “now” should start to feel very suspicious.

Which brings us to our final, and most impossible, assumption.

Assumption #3: Universality

We’ve already faced two uncomfortable truths: clocks aren’t as accurate as they look, and even on a single machine, time doesn’t reliably move forward.

And yet, there’s usually one final assumption quietly holding our sanity together: The Backup Plan. We assume that even if my local clock is a little bit of a mess, the system as a whole is at least roughly on the same page. We believe in a shared, global “now.”

This assumption feels reasonable. We have GPS satellites, atomic clocks, and NTP (Network Time Protocol) servers. Surely, modern infrastructure can keep a handful of machines within a few milliseconds of each other. That should be good enough, right?

It isn’t.

There is no global clock

In a distributed system, "Now" isn't a fact. It is a local, subjective experience. Every machine is an island with its own clock.

Those clocks:

• Tick at slightly different speeds (thanks to factors such as manufacturing differences, temperature, humidity and air pressure)

• Drift apart over time

• Are corrected asynchronously

• Receive updates over unreliable networks

There is no central authority whispering the true time into every CPU at the same instant. What we call “clock synchronisation” is really just a best-effort agreement that is being continuously negotiated and violated the moment it’s made.

Clocks can be “in sync” and still disagree.

Synchronisation doesn’t mean agreement

Protocols like NTP don’t say: “All machines now have the same time.” They say something like “Your clock is probably within some bounds of real time.”

That bound varies and is often unknown at the moment you read the clock. It can be temporarily blown out by a spike in network traffic or a busy CPU. Two machines can both be perfectly "healthy" and "in sync" and still disagree by 20 milliseconds. In a world where a database can process thousands of transactions in a single millisecond, a 20ms gap is a canyon where assumptions go to die.

The "God's Eye View" is a Fantasy

Even if clocks were perfect (they aren’t), messages are not.

When Machine A sends a message to Machine B:

• it leaves at some time

• arrives later

• after an unpredictable delay

By the time B sees the message, A’s idea of “now” may already be outdated.

So when B compares:

• “the time in the message”

• with “my current time”

It’s comparing two local opinions, separated by uncertainty and delay.

Why this breaks ordering completely

Ordering is supposed to answer a simple, vital question: Did this happen before that? Did event A happen before event B in a way that mattered?

Timestamps look like they answer this perfectly. Smaller number first, bigger number later. Case closed.

But once you accept that clocks aren’t precise, time isn’t monotonic, and machines don’t agree on “now,” you realise that timestamps have stopped answering the question you’re actually asking.

They’ve started answering a much less useful question: “Which machine had a slightly higher number on its local, unreliable crystal at the moment this happened?”

When you rely on those numbers, you're building on sand. Two events can have timestamps that suggest a clear order, while the physical reality of the system supports the exact opposite.

And the worst part? There is no way to tell which is correct.

At that point, ordering based on time isn't just fragile, it's logically meaningless. It’s like trying to measure the distance between two moving cars using a rubber band.

This isn’t a bug. This isn’t a misconfiguration. This is just physics, networks, and independent machines doing exactly what they were built to do.

Takeaway

Precision failed quietly.

Monotonicity failed loudly.

Universality never existed in the first place.

There is no shared global time in a distributed system, only local clocks, imperfectly stitched together. And once you accept that, you get to the realisation that Time is the wrong tool for reasoning about order.

Which raises the obvious question: If we can’t safely use time to order events, what on earth do we do instead?