The right time.
Not just arrival time.
Watermarks track event-time progress across every input. A window closes only when the slowest channel says it is safe.
channel 008.0s
channel 105.0s
OPEN SOURCE · BUILT IN GO
Every event.
Accounted for.
Streams keep moving. Correctness should keep up. Process in event time, checkpoint with confidence, and recover without counting anything twice.
Go 1.25·Distributed stream processing A STREAM WITH A MEMORYFIG. 001
SOURCE 00event · 104209:41:05.000
SOURCE 01event · 104309:41:04.800
watermark 04.800
CHECKPOINT 008Every event, committed.State saved. Nothing counted twice.
ORDER IN THE FLOW.CONSISTENCY BY DESIGN ↗
Correctness is the deliverable.
Event-time windows
Consistent snapshots
Exactly-once output
01 / THE PRINCIPLES
Built for the moments
when streams get messy.
Events arrive out of order. Workers fail mid-job. Tidemark makes those cases part of the design.
The right time.
A consistent point
to come back to.
Chandy–Lamport barriers travel alongside records. Aligned barriers define a consistent snapshot across the dataflow.
Recover the work.
Keep the result.
Restore a completed checkpoint and replay deterministic sources. Sinks commit only when the checkpoint is confirmed.
Clean run
[12, 18, 24]Recovered run[12, 18, 24]02 / THE LIVE PLAYGROUND
Break a worker.
Not your results.
Follow records through a two-input dataflow. Slow an input or trigger a crash, then inspect what gets committed.
Browser simulation · 80 synthetic recordsA balanced stream. Watch the slowest input set the watermark and completed checkpoints commit the output.
RECORDS PROCESSED0 / 80
INPUT WATERMARK — event time
LAST COMPLETE CHECKPOINT— waiting
COMMITTED ROWS0 durable output
DATAFLOW TOPOLOGY record barrier watermark
Source 0
wm —0 / 40
Source 1
wm —0 / 40
Input gate
Wait for every input.
min(wm₀, wm₁)Awaiting input
Window
Group by event time.
tumbling(5s)0 open windows
Sink
Commit on confirmation.
exactly-once0 pending rows
Waiting for first barrierA checkpoint every 16 records.
WINDOW LIFECYCLE
open staged committed
Run the stream to see event-time windows fill.
LATEST RECORDS
Your first event is one step away.
TRANSACTIONAL SINK
THE STORY OF THIS RUNNewest first
At completion, compare final sink contents with an independent batch result.
0 recoveries0 replayed
This interactive model demonstrates the documented behavior using synthetic data in your browser. It does not execute the Go engine or measure its performance. Crash recovery compares final sorted sink contents; record delivery order can change.
THREE THINGS TO TRY
03 / UNDER THE SURFACE
A few strong rules.
A consistent system.
Recovery is a property of the whole dataflow. Every part has a small, explicit contract.
THE CHECKPOINT CONTRACT01—04
01
Inject
Sources emit barriers at fixed element intervals.
02
Align
Operators wait for the barrier on every input.
03
Snapshot
Capture operator state at a consistent cut.
04
Confirm & commit
Complete the checkpoint. Make sink output durable.
RECOVERY YOU CAN REASON ABOUTGo back to a known point.
Go back to a known point.
Then move forward.
Sources derive each record from its seed and offset. Seeking to a saved position produces the same sequence, so a recovered run can be checked against a clean one.
- Watermarks and barriers stay in-band with records.
- Faults occur at logical positions, not wall-clock times.
- Recovery uses only checkpoints marked complete.
- Windowed results are validated against a batch oracle.
04 / EVIDENCE OVER ESTIMATES
Numbers with
their context intact.
Throughput and recovery measurements are pending on reference hardware. The documented state-size results are available now, with parallelism kept in view.
Benchmark methodologyDOCUMENTED STATE SIZEq7 / 1,000 auctions
| Parallelism | Peak entries | Checkpoint state |
|---|---|---|
| p = 1 | 5,965 | 79.47 KiB |
| p = 2 | 397,846 | 15.56 MiB |
| p = 4 | 600,210 | 23.30 MiB |
2 million events · 5-second tumbling windows · 1 ms event time per element. Contiguous source ranges can hold back the watermark and accumulate state as parallelism increases.
Source: supplied Tidemark README · Phase 6b05 / GO A LITTLE DEEPER
From the playground
to the source.
Explore the engine locally, inspect its invariants, and run the checks that protect them.
GET STARTED / Go 1.25
# Get the engine $ git clone https://github.com/AarinB1/tidemark.git $ cd tidemark # Verify correctness, then run the identity demo $ make check $ make demo
Runs locally with Go and the repository dependencies.
What does “exactly-once” mean here?+
The final sorted sink contents match a clean run, even after a worker failure. Delivery can be at-least-once, and record emission order can change during recovery.
What is a watermark?+
A watermark tracks event-time progress. At a multi-input gate, the output watermark is the minimum across input channels. A fast source cannot close a window before a slower source has caught up.
Is this playground running the Go engine?+
No. It is a deterministic browser model built from the supplied project documentation. Run the source locally to exercise the actual Go implementation, race tests, and chaos suites.
Can I reproduce a recovery run?+
The engine uses deterministic seekable sources and fault schedules tied to logical positions. In this playground, reset a scenario to replay the same synthetic input. The worker-failure scenario injects one crash after record 24.
KEEP THE STREAM. KEEP THE TRUTH.Make every event count.
Explore the playground