tidemarkIn progress
Exactly-Once Stream Processing in Go
- Go
- Pebble
- Stream Processing
- Nexmark
- A distributed stream processing engine in Go. Watermarks propagate through the dataflow graph as the minimum across a gate's input channels, so a window fires only once every upstream partition has passed it.
- Chandy-Lamport barrier checkpointing with barrier alignment over Pebble-backed operator state. Sinks commit only once a checkpoint is confirmed, so a job that dies mid-run and recovers ends with the same sink contents as a clean run.
- Faults are injected at deterministic logical positions rather than on a wall clock, so a recovery run is reproducible from a seed, and every windowed result is checked against an independent batch oracle.