Field notes from running Kafka in production.
September 16, 2026
Stateful Kafka Streams applications fail in predictable ways: off-heap RocksDB memory, eleven-minute restores, changelog topics nobody sized. The settings and habits that keep them boring.
Read moreSeptember 15, 2026 · MessagePilots
Connect is easy to start and easy to run badly. Field notes on tasks.max, the rebalance protocol, internal topics, error handling and DLQs, converters, and what exactly-once source support really requires.
Read moreSeptember 14, 2026 · MessagePilots
Replicating a Kafka cluster is easy to start and hard to fail over to. What MirrorMaker 2 copies, what offset translation can and cannot promise, how active/active avoids loops, and the failover rehearsal that tells you whether any of it works.
Read moreSeptember 14, 2026 · MessagePilots
Kafka 4.x ships a broker-side rebalance protocol that removes the global sync barrier and moves timeouts into group configs. What actually changes for your consumers, and a migration order that does not need a maintenance window.
Read moreSeptember 14, 2026 · MessagePilots
Kafka 4.1 made share groups generally available: many consumers per partition, per-record acknowledgement, broker-tracked redelivery. What that buys job-queue workloads, what it costs in ordering and exactly-once, and how to evaluate it.
Read more