Apache Kafka® 3.0 is out! To spotlight major enhancements in this release, Tim Berglund (Apache Kafka Developer Advocate) provides a summary of what’s new in the Kafka 3.0 release from Krakow, Poland, including API changes and improvements to the early-access Kafka Raft (KRaft).
KRaft is a built-in Kafka consensus mechanism that’s replacing Apache ZooKeeper going forward. It is recommended to try out new KRaft features in a development environment, as KRaft is not advised for production yet. One of the major features in Kafka 3.0 is the efficiency for KRaft controllers and brokers to store, load, and replicate snapshots into a Kafka cluster for metadata topic partitioning. The Kafka controller is now responsible for generating a Kafka producer ID in both ZooKeeper and KRaft, easing the transition from ZooKeeper to KRaft on the Kafka 3.X version line. This update also moves us closer to the ZooKeeper-to-KRaft bridge release. Additionally, this release includes metadata improvements, exactly-once semantics, and KRaft reassignments.
To enable a stronger record delivery guarantee, Kafka producers turn on by default idempotency, together with acknowledgment delivery by all the replicas. This release also comprises enhancements to Kafka Connect task restarts, Kafka Streams timestamp based synchronization and more flexible configuration options for MirrorMaker2 (MM2). The first version of MirrorMaker has been deprecated, and MirrorMaker2 will be the focus for future developments. Besides that, this release drops support for older message formats, V0 and V1, as well as initiates the removal of Java 8 and Scala 2.12 across all components in Apache Kafka. The universal Java 8 and Scala 2.12 deprecation is anticipated to complete in the future Apache Kafka 4.0 release.
Apache Kafka 3.0 is a major release and step forward for the Apache Kafka project!
EPISODE LINKS
The Pro’s Guide to Fully Managed Apache Kafka Services ft. Ricardo Ferreira
Kafka Screams: The Scariest JIRAs and How To Survive Them ft. Anna McDonald
Data Integration with Apache Kafka and Attunity
Distributed Systems Engineering with Apache Kafka ft. Colin McCabe
Apache Kafka on Kubernetes, Microsoft Azure, and ZooKeeper with Lena Hall
Improving Fairness Through Connection Throttling in the Cloud with KIP-402 ft. Gwen Shapira
Data Modeling for Apache Kafka – Streams, Topics & More with Dani Traphagen
MySQL, Cassandra, BigQuery, and Streaming Analytics with Joy Gao
Scaling Apache Kafka with Todd Palino
Understand What’s Flying Above You with Kafka Streams ft. Neil Buesing
KIP-500: Apache Kafka Without ZooKeeper ft. Colin McCabe and Jason Gustafson
Should You Run Apache Kafka on Kubernetes? ft. Balthazar Rouberol
Jay Kreps on the Last 10 Years of Apache Kafka and Event Streaming
Connecting to Apache Kafka with Neo4j
Ask Confluent #15: Attack of the Zombie Controller
Helping Healthcare with Apache Kafka and KSQL ft. Ramesh Sringeri
Contributing to Open Source with the Kafka Connect MongoDB Sink ft. Hans-Peter Grahsl
Teaching Apache Kafka Online with Stéphane Maarek
Connecting Apache Cassandra to Apache Kafka with Jeff Carpenter from DataStax
Transparent GDPR Encryption with David Jacot
Create your
podcast in
minutes
It is Free
Insight Story: Tech Trends Unpacked
Zero-Shot
Fast Forward by Tomorrow Unlocked: Tech past, tech future
The Unbelivable Truth - Series 1 - 26 including specials and pilot
Lex Fridman Podcast