Apache Kafka® 3.0 is out! To spotlight major enhancements in this release, Tim Berglund (Apache Kafka Developer Advocate) provides a summary of what’s new in the Kafka 3.0 release from Krakow, Poland, including API changes and improvements to the early-access Kafka Raft (KRaft).
KRaft is a built-in Kafka consensus mechanism that’s replacing Apache ZooKeeper going forward. It is recommended to try out new KRaft features in a development environment, as KRaft is not advised for production yet. One of the major features in Kafka 3.0 is the efficiency for KRaft controllers and brokers to store, load, and replicate snapshots into a Kafka cluster for metadata topic partitioning. The Kafka controller is now responsible for generating a Kafka producer ID in both ZooKeeper and KRaft, easing the transition from ZooKeeper to KRaft on the Kafka 3.X version line. This update also moves us closer to the ZooKeeper-to-KRaft bridge release. Additionally, this release includes metadata improvements, exactly-once semantics, and KRaft reassignments.
To enable a stronger record delivery guarantee, Kafka producers turn on by default idempotency, together with acknowledgment delivery by all the replicas. This release also comprises enhancements to Kafka Connect task restarts, Kafka Streams timestamp based synchronization and more flexible configuration options for MirrorMaker2 (MM2). The first version of MirrorMaker has been deprecated, and MirrorMaker2 will be the focus for future developments. Besides that, this release drops support for older message formats, V0 and V1, as well as initiates the removal of Java 8 and Scala 2.12 across all components in Apache Kafka. The universal Java 8 and Scala 2.12 deprecation is anticipated to complete in the future Apache Kafka 4.0 release.
Apache Kafka 3.0 is a major release and step forward for the Apache Kafka project!
EPISODE LINKS
Confluent Platform 5.3 | What's New in This Release
How to Convert Python Batch Jobs into Kafka Streams Applications with Rishi Dhanaraj
Ask Confluent #14: In Control of Kafka with Dan Norwood
Kafka in Action with Dylan Scott
Change Data Capture with Debezium ft. Gunnar Morling
Distributed Systems Engineering with Apache Kafka ft. Jason Gustafson
Apache Kafka 2.3 | What's New in This Release + Updates and KIPs
Rolling Kafka Upgrades and Confluent Cloud ft. Gwen Shapira
Deploying Confluent Platform, from Zero to Hero ft. Mitch Henderson
Why Kafka Connect? ft. Robin Moffatt
Schema Registry Made Simple by Confluent Cloud ft. Magesh Nandakumar
Why is Stream Processing Hard? ft. Michael Drogalis
Testing Kafka Streams Applications with Viktor Gamov
Chris Riccomini on the History of Apache Kafka and Stream Processing
Ask Confluent #13: Machine Learning with Kai Waehner
Diving into Exactly Once Semantics with Guozhang Wang
Ask Confluent #12: In Search of the Lost Offsets
Ben Stopford on Microservices and Event Streaming
Magnus Edenhill on librdkafka 1.0
Ask Confluent #11: More Services, More Metrics, More Fun
Create your
podcast in
minutes
It is Free
Insight Story: Tech Trends Unpacked
Zero-Shot
Fast Forward by Tomorrow Unlocked: Tech past, tech future
The Unbelivable Truth - Series 1 - 26 including specials and pilot
Lex Fridman Podcast