Apache Kafka® 3.0 is out! To spotlight major enhancements in this release, Tim Berglund (Apache Kafka Developer Advocate) provides a summary of what’s new in the Kafka 3.0 release from Krakow, Poland, including API changes and improvements to the early-access Kafka Raft (KRaft).
KRaft is a built-in Kafka consensus mechanism that’s replacing Apache ZooKeeper going forward. It is recommended to try out new KRaft features in a development environment, as KRaft is not advised for production yet. One of the major features in Kafka 3.0 is the efficiency for KRaft controllers and brokers to store, load, and replicate snapshots into a Kafka cluster for metadata topic partitioning. The Kafka controller is now responsible for generating a Kafka producer ID in both ZooKeeper and KRaft, easing the transition from ZooKeeper to KRaft on the Kafka 3.X version line. This update also moves us closer to the ZooKeeper-to-KRaft bridge release. Additionally, this release includes metadata improvements, exactly-once semantics, and KRaft reassignments.
To enable a stronger record delivery guarantee, Kafka producers turn on by default idempotency, together with acknowledgment delivery by all the replicas. This release also comprises enhancements to Kafka Connect task restarts, Kafka Streams timestamp based synchronization and more flexible configuration options for MirrorMaker2 (MM2). The first version of MirrorMaker has been deprecated, and MirrorMaker2 will be the focus for future developments. Besides that, this release drops support for older message formats, V0 and V1, as well as initiates the removal of Java 8 and Scala 2.12 across all components in Apache Kafka. The universal Java 8 and Scala 2.12 deprecation is anticipated to complete in the future Apache Kafka 4.0 release.
Apache Kafka 3.0 is a major release and step forward for the Apache Kafka project!
EPISODE LINKS
Becoming Data Driven with Apache Kafka and Stream Processing ft. Daniel Jagielski
Integrating Spring Boot with Apache Kafka ft. Viktor Gamov
Confluent Platform 6.1 | What’s New in This Release + Updates
Building a Microservices Architecture with Apache Kafka at Nationwide Building Society ft. Rob Jackson
Examining Apache Kafka Performance Metrics ft. Alok Nikhil
Distributed Systems Engineering with Apache Kafka ft. Guozhang Wang
Scaling Developer Productivity with Apache Kafka ft. Mohinish Shaikh
Change Data Capture and Kafka Connect on Microsoft Azure ft. Abhishek Gupta
Event Streaming Trends and Predictions for 2021 ft. Gwen Shapira, Ben Stopford, and Michael Noll
How to Become a Certified Apache Kafka Expert ft. Niamh O’Byrne and Barry Ballard
Mastering DevOps with Apache Kafka, Kubernetes, and Confluent Cloud ft. Rick Spurgeon and Allison Walther
Apache Kafka 2.7 - Overview of Latest Features, Updates, and KIPs
Choreographing the Saga Pattern in Microservices ft. Chris Richardson
Apache Kafka and Porsche: Fast Cars and Fast Data ft. Sridhar Mamella
Tales from the Frontline of Apache Kafka DevOps ft. Jason Bell
Multi-Tenancy in Apache Kafka ft. Anna Pozvner
Distributed Systems Engineering with Apache Kafka ft. Roger Hoover
Why Kafka Streams Does Not Use Watermarks ft. Matthias J. Sax
Distributed Systems Engineering with Apache Kafka ft. Apurva Mehta
Most Terrifying Apache Kafka JIRAs of 2020 ft. Anna McDonald
Create your
podcast in
minutes
It is Free
Insight Story: Tech Trends Unpacked
Zero-Shot
Fast Forward by Tomorrow Unlocked: Tech past, tech future
The Unbelivable Truth - Series 1 - 26 including specials and pilot
Lex Fridman Podcast