3x20: GPUs and AI accelerators - What is the difference?

AI is everywhere, and so are AI accelerators, from CPU to GPU to special-purpose hardware. Eitan Medina, Chief Operating Officer of Habana Labs, an Intel Company, joins Frederic Van Haren and Stephen Foskett to discuss the various specialized AI processors being developed today. Habana Labs has created a special-purpose AI training and inferencing processor with many unique features. Since deep learning is done at scale today, it makes sense to integrate enterprise networking with an accelerator like Habana Gaudi to increase overall system performance thanks to rDMA over Ethernet (RoCE) technology. Habana Gaudi is optimized for matrix math and also includes a fully-programmable vector core for Tensor processing. In October 2021, Amazon AWS launched the new DL1 instance based on Habana Gaudi, offering more performance than many GPU-based instances for a much lower total cost. Habana is very developer-focused as well, working with partners, data scientists, and end users to expand the accessibility of the platform in channels like GitHub and their own developer forum. Habana will soon introduce a 7 nm Gaudi 2 processor with much-improved performance and power efficiency. Habana Labs is also making their hardware more accessible thanks to their SynapseAPI and recently acquired cnvrg.io to bring a higher-level MLOps pipeline to AI.

Three Questions:

• Frederic: At what point in time do you believe AI will be able to show compassion (like humans) if ever?
• Stephen: How big can ML models get? Will today's hundred-billion parameter model look small tomorrow or have we reached the limit?
• Edward Cui Founder of Graviti: Which will be more important in the future: Bigger and bigger ML or smaller ML?

Gests and Hosts

• Eitan Medina, Chief Operating Officer of Habana Labs, an Intel Company. Visit habana.ai to learn more.
• Frederic Van Haren, Founder at HighFens Inc., Consultancy & Services. Connect with Frederic on Highfens.com or on Twitter at @FredericVHaren.
• Stephen Foskett, Publisher of Gestalt IT and Organizer of Tech Field Day. Find Stephen’s writing at GestaltIT.com and on Twitter at @SFoskett.

Date: 2/01/2022 Tags: @SFoskett, @FredericVHaren, @HabanaLabs