Podbean logo
  • Discover
  • Podcast Features

    Your all-in-one podcasting solution.

    Podcast Studio

    Easy-to-use audio recorder app.

  • Livestream

    High-performing audio live, without limits.

  • Podcast App

    The best podcast player & podcast app.

  • Ads Marketplace

    Join Ads Marketplace to earn money
    through sponsorship on your podcast.

    PodAds

    Manage your ads with dynamic ad insertion capability.

  • Patron & Paid Content

    The seamless way for fans to support you directly
    from your podcast.

  • Apple Podcasts Subscriptions Integration

    Effortlessly publish and manage exclusive episodes for your
    Apple Podcasts subscribers directly from Podbean.

  • All Arts Business Comedy Education
  • Fiction Government Health & Fitness History Kids & Family
  • Leisure Music News Religion & Spirituality Science
  • Society & Culture Sports Technology True Crime TV & Film
  • Live
  • How to Start a Podcast
  • How to Start a Live Podcast
  • How to Monetize a podcast
  • How to Promote Your Podcast
  • How to Use Group Recording
  • Log in
  • Start your podcast for free
  • Podcasting
    • Podcast Features
    • Live Stream
    • PodAds
    • Podcast App
    • Podcast Studio
  • Monetization
    • Premium
    • Patron
    • Apple Podcasts Subscriptions Integration
    • Ads Marketplace
  • Enterprise
  • Pricing
  • Discover
  • Log in
    Sign up free
Data Science at Home

Data Science at Home

Technology

How to improve the stability of training a GAN (Ep. 88)

How to improve the stability of training a GAN (Ep. 88)

2019-11-18
Download

Generative Adversarial Networks or GANs are very powerful tools to generate data. However, training a GAN is not easy. More specifically, GANs suffer of three major issues such as instability of the training procedure, mode collapse and vanishing gradients.

 

In this episode I not only explain the most challenging issues one would encounter while designing and training Generative Adversarial Networks. But also some methods and architectures to mitigate them. In addition I elucidate the three specific strategies that researchers are considering to improve the accuracy and the reliability of GANs.

 

The most tedious issues of GANs

 

Convergence to equilibrium

 

A typical GAN is formed by at least two networks: a generator G and a discriminator D. The generator's task is to generate samples from random noise. In turn, the discriminator has to learn to distinguish fake samples from real ones. While it is theoretically possible that generators and discriminators converge to a Nash Equilibrium (at which both networks are in their optimal state), reaching such equilibrium is not easy. 

 

Vanishing gradients

 

Moreover, a very accurate discriminator would push the loss function towards lower and lower values. This in turn, might cause the gradient to vanish and the entire network to stop learning completely. 

 

Mode collapse

 

Another phenomenon that is easy to observe when dealing with GANs is mode collapse. That is the incapability of the model to generate diverse samples. This in turn, leads to generated data that are more and more similar to the previous ones. Hence, the entire generated dataset would be just concentrated around a particular statistical value. 

 

The solution

 

Researchers have taken into consideration several approaches to overcome such issues. They have been playing with architectural changes, different loss functions and game theory.

 

Listen to the full episode to know more about the most effective strategies to build GANs that are reliable and robust.
Don't forget to join the conversation on our new Discord channel. See you there!

 

view more

More Episodes

Debunking AGI Hype and Embracing Reality [RB] (Ep. 245)
2023-12-04
Destroy your toaster before it kills you. Drama at OpenAI and other stories (Ep. 244)
2023-12-01
The AI Chip Chat 🤖💻 (Ep. 243)
2023-11-20
Rolling the Dice: Engineering in an Uncertain World (Ep. 242)
2023-11-09
How Language Models Are the Ultimate Database(Ep. 241)
2023-10-22
Elon is right this time: Rust is the language of AI (Ep. 240)
2023-10-09
Attacking LLMs for fun and profit (Ep. 239)
2023-09-18
Unlocking Language Models: The Power of Prompt Engineering (Ep. 238)
2023-09-11
Erosion of Software Architecture Quality in the Age of AI Code Generation (Ep. 237)
2023-08-30
The new dimension of AI: Vector Databases (Ep. 236)
2023-08-17
Building Self Serve Business Intelligence With AI and LLMs at Zenlytic (Ep. 235)
2023-08-04
Money, Cryptocurrencies, and AI: Exploring the Future of Finance with Chris Skinner (Ep. 234)
2023-07-09
Debunking AGI Hype and Embracing Reality (Ep. 233)
2023-06-26
Full steam ahead! Unraveling Forward-Forward Neural Networks (Ep. 232)
2023-06-15
The LLM Battle Begins: Google Bard vs ChatGPT (Ep. 231)
2023-06-06
Unleashing the Force: Blending Neural Networks and Physics for Epic Predictions (Ep. 230)
2023-05-30
AI’s Impact on Software Engineering: Killing Old Principles? [RB] (Ep. 229)
2023-05-25
Warning! Mathematical Mayhem Ahead: Demystifying Liquid Time-Constant Networks (Ep. 228)
2023-05-16
Efficiently Retraining Language Models: How to Level Up Without Breaking the Bank (Ep. 227)
2023-05-11
Revolutionize Your AI Game: How Running Large Language Models Locally Gives You an Unfair Advantage Over Big Tech Giants (Ep. 226)
2023-05-03
  • ←
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • →
012345678910111213141516171819

Get this podcast on your
phone, FREE

Download Podbean app on App Store Download Podbean app on Google Play

Create your
podcast in
minutes

  • Full-featured podcast site
  • Unlimited storage and bandwidth
  • Comprehensive podcast stats
  • Distribute to Apple Podcasts, Spotify, and more
  • Make money with your podcast
Get started

It is Free

  • Podcast Services

    • Podcast Features
    • Pricing
    • Enterprise Solution
    • Private Podcast
    • The Podcast App
    • Live Stream
    • Audio Recorder
    • Remote Recording
  •  
    • Create a Podcast
    • Video Podcast
    • Start Podcasting
    • Start Radio Talk Show
    • Education Podcast
    • Church Podcast
    • Nonprofit Podcast
    • Get Sermons Online
    • Free Audiobooks
  • MONETIZATION & MORE

    • Podcast Advertising
    • Dynamic Ads Insertion
    • Patron Program
    • Apple Podcasts Subscriptions
    • Switch to Podbean
    • Submit Your Podcast
    • Podbean Plugins
    • Developers
  • KNOWLEDGE BASE

    • How to Start a Podcast
    • How to Start a Live Podcast
    • How to Monetize a podcast
    • How to Promote Your Podcast
    • How to Use Group Recording
  • Support

    • Support Center
    • What’s New
    • Free Webinars
    • Podcast Events
    • Podbean Academy
    • Podcasting Smarter
    • Badges
    • Resources
  • Podbean

    • About Us
    • Podbean Blog
    • Careers
    • Press and Media
    • Green Initiative
    • Affiliate Program
    • Contact Us
  • Privacy Policy
  • Cookie Policy
  • Terms of Use
  • Consent Preferences
  • Copyright © 2015-2023 Podbean.com