CNN vs. GAN: How are they different?

Understanding convolutional neural networks (CNNs)

History. French computer scientist Yann LeCun, a professor at New York University and chief AI scientist at Meta, invented CNNs in the 1980s when he was a researcher at the University of Toronto. His aim was to improve the tools for recognizing handwritten digits by using neural networks. Although his work on optical character recognition was seminal, it stalled due to limited training data sets and computing power.

Interest in the technique exploded after 2010, following the introduction of ImageNet -- a large, labeled database of images -- and the launch of its annual ImageNet Large Scale Visual Recognition Challenge (ILSVRC). One of the most promising entries in the inaugural year of the competition was the AlexNet model based on CNNs, which was optimized for GPUs. Its success demonstrated that CNNs could efficiently scale to achieve good performance on even the largest image databases.

How they work. "CNNs are designed to use data with spatial structure such as images or video," said Donncha Carroll, a partner at Lotis Blue Consulting who leads the firm's Data Science Center of Excellence.

The convolutional neural network is composed of filters that move across the data and produce an output at every position. For example, a convolutional neural network designed to recognize animals in an image would activate when it recognizes legs, a body or a head.

It's also important to note that CNNs are designed to recognize the lines, edges and textures in patterns near each other, said Blankenbaker. "The 'C' in CNNs stands for convolutional, which means that we are processing something where the idea of neighborhood is important -- such as, for example, pixels around a given pixel or signal values slightly before and after a given moment."

Understanding generative adversarial networks (GANs)

History. GANs were invented by American computer scientist Ian Goodfellow, currently a research scientist at DeepMind, when he was working at Google Brain from 2014 to 2016.

GANs, as noted, are a type of deep learning model used to generate images of numbers and realistic-looking faces. The field exploded once researchers discovered it could be applied to synthesizing voices, drugs and other types of images. GANs and their variations were heralded by CNN inventor LeCun as the most interesting idea of the last 10 years in machine learning.

How they work. The term adversarial comes from the two competing networks creating and discerning content -- a generator network and a discriminator network. For example, in an image-generation use case, the generator network creates new images that look like faces. In contrast, the discriminator network tries to tell the difference between authentic and generated images. The discriminator performance data then helps to train the overall system.

CNN vs. GAN: Key differences and uses, explained

One important distinction between CNNs and GANs, Carroll said, is that the generator in GANs reverses the convolution process. "Convolution extracts features from images, while deconvolution expands images from features."

Here is a rundown of the chief differences between CNNs and GANs and their respective use cases.

CNN

• CNNs are used for recognizing objects, sounds or characteristics such as faces, biometrics, faulty parts or medical conditions. They are also ideal for interpreting images, speech or other audio signals.
• CNNs are trained using a supervised learning approach, with input data labeled for a particular output.
• The convolutional aspect of CNNs extracts features from images.
• Common use cases include reading documents, visually inspecting machine parts, listening to machinery to detect wear and hearing customer sentiment in customer service or sales calls.

GAN

• GANs are used to generate realistic-looking people, objects, sounds or characteristics.
• GANs are trained using an unsupervised learning approach -- i.e. they can be trained independently without requiring humans to label data.
• An inverse convolutional process, called deconvolution, expands images from features.
• Common use cases include generating realistic human-looking faces or an image of a specific individual, giving rise to the phenomenon known as deepfakes. They are also good at generating voices that sound like an individual or synthesizing someone's voice and tone in another language for more realistic dubbing. Other common use cases include generating all kinds of text, including news, poetry and code; speeding up drug discovery; and detecting fraud.

How can CNNs and GANs work together?

Although GANs are getting a lot of the attention lately, CNNs continue to be used under the hood -- that is, within GANs for generating and discerning authenticity. Indeed, Pierre Custeau, CTO of ToolsGroup, a supply chain planning and optimization firm, considers the two neural networks to be complementary in terms of function. "Since CNNs are so effective at image processing, both the generator and discriminator networks are by default CNNs," he said.

It is important to note that CNNs and GANs only tend to be combined in one way, said Matthew Mead, CTO at IT consultancy SPR.

"GANs typically work with image data and can use CNNs as the discriminator. But this doesn't work the other way around, meaning a CNN cannot use a GAN," Mead said.

One of the biggest challenges is always the data quality itself for training the models, especially when we're talking about business-specific solutions instead something as generic as a cat.

Early GANs generated relatively simple, low-resolution faces. One of the reasons interest in GANs has grown is the dramatic decline in cost per unit of compute, which has enabled teams to build more complex neural networks, Carroll pointed out. Advancements in hardware, software and neural network design have also fueled the growth of other generative AI models like transformers, variational autoencoders and diffusion.

Blankenbaker cautions against getting caught up in the latest model rather than focusing on specific goals and the underlying data. "We see too many companies getting excited about the buzzwords and trying to fit a square peg into a round hole, resulting in overspending on overkill solutions," Blakenbaker said.

"One of the biggest challenges is always the data quality itself for training the models, especially when we're talking about business-specific solutions instead something as generic as a cat," he said.