Generative adversarial network

The original GAN is defined as the following game:

Each probability space $(\backslash Omega,\; \backslash mu\_\{\backslash text\{ref\}\})$ defines a GAN game.

There are 2 players: generator and discriminator.

The generator's strategy set is $\backslash mathcal\; P(\backslash Omega)$, the set of all probability measures $\backslash mu\_G$ on $\backslash Omega$.

The discriminator's strategy set is the set of Markov kernels $\backslash mu\_D:\; \backslash Omega\; \backslash to\; \backslash mathcal\; P[0,\; 1]$, where $\backslash mathcal\; P[0,\; 1]$ is the set of probability measures on $[0,\; 1]$.

The GAN game is a zero-sum game, with objective function$$L(\backslash mu\_G,\; \backslash mu\_D)\; :=\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\},\; y\backslash sim\; \backslash mu\_D(x)\}[\backslash ln\; y]\; +\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_G,\; y\backslash sim\; \backslash mu\_D(x)\}[\backslash ln\; (1-y)].$$

The generator aims to minimize the objective, and the discriminator aims to maximize the objective.

The generative network generates candidates while the discriminative network evaluates them. The contest operates in terms of data distributions. Typically, the generative network learns to map from a latent space to a data distribution of interest, while the discriminative network distinguishes candidates produced by the generator from the true data distribution. The generative network's training objective is to increase the error rate of the discriminative network (i.e., "fool" the discriminator network by producing novel candidates that the discriminator thinks are not synthesized (are part of the true data distribution)).{{cite journal |last1=Luc |first1=Pauline |last2=Couprie |first2=Camille |last3=Chintala |first3=Soumith |last4=Verbeek |first4=Jakob |date=2016-11-25 |title=Semantic Segmentation using Adversarial Networks |journal=NIPS Workshop on Adversarial Training, Dec, Barcelona, Spain |volume=2016 |arxiv=1611.08408 }}

A known dataset serves as the initial training data for the discriminator. Training involves presenting it with samples from the training dataset until it achieves acceptable accuracy. The generator is trained based on whether it succeeds in fooling the discriminator. Typically, the generator is seeded with randomized input that is sampled from a predefined latent space (e.g. a multivariate normal distribution). Thereafter, candidates synthesized by the generator are evaluated by the discriminator. Independent backpropagation procedures are applied to both networks so that the generator produces better samples, while the discriminator becomes more skilled at flagging synthetic samples.{{citation |author1=Andrej Karpathy |title=Generative Models |url=https://openai.com/blog/generative-models/ |publisher=OpenAI |access-date=April 7, 2016 |author2=Pieter Abbeel |author3=Greg Brockman |author4=Peter Chen |author5=Vicki Cheung |author6=Rocky Duan |author7=Ian Goodfellow |author8=Durk Kingma |author9=Jonathan Ho |author-link1=Andrej Karpathy |author-link2=Pieter Abbeel |author10=Rein Houthooft |author11=Tim Salimans |author12=John Schulman |author13=Ilya Sutskever |author14=Wojciech Zaremba}} When used for image generation, the generator is typically a deconvolutional neural network, and the discriminator is a convolutional neural network.

GANs are implicit generative models,{{cite arXiv |eprint=1610.03483 |class=stat.ML |first1=Shakir |last1=Mohamed |first2=Balaji |last2=Lakshminarayanan |title=Learning in Implicit Generative Models |year=2016}} which means that they do not explicitly model the likelihood function nor provide a means for finding the latent variable corresponding to a given sample, unlike alternatives such as flow-based generative model.

File:Types of deep generative models.png

Compared to fully visible belief networks such as WaveNet and PixelRNN and autoregressive models in general, GANs can generate one complete sample in one pass, rather than multiple passes through the network.

Compared to Boltzmann machines and linear ICA, there is no restriction on the type of function used by the network.

Since neural networks are universal approximators, GANs are asymptotically consistent. Variational autoencoders might be universal approximators, but it is not proven as of 2017.{{cite arXiv |last=Goodfellow |first=Ian |date=2017-04-03 |title=NIPS 2016 Tutorial: Generative Adversarial Networks |class=cs.LG |eprint=1701.00160 }}

This section provides some of the mathematical theory behind these methods.

In modern probability theory based on measure theory, a probability space also needs to be equipped with a σ-algebra. As a result, a more rigorous definition of the GAN game would make the following changes:

Each probability space $(\backslash Omega,\; \backslash mathcal\; B,\; \backslash mu\_\{\backslash text\{ref\}\})$ defines a GAN game.

The generator's strategy set is $\backslash mathcal\; P(\backslash Omega,\; \backslash mathcal\; B)$, the set of all probability measures $\backslash mu\_G$ on the measure-space $(\backslash Omega,\; \backslash mathcal\; B)$.

The discriminator's strategy set is the set of Markov kernels $\backslash mu\_D:\; (\backslash Omega,\; \backslash mathcal\; B)\; \backslash to\; \backslash mathcal\; P([0,\; 1],\; \backslash mathcal\; B([0,\; 1]))$, where $\backslash mathcal\; B([0,\; 1])$ is the Borel σ-algebra on $[0,\; 1]$.

In the most generic version of the GAN game described above, the strategy set for the discriminator contains all Markov kernels $\backslash mu\_D:\; \backslash Omega\; \backslash to\; \{\backslash mathcal\; \{P\}\}[0,1]$, and the strategy set for the generator contains arbitrary probability distributions $\backslash mu\_G$ on $\backslash Omega$.

However, as shown below, the optimal discriminator strategy against any $\backslash mu\_G$ is deterministic, so there is no loss of generality in restricting the discriminator's strategies to deterministic functions $D:\backslash Omega\; \backslash to\; [0,\; 1]$. In most applications, $D$ is a deep neural network function.

As for the generator, while $\backslash mu\_G$ could theoretically be any computable probability distribution, in practice, it is usually implemented as a pushforward: $\backslash mu\_G\; =\; \backslash mu\_Z\; \backslash circ\; G^\{-1\}$. That is, start with a random variable $z\; \backslash sim\; \backslash mu\_Z$, where $\backslash mu\_Z$ is a probability distribution that is easy to compute (such as the uniform distribution, or the Gaussian distribution), then define a function $G:\; \backslash Omega\_Z\; \backslash to\; \backslash Omega$. Then the distribution $\backslash mu\_G$ is the distribution of $G(z)$.

Consequently, the generator's strategy is usually defined as just $G$, leaving $z\; \backslash sim\; \backslash mu\_Z$ implicit. In this formalism, the GAN game objective is$$L(G,\; D)\; :=\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\}\}[\backslash ln\; D(x)]\; +\; \backslash operatorname\; E\_\{z\backslash sim\; \backslash mu\_Z\}[\backslash ln\; (1-D(G(z)))].$$

The GAN architecture has two main components. One is casting optimization into a game, of form $\backslash min\_G\; \backslash max\_D\; L(G,\; D)$, which is different from the usual kind of optimization, of form $\backslash min\_\backslash theta\; L(\backslash theta)$. The other is the decomposition of $\backslash mu\_G$ into $\backslash mu\_Z\; \backslash circ\; G^\{-1\}$, which can be understood as a reparametrization trick.

To see its significance, one must compare GAN with previous methods for learning generative models, which were plagued with "intractable probabilistic computations that arise in maximum likelihood estimation and related strategies".

At the same time, Kingma and Welling{{cite arXiv |last1=Kingma |first1=Diederik P. |last2=Welling |first2=Max |date=2014-05-01 |title=Auto-Encoding Variational Bayes |class=stat.ML |eprint=1312.6114 }} and Rezende et al.{{Cite journal |last1=Rezende |first1=Danilo Jimenez |last2=Mohamed |first2=Shakir |last3=Wierstra |first3=Daan |date=2014 |title=Stochastic Backpropagation and Approximate Inference in Deep Generative Models |url=https://proceedings.mlr.press/v32/rezende14.html |journal=Journal of Machine Learning Research |volume=32 |issue=2 |pages=1278–1286 |arxiv=1401.4082}} developed the same idea of reparametrization into a general stochastic backpropagation method. Among its first applications was the variational autoencoder.

In the original paper, as well as most subsequent papers, it is usually assumed that the generator moves first, and the discriminator moves second, thus giving the following minimax game:$$\backslash min\_\{\backslash mu\_G\}\backslash max\_\{\backslash mu\_D\}\; L(\backslash mu\_G,\; \backslash mu\_D)\; :=\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\},\; y\backslash sim\; \backslash mu\_D(x)\}[\backslash ln\; y]\; +\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_G,\; y\backslash sim\; \backslash mu\_D(x)\}[\backslash ln\; (1-y)].$$

If both the generator's and the discriminator's strategy sets are spanned by a finite number of strategies, then by the minimax theorem,$$\backslash min\_\{\backslash mu\_G\}\backslash max\_\{\backslash mu\_D\}\; L(\backslash mu\_G,\; \backslash mu\_D)=\; \backslash max\_\{\backslash mu\_D\}\backslash min\_\{\backslash mu\_G\}\; L(\backslash mu\_G,\; \backslash mu\_D)$$that is, the move order does not matter.

However, since the strategy sets are both not finitely spanned, the minimax theorem does not apply, and the idea of an "equilibrium" becomes delicate. To wit, there are the following different concepts of equilibrium:

• Equilibrium when generator moves first, and discriminator moves second:$$\backslash hat\; \backslash mu\_G\; \backslash in\; \backslash arg\backslash min\_\{\backslash mu\_G\}\backslash max\_\{\backslash mu\_D\}\; L(\backslash mu\_G,\backslash mu\_D),\backslash quad$$

\hat \mu_D \in \arg\max_{\mu_D} L(\hat\mu_G, \mu_D), \quad

• Equilibrium when discriminator moves first, and generator moves second:$$\backslash hat\; \backslash mu\_D\; \backslash in\; \backslash arg\backslash max\_\{\backslash mu\_D\}\backslash min\_\{\backslash mu\_G\}\; L(\backslash mu\_G,\; \backslash mu\_D),\; \backslash quad$$

\hat \mu_G \in \arg\min_{\mu_G} L(\mu_G,\hat \mu_D),

• Nash equilibrium $(\backslash hat\; \backslash mu\_D,\; \backslash hat\backslash mu\_G)$

, which is stable under simultaneous move order:$$\backslash hat\; \backslash mu\_D\; \backslash in\; \backslash arg\backslash max\_\{\backslash mu\_D\}\; L(\backslash hat\backslash mu\_G,\; \backslash mu\_D),\; \backslash quad\; \backslash hat\; \backslash mu\_G\; \backslash in\; \backslash arg\backslash min\_\{\backslash mu\_G\}\; L(\backslash mu\_G,\; \backslash hat\backslash mu\_D)$$

For general games, these equilibria do not have to agree, or even to exist. For the original GAN game, these equilibria all exist, and are all equal. However, for more general GAN games, these do not necessarily exist, or agree.{{Cite book |last1=Farnia |first1=Farzan |last2=Ozdaglar |first2=Asuman |date=2020-11-21 |chapter=Do GANs always have Nash equilibria? |chapter-url=https://proceedings.mlr.press/v119/farnia20a.html |title=Proceedings of the 37th International Conference on Machine Learning |language=en |publisher=PMLR |volume=119 |pages=3029–3039}}

The original GAN paper proved the following two theorems:{{Math theorem

| math_statement = For any fixed generator strategy $\backslash mu\_G$, let the optimal reply be $D^*\; =\; \backslash arg\backslash max\_\{D\}\; L(\backslash mu\_G,\; D)$, then

$$\backslash begin\{align\}$$

D^*(x) &= \frac{d\mu_{\text{ref}}}{d(\mu_{\text{ref}} + \mu_G)}\\[6pt]

L(\mu_G, D^*) &= 2D_{JS}(\mu_{\text{ref}}; \mu_G) - 2\ln 2

\end{align}

where the derivative is the Radon–Nikodym derivative, and $D\_\{JS\}$ is the Jensen–Shannon divergence.

| name = Theorem

| note = the optimal discriminator computes the Jensen–Shannon divergence

}}{{Math proof|proof=

By Jensen's inequality,

$$\backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\},\; y\backslash sim\; \backslash mu\_D(x)\}[\backslash ln\; y]\; \backslash leq\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\}\}[\backslash ln\; \backslash operatorname\; E\_\{y\backslash sim\; \backslash mu\_D(x)\}[y]]$$

and similarly for the other term. Therefore, the optimal reply can be deterministic, i.e. $\backslash mu\_D(x)\; =\; \backslash delta\_\{D(x)\}$ for some function $D:\; \backslash Omega\; \backslash to\; [0,\; 1]$, in which case

$$L(\backslash mu\_G,\; \backslash mu\_D)\; :=\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\}\}[\backslash ln\; D(x)]\; +\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{G\}\}[\backslash ln\; (1-D(x))].$$

To define suitable density functions, we define a base measure $\backslash mu\; :=\; \backslash mu\_\{\backslash text\{ref\}\}\; +\; \backslash mu\_G$, which allows us to take the Radon–Nikodym derivatives

$$\backslash rho\_\{\backslash text\{ref\}\}\; =\; \backslash frac\{d\backslash mu\_\{\backslash text\{ref\}\}\}\{d\backslash mu\}\; \backslash quad\; \backslash rho\_\{G\}\; =\; \backslash frac\{d\backslash mu\_\{G\}\}\{d\backslash mu\}$$

with $\backslash rho\_\{\backslash text\{ref\}\}\; +\; \backslash rho\_G\; =\; 1$.

We then have

$$L(\backslash mu\_G,\; \backslash mu\_D)\; :=\; \backslash int\; \backslash mu(dx)\; \backslash left[\backslash rho\_\{\backslash text\{ref\}\}(x)\; \backslash ln(D(x))\; +\; \backslash rho\_G(x)\; \backslash ln(1-D(x))\backslash right].$$

The integrand is just the negative cross-entropy between two Bernoulli random variables with parameters $\backslash rho\_\{\backslash text\{ref\}\}(x)$ and $D(x)$. We can write this as $-H(\backslash rho\_\{\backslash text\{ref\}\}(x))-D\_\{KL\}(\backslash rho\_\{\backslash text\{ref\}\}(x)\; \backslash parallel\; D(x))$, where $H$ is the binary entropy function, so

$$L(\backslash mu\_G,\; \backslash mu\_D)\; =\; -\backslash int\; \backslash mu(dx)\; (H(\backslash rho\_\{\backslash text\{ref\}\}(x))\; +\; D\_\{KL\}(\backslash rho\_\{\backslash text\{ref\}\}(x)\; \backslash parallel\; D(x))).$$

This means that the optimal strategy for the discriminator is $D(x)\; =\; \backslash rho\_\{\backslash text\{ref\}\}(x)$, with

$$L(\backslash mu\_G,\; \backslash mu\_D^*)\; =\; -\backslash int\; \backslash mu(dx)\; H(\backslash rho\_\{\backslash text\{ref\}\}(x))\; =\; D\_\{JS\}(\backslash mu\_\{\backslash text\{ref\}\}\; \backslash parallel\; \backslash mu\_G)\; -2\; \backslash ln\; 2$$

after routine calculation.

}}

Interpretation: For any fixed generator strategy $\backslash mu\_G$, the optimal discriminator keeps track of the likelihood ratio between the reference distribution and the generator distribution:$$\backslash frac\{D(x)\}\{1-D(x)\}\; =\; \backslash frac\{d\backslash mu\_\{\backslash text\{ref\}\}\}\{d\backslash mu\_G\}(x)\; =\; \backslash frac\{\backslash mu\_\{\backslash text\{ref\}\}(dx)\}\{\backslash mu\_G(dx)\};\; \backslash quad\; D(x)\; =\; \backslash sigma(\backslash ln\backslash mu\_\{\backslash text\{ref\}\}(dx)\; -\; \backslash ln\backslash mu\_G(dx))$$where $\backslash sigma$ is the logistic function.

In particular, if the prior probability for an image $x$ to come from the reference distribution is equal to $\backslash frac\; 12$, then $D(x)$ is just the posterior probability that $x$ came from the reference distribution:$$D(x)\; =\; \backslash Pr(x\; \backslash text\{\; came\; from\; reference\; distribution\}\; \backslash mid\; x).$$

{{Math theorem

| math_statement = For any GAN game, there exists a pair $(\backslash hat\backslash mu\_D,\; \backslash hat\backslash mu\_G)$ that is both a sequential equilibrium and a Nash equilibrium:

$$\backslash begin\{align\}$$

& L(\hat\mu_G, \hat\mu_D) = \min_{\mu_G}\max_{\mu_D} L(\mu_G, \mu_D) =& \max_{\mu_D}\min_{\mu_G} L(\mu_G, \mu_D) = -2\ln 2\\[6pt]

& \hat \mu_D \in \arg\max_{\mu_D}\min_{\mu_G} L(\mu_G, \mu_D), &\quad \hat \mu_G \in \arg\min_{\mu_G}\max_{\mu_D} L(\mu_G, \mu_D)\\[6pt]

& \hat \mu_D \in \arg\max_{\mu_D} L(\hat\mu_G, \mu_D), &\quad \hat \mu_G \in \arg\min_{\mu_G} L(\mu_G, \hat\mu_D)\\[6pt]

& \forall x\in \Omega, \hat \mu_D(x) = \delta_{\frac 1 2}, &\quad\hat \mu_G = \mu_{\text{ref}}

\end{align}

That is, the generator perfectly mimics the reference, and the discriminator outputs $\backslash frac\; 12$ deterministically on all inputs.

| name = Theorem

| note = the unique equilibrium point

}}

{{Math proof|proof=

From the previous proposition,

$$\backslash arg\backslash min\_\{\backslash mu\_G\}\backslash max\_\{\backslash mu\_D\}\; L(\backslash mu\_G,\; \backslash mu\_D)=\; \backslash mu\_\{\backslash text\{ref\}\};\; \backslash quad\; \backslash min\_\{\backslash mu\_G\}\backslash max\_\{\backslash mu\_D\}\; L(\backslash mu\_G,\; \backslash mu\_D)\; =\; -2\backslash ln\; 2.$$

For any fixed discriminator strategy $\backslash mu\_D$, any $\backslash mu\_G$ concentrated on the set

$$\backslash \{\; x\; \backslash mid$$

\operatorname E_{y\sim\mu_D(x)}[\ln(1-y)] = \inf_x \operatorname E_{y\sim \mu_D(x)}[\ln(1-y)] \}

is an optimal strategy for the generator. Thus,

$$\backslash arg\backslash max\_\{\backslash mu\_D\}\backslash min\_\{\backslash mu\_G\}\; L(\backslash mu\_G,\; \backslash mu\_D)\; =\; \backslash arg\backslash max\_\{\backslash mu\_D\}\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\},\; y\backslash sim\; \backslash mu\_D(x)\}[\backslash ln\; y]\; +\; \backslash inf\_x\; \backslash operatorname\; E\_\{y\backslash sim\; \backslash mu\_D(x)\}[\backslash ln\; (1-y)].$$

By Jensen's inequality, the discriminator can only improve by adopting the deterministic strategy of always playing $D(x)\; =\; \backslash operatorname\; E\_\{y\backslash sim\; \backslash mu\_D(x)\}[y]$. Therefore,

$$\backslash arg\backslash max\_\{\backslash mu\_D\}\backslash min\_\{\backslash mu\_G\}\; L(\backslash mu\_G,\; \backslash mu\_D)\; =\; \backslash arg\backslash max\_\{D\}\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\}\}[\backslash ln\; D(x)]\; +\; \backslash inf\_x\; \backslash ln\; (1-D(x))$$

By Jensen's inequality,

\begin{align}

& \ln \operatorname E_{x\sim \mu_{\text{ref}}}[ D(x)] + \inf_x \ln (1-D(x)) \\[6pt]

= {} & \ln \operatorname E_{x\sim \mu_{\text{ref}}}[ D(x)] + \ln (1-\sup_x D(x)) \\[6pt]

= {} & \ln [\operatorname E_{x\sim \mu_{\text{ref}}}[ D(x)] (1-\sup_x D(x))] \leq \ln [ \sup_x D(x)) (1-\sup_x D(x))] \leq \ln\frac 14,

\end{align}

with equality if $D(x)\; =\; \backslash frac\; 12$, so

$$\backslash forall\; x\backslash in\; \backslash Omega,\; \backslash hat\; \backslash mu\_D(x)\; =\; \backslash delta\_\{\backslash frac\; 1\; 2\};\; \backslash quad\; \backslash max\_\{\backslash mu\_D\}\backslash min\_\{\backslash mu\_G\}\; L(\backslash mu\_G,\; \backslash mu\_D)\; =\; -2\backslash ln\; 2.$$

Finally, to check that this is a Nash equilibrium, note that when $\backslash mu\_G\; =\; \backslash mu\_\{\backslash text\{ref\}\}$, we have

$$L(\backslash mu\_G,\; \backslash mu\_D)\; :=\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\},\; y\backslash sim\; \backslash mu\_D(x)\}[\backslash ln\; (y\; (1-y))]$$

which is always maximized by $y\; =\; \backslash frac\; 12$.

When $\backslash forall\; x\backslash in\; \backslash Omega,\; \backslash mu\_D(x)\; =\; \backslash delta\_\{\backslash frac\; 1\; 2\}$, any strategy is optimal for the generator.}}

While the GAN game has a unique global equilibrium point when both the generator and discriminator have access to their entire strategy sets, the equilibrium is no longer guaranteed when they have a restricted strategy set.

In practice, the generator has access only to measures of form $\backslash mu\_Z\; \backslash circ\; G\_\backslash theta^\{-1\}$, where $G\_\backslash theta$ is a function computed by a neural network with parameters $\backslash theta$, and $\backslash mu\_Z$ is an easily sampled distribution, such as the uniform or normal distribution. Similarly, the discriminator has access only to functions of form $D\_\backslash zeta$, a function computed by a neural network with parameters $\backslash zeta$. These restricted strategy sets take up a vanishingly small proportion of their entire strategy sets.{{cite arXiv |last=Weng |first=Lilian |date=2019-04-18 |title=From GAN to WGAN |class=cs.LG |eprint=1904.08994 }}

Further, even if an equilibrium still exists, it can only be found by searching in the high-dimensional space of all possible neural network functions. The standard strategy of using gradient descent to find the equilibrium often does not work for GAN, and often the game "collapses" into one of several failure modes. To improve the convergence stability, some training strategies start with an easier task, such as generating low-resolution images or simple images (one object with uniform background),{{Cite arXiv |last1=Soviany |first1=Petru |last2=Ardei |first2=Claudiu |last3=Ionescu |first3=Radu Tudor |last4=Leordeanu |first4=Marius |date=2019-10-22 |title=Image Difficulty Curriculum for Generative Adversarial Networks (CuGAN) |class=cs.LG |eprint=1910.08967}} and gradually increase the difficulty of the task during training. This essentially translates to applying a curriculum learning scheme.{{Cite journal |last1=Hacohen |first1=Guy |last2=Weinshall |first2=Daphna |date=2019-05-24 |title=On The Power of Curriculum Learning in Training Deep Networks |url=https://proceedings.mlr.press/v97/hacohen19a.html |journal=International Conference on Machine Learning |language=en |publisher=PMLR |pages=2535–2544|arxiv=1904.03626 }}

{{Main|Mode collapse}}

GANs often suffer from mode collapse where they fail to generalize properly, missing entire modes from the input data. For example, a GAN trained on the MNIST dataset containing many samples of each digit might only generate pictures of digit 0. This was termed "the Helvetica scenario".

One way this can happen is if the generator learns too fast compared to the discriminator. If the discriminator $D$ is held constant, then the optimal generator would only output elements of $\backslash arg\backslash max\_x\; D(x)$.{{Cite web |title=r/MachineLearning - Comment by u/ian_goodfellow on "[R] [1701.07875] Wasserstein GAN |url=https://www.reddit.com/r/MachineLearning/comments/5qxoaz/comment/dd4041v/?context=3 |access-date=2022-07-15 |website=reddit | date=January 30, 2017 |language=en-US}} So for example, if during GAN training for generating MNIST dataset, for a few epochs, the discriminator somehow prefers the digit 0 slightly more than other digits, the generator may seize the opportunity to generate only digit 0, then be unable to escape the local minimum after the discriminator improves.

Some researchers perceive the root problem to be a weak discriminative network that fails to notice the pattern of omission, while others assign blame to a bad choice of objective function. Many solutions have been proposed, but it is still an open problem.{{cite conference |last1=Lin |first1=Zinan |last2=Khetan |first2=Ashish |last3=Fanti |first3=Giulia |last4=Oh |first4=Sewoong |display-authors=1 |date=December 2018 |title=PacGAN: the power of two samples in generative adversarial networks |url=https://dl.acm.org/doi/10.5555/3326943.3327081 |conference=32nd International Conference on Neural Information Processing Systems |pages=1505–1514 |arxiv=1712.04086}}{{cite arXiv |eprint=1801.04406 |class=cs.LG |first1=Lars |last1=Mescheder |first2=Andreas |last2=Geiger |title=Which Training Methods for GANs do actually Converge? |date=2018-07-31 |last3=Nowozin |first3=Sebastian}}

Even the state-of-the-art architecture, BigGAN (2019), could not avoid mode collapse. The authors resorted to "allowing collapse to occur at the later stages of training, by which time a model is sufficiently trained to achieve good results".

The two time-scale update rule (TTUR) is proposed to make GAN convergence more stable by making the learning rate of the generator lower than that of the discriminator. The authors argued that the generator should move slower than the discriminator, so that it does not "drive the discriminator steadily into new regions without capturing its gathered information".

They proved that a general class of games that included the GAN game, when trained under TTUR, "converges under mild assumptions to a stationary local Nash equilibrium".{{Cite journal |last1=Heusel |first1=Martin |last2=Ramsauer |first2=Hubert |last3=Unterthiner |first3=Thomas |last4=Nessler |first4=Bernhard |last5=Hochreiter |first5=Sepp |date=2017 |title=GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium |url=https://proceedings.neurips.cc/paper/2017/hash/8a1d694707eb0fefe65871369074926d-Abstract.html |journal=Advances in Neural Information Processing Systems |publisher=Curran Associates, Inc. |volume=30|arxiv=1706.08500 }}

They also proposed using the Adam stochastic optimization{{cite arXiv |last1=Kingma |first1=Diederik P. |last2=Ba |first2=Jimmy |date=2017-01-29 |title=Adam: A Method for Stochastic Optimization |class=cs.LG |eprint=1412.6980 }} to avoid mode collapse, as well as the Fréchet inception distance for evaluating GAN performances.

Conversely, if the discriminator learns too fast compared to the generator, then the discriminator could almost perfectly distinguish $\backslash mu\_\{G\_\backslash theta\},\; \backslash mu\_\{\backslash text\{ref\}\}$. In such case, the generator $G\_\backslash theta$ could be stuck with a very high loss no matter which direction it changes its $\backslash theta$, meaning that the gradient $\backslash nabla\_\backslash theta\; L(G\_\backslash theta,\; D\_\backslash zeta)$ would be close to zero. In such case, the generator cannot learn, a case of the vanishing gradient problem.

Intuitively speaking, the discriminator is too good, and since the generator cannot take any small step (only small steps are considered in gradient descent) to improve its payoff, it does not even try.

One important method for solving this problem is the Wasserstein GAN.

GANs are usually evaluated by Inception score (IS), which measures how varied the generator's outputs are (as classified by an image classifier, usually Inception-v3), or Fréchet inception distance (FID), which measures how similar the generator's outputs are to a reference set (as classified by a learned image featurizer, such as Inception-v3 without its final layer). Many papers that propose new GAN architectures for image generation report how their architectures break the state of the art on FID or IS.

Another evaluation method is the Learned Perceptual Image Patch Similarity (LPIPS), which starts with a learned image featurizer $f\_\backslash theta:\; \backslash text\{Image\}\; \backslash to\; \backslash R^n$, and finetunes it by supervised learning on a set of $(x,\; x\text{'},\; \backslash operatorname\{perceptual~difference\}(x,\; x\text{'}))$, where $x$ is an image, $x\text{'}$ is a perturbed version of it, and $\backslash operatorname\{perceptual~difference\}(x,\; x\text{'})$ is how much they differ, as reported by human subjects. The model is finetuned so that it can approximate $\backslash |f\_\backslash theta(x)\; -\; f\_\backslash theta(x\text{'})\backslash |\; \backslash approx\; \backslash operatorname\{perceptual~difference\}(x,\; x\text{'})$. This finetuned model is then used to define $\backslash operatorname\{LPIPS\}(x,\; x\text{'})\; :=\; \backslash |f\_\backslash theta(x)\; -\; f\_\backslash theta(x\text{'})\backslash |$.{{cite arXiv |last1=Zhang |first1=Richard |last2=Isola |first2=Phillip |last3=Efros |first3=Alexei A. |last4=Shechtman |first4=Eli |last5=Wang |first5=Oliver |date=2018 |title=The Unreasonable Effectiveness of Deep Features as a Perceptual Metric |pages=586–595|class=cs.CV |eprint=1801.03924 }}

Other evaluation methods are reviewed in.{{Cite journal |last=Borji |first=Ali |date=2019-02-01 |title=Pros and cons of GAN evaluation measures |url=https://www.sciencedirect.com/science/article/pii/S1077314218304272 |journal=Computer Vision and Image Understanding |language=en |volume=179 |pages=41–65 |doi=10.1016/j.cviu.2018.10.009 |arxiv=1802.03446 |s2cid=3627712 |issn=1077-3142}}

There is a veritable zoo of GAN variants.{{Citation |last=Hindupur |first=Avinash |title=The GAN Zoo |date=2022-07-15 |url=https://github.com/hindupuravinash/the-gan-zoo |access-date=2022-07-15}} Some of the most prominent are as follows:

Conditional GANs are similar to standard GANs except they allow the model to conditionally generate samples based on additional information. For example, if we want to generate a cat face given a dog picture, we could use a conditional GAN.

The generator in a GAN game generates $\backslash mu\_G$, a probability distribution on the probability space $\backslash Omega$. This leads to the idea of a conditional GAN, where instead of generating one probability distribution on $\backslash Omega$, the generator generates a different probability distribution $\backslash mu\_G(c)$ on $\backslash Omega$, for each given class label $c$.

For example, for generating images that look like ImageNet, the generator should be able to generate a picture of cat when given the class label "cat".

In the original paper, the authors noted that GAN can be trivially extended to conditional GAN by providing the labels to both the generator and the discriminator.

Concretely, the conditional GAN game is just the GAN game with class labels provided:$$L(\backslash mu\_G,\; D)\; :=\; \backslash operatorname\; E\_\{c\; \backslash sim\; \backslash mu\_C,\; x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\}(c)\}[\backslash ln\; D(x,\; c)]\; +\; \backslash operatorname\; E\_\{c\; \backslash sim\; \backslash mu\_C,\; x\backslash sim\; \backslash mu\_G(c)\}[\backslash ln\; (1-D(x,\; c))]$$where $\backslash mu\_C$ is a probability distribution over classes, $\backslash mu\_\{\backslash text\{ref\}\}(c)$ is the probability distribution of real images of class $c$, and $\backslash mu\_G(c)$ the probability distribution of images generated by the generator when given class label $c$.

In 2017, a conditional GAN learned to generate 1000 image classes of ImageNet.{{Cite journal |last1=Odena |first1=Augustus |last2=Olah |first2=Christopher |last3=Shlens |first3=Jonathon |date=2017-07-17 |title=Conditional Image Synthesis with Auxiliary Classifier GANs |url=https://proceedings.mlr.press/v70/odena17a.html |journal=International Conference on Machine Learning |language=en |publisher=PMLR |pages=2642–2651|arxiv=1610.09585 }}

The GAN game is a general framework and can be run with any reasonable parametrization of the generator $G$ and discriminator $D$. In the original paper, the authors demonstrated it using multilayer perceptron networks and convolutional neural networks. Many alternative architectures have been tried.

Deep convolutional GAN (DCGAN):{{Cite journal |last1=Radford |first1=Alec |last2=Metz |first2=Luke |last3=Chintala |first3=Soumith |date=2016 |title=Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks |journal=ICLR|s2cid=11758569 }} For both generator and discriminator, uses only deep networks consisting entirely of convolution-deconvolution layers, that is, fully convolutional networks.{{Cite journal |last1=Long |first1=Jonathan |last2=Shelhamer |first2=Evan |last3=Darrell |first3=Trevor |date=2015 |title=Fully Convolutional Networks for Semantic Segmentation |url=https://openaccess.thecvf.com/content_cvpr_2015/html/Long_Fully_Convolutional_Networks_2015_CVPR_paper.html |pages=3431–3440|journal=CVF}}

Self-attention GAN (SAGAN):{{Cite journal |last1=Zhang |first1=Han |last2=Goodfellow |first2=Ian |last3=Metaxas |first3=Dimitris |last4=Odena |first4=Augustus |date=2019-05-24 |title=Self-Attention Generative Adversarial Networks |url=https://proceedings.mlr.press/v97/zhang19d.html |journal=International Conference on Machine Learning |language=en |publisher=PMLR |pages=7354–7363}} Starts with the DCGAN, then adds residually-connected standard self-attention modules to the generator and discriminator.

Variational autoencoder GAN (VAEGAN):{{Cite journal |last1=Larsen |first1=Anders Boesen Lindbo |last2=Sønderby |first2=Søren Kaae |last3=Larochelle |first3=Hugo |last4=Winther |first4=Ole |date=2016-06-11 |title=Autoencoding beyond pixels using a learned similarity metric |url=https://proceedings.mlr.press/v48/larsen16.html |journal=International Conference on Machine Learning |language=en |publisher=PMLR |pages=1558–1566|arxiv=1512.09300 }} Uses a variational autoencoder (VAE) for the generator.

Transformer GAN (TransGAN):{{cite arXiv |last1=Jiang |first1=Yifan |last2=Chang |first2=Shiyu |last3=Wang |first3=Zhangyang |date=2021-12-08 |title=TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up |class=cs.CV |eprint=2102.07074 }} Uses the pure transformer architecture for both the generator and discriminator, entirely devoid of convolution-deconvolution layers.

Flow-GAN:{{cite arXiv |last1=Grover |first1=Aditya |last2=Dhar |first2=Manik |last3=Ermon |first3=Stefano |date=2017-05-01 |title=Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models |class=cs.LG |eprint=1705.08868 }} Uses flow-based generative model for the generator, allowing efficient computation of the likelihood function.

Many GAN variants are merely obtained by changing the loss functions for the generator and discriminator.

Original GAN:

We recast the original GAN objective into a form more convenient for comparison:$$\backslash begin\{cases\}$$

\min_D L_D(D, \mu_G) = -\operatorname E_{x\sim \mu_{G}}[\ln D(x)] - \operatorname E_{x\sim \mu_{\text{ref}}}[\ln (1-D(x))]\\

\min_G L_G(D, \mu_G) = -\operatorname E_{x\sim \mu_G}[\ln (1-D(x))]

\end{cases}

Original GAN, non-saturating loss:

This objective for generator was recommended in the original paper for faster convergence.$$L\_G\; =\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_G\}[\backslash ln\; D(x)]$$The effect of using this objective is analyzed in Section 2.2.2 of Arjovsky et al.{{cite arXiv |last1=Arjovsky |first1=Martin |last2=Bottou |first2=Léon |date=2017-01-01 |title=Towards Principled Methods for Training Generative Adversarial Networks |class=stat.ML |eprint=1701.04862 }}

Original GAN, maximum likelihood:

$$L\_G\; =\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_G\}[(\{\backslash exp\}\; \backslash circ\; \backslash sigma^\{-1\}\; \backslash circ\; D)\; (x)]$$where $\backslash sigma$ is the logistic function. When the discriminator is optimal, the generator gradient is the same as in maximum likelihood estimation, even though GAN cannot perform maximum likelihood estimation itself.{{cite arXiv |last=Goodfellow |first=Ian J. |date=2014-12-01 |title=On distinguishability criteria for estimating generative models |class=stat.ML |eprint=1412.6515 }}{{Cite web |last=Goodfellow |first=Ian |date=2016-08-31 |title=Generative Adversarial Networks (GANs), Presentation at Berkeley Artificial Intelligence Lab |url=https://www.iangoodfellow.com/slides/2016-08-31-Berkeley.pdf |url-status=live |archive-url=https://web.archive.org/web/20220508101103/https://www.iangoodfellow.com/slides/2016-08-31-Berkeley.pdf |archive-date=8 May 2022}}

Hinge loss GAN:{{cite arXiv |last1=Lim |first1=Jae Hyun |last2=Ye |first2=Jong Chul |date=2017-05-08 |title=Geometric GAN |class=stat.ML |eprint=1705.02894 }}$$L\_D\; =\; -\backslash operatorname\; E\_\{x\backslash sim\; p\_\{\backslash text\{ref\}\}\}\backslash left[\backslash min\backslash left(0,\; -1\; +\; D(x)\backslash right)\backslash right]\; -\backslash operatorname\; E\_\{x\backslash sim\backslash mu\_G\}\backslash left[\backslash min\backslash left(0,\; -1\; -\; D\backslash left(x\backslash right)\backslash right)\backslash right]$$$$L\_G\; =\; -\backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_G\}\; [D(x)]$$Least squares GAN:{{Cite book |last1=Mao |first1=Xudong |last2=Li |first2=Qing |last3=Xie |first3=Haoran |last4=Lau |first4=Raymond Y. K. |last5=Wang |first5=Zhen |last6=Paul Smolley |first6=Stephen |title=2017 IEEE International Conference on Computer Vision (ICCV) |date=2017 |chapter=Least Squares Generative Adversarial Networks |chapter-url=https://openaccess.thecvf.com/content_iccv_2017/html/Mao_Least_Squares_Generative_ICCV_2017_paper.html |pages=2794–2802 |doi=10.1109/ICCV.2017.304 |arxiv=1611.04076|isbn=978-1-5386-1032-9 }}$$L\_D\; =\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\}\}[(D(x)-b)^2]\; +\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_G\}[(D(x)-a)^2]$$$$L\_G\; =\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_G\}[(D(x)-c)^2]$$where $a,\; b,\; c$ are parameters to be chosen. The authors recommended $a\; =\; -1,\; b\; =\; 1,\; c\; =\; 0$.

{{Main|Wasserstein GAN}}

The Wasserstein GAN modifies the GAN game at two points:

• The discriminator's strategy set is the set of measurable functions of type $D:\; \backslash Omega\; \backslash to\; \backslash R$ with bounded Lipschitz norm: $\backslash |D\backslash |\_L\; \backslash leq\; K$, where $K$ is a fixed positive constant.
• The objective is$$L\_\{WGAN\}(\backslash mu\_G,\; D)\; :=\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_G\}[D(x)]\; -\backslash mathbb\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\}\}[D(x)]$$

One of its purposes is to solve the problem of mode collapse (see above). The authors claim "In no experiment did we see evidence of mode collapse for the WGAN algorithm".

An adversarial autoencoder (AAE){{cite arXiv |eprint=1511.05644 |class=cs.LG |first1=Alireza |last1=Makhzani |first2=Jonathon |last2=Shlens |title=Adversarial Autoencoders |first3=Navdeep |last3=Jaitly |first4=Ian |last4=Goodfellow |first5=Brendan |last5=Frey |author-link4=Ian Goodfellow |author-link5=Brendan Frey |year=2016}} is more autoencoder than GAN. The idea is to start with a plain autoencoder, but train a discriminator to discriminate the latent vectors from a reference distribution (often the normal distribution).

In conditional GAN, the generator receives both a noise vector $z$ and a label $c$, and produces an image $G(z,\; c)$. The discriminator receives image-label pairs $(x,\; c)$, and computes $D(x,\; c)$.

When the training dataset is unlabeled, conditional GAN does not work directly.

The idea of InfoGAN is to decree that every latent vector in the latent space can be decomposed as $(z,\; c)$: an incompressible noise part $z$, and an informative label part $c$, and encourage the generator to comply with the decree, by encouraging it to maximize $I(c,\; G(z,\; c))$, the mutual information between $c$ and $G(z,\; c)$, while making no demands on the mutual information $z$ between $G(z,\; c)$.

Unfortunately, $I(c,\; G(z,\; c))$ is intractable in general, The key idea of InfoGAN is Variational Mutual Information Maximization:{{Cite journal |last1=Barber |first1=David |last2=Agakov |first2=Felix |date=2003-12-09 |title=The IM algorithm: a variational approach to Information Maximization |url=https://dl.acm.org/doi/abs/10.5555/2981345.2981371 |journal=Proceedings of the 16th International Conference on Neural Information Processing Systems |series=NIPS'03 |location=Cambridge, MA, USA |publisher=MIT Press |pages=201–208 }} indirectly maximize it by maximizing a lower bound$$\{\backslash hat\; \{I\}\}(G,Q)=\backslash mathbb\; \{E\}\; \_\{z\backslash sim\; \backslash mu\_Z,\; c\backslash sim\; \backslash mu\; \_\{C\}\}[\backslash ln\; Q(c\backslash mid\; G(z,c))];\; \backslash quad\; I(c,\; G(z,\; c))\; \backslash geq\; \backslash sup\_Q\; \backslash hat\; I(G,\; Q)$$where $Q$ ranges over all Markov kernels of type $Q:\; \backslash Omega\_Y\; \backslash to\; \backslash mathcal\; P(\backslash Omega\_C)$.

The InfoGAN game is defined as follows:{{Cite journal |last1=Chen |first1=Xi |last2=Duan |first2=Yan |last3=Houthooft |first3=Rein |last4=Schulman |first4=John |last5=Sutskever |first5=Ilya |last6=Abbeel |first6=Pieter |date=2016 |title=InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets |url=https://proceedings.neurips.cc/paper/2016/hash/7c9d0b1f96aebd7b5eca8c3edaa19ebb-Abstract.html |journal=Advances in Neural Information Processing Systems |publisher=Curran Associates, Inc. |volume=29|arxiv=1606.03657 }}

Three probability spaces define an InfoGAN game:

• $(\backslash Omega\_X,\; \backslash mu\_\{\backslash text\{ref\}\})$, the space of reference images.
• $(\backslash Omega\_Z,\; \backslash mu\_Z)$, the fixed random noise generator.
• $(\backslash Omega\_C,\; \backslash mu\_C)$, the fixed random information generator.

There are 3 players in 2 teams: generator, Q, and discriminator. The generator and Q are on one team, and the discriminator on the other team.

The objective function is$$L(G,\; Q,\; D)\; =\; L\_\{GAN\}(G,\; D)\; -\; \backslash lambda\; \backslash hat\; I(G,\; Q)$$where $L\_\{GAN\}(G,\; D)\; =\; \backslash operatorname\; E\_\{x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\},\; \}[\backslash ln\; D(x)]\; +\; \backslash operatorname\; E\_\{z\backslash sim\; \backslash mu\_Z\}[\backslash ln\; (1-D(G(z,\; c)))]$ is the original GAN game objective, and $\backslash hat\; I(G,\; Q)\; =\; \backslash mathbb\; E\_\{z\backslash sim\backslash mu\_Z,\; c\backslash sim\backslash mu\_C\}[\backslash ln\; Q(c\; \backslash mid\; G(z,\; c))]$

Generator-Q team aims to minimize the objective, and discriminator aims to maximize it:$$\backslash min\_\{G,\; Q\}\; \backslash max\_D\; L(G,\; Q,\; D)$$

The standard GAN generator is a function of type $G:\; \backslash Omega\_Z\backslash to\; \backslash Omega\_X$, that is, it is a mapping from a latent space $\backslash Omega\_Z$ to the image space $\backslash Omega\_X$. This can be understood as a "decoding" process, whereby every latent vector $z\backslash in\; \backslash Omega\_Z$ is a code for an image $x\backslash in\; \backslash Omega\_X$, and the generator performs the decoding. This naturally leads to the idea of training another network that performs "encoding", creating an autoencoder out of the encoder-generator pair.

Already in the original paper, the authors noted that "Learned approximate inference can be performed by training an auxiliary network to predict $z$ given $x$". The bidirectional GAN architecture performs exactly this.{{cite arXiv |eprint=1605.09782 |class=cs.LG |first1=Jeff |last1=Donahue |first2=Philipp |last2=Krähenbühl |title=Adversarial Feature Learning |first3=Trevor |last3=Darrell |author-link3=Trevor Darrell |year=2016}}

The BiGAN is defined as follows:

Two probability spaces define a BiGAN game:

• $(\backslash Omega\_X,\; \backslash mu\_\{X\})$, the space of reference images.
• $(\backslash Omega\_Z,\; \backslash mu\_Z)$, the latent space.

There are 3 players in 2 teams: generator, encoder, and discriminator. The generator and encoder are on one team, and the discriminator on the other team.

The generator's strategies are functions $G:\backslash Omega\_Z\; \backslash to\; \backslash Omega\_X$, and the encoder's strategies are functions $E:\backslash Omega\_X\; \backslash to\; \backslash Omega\_Z$. The discriminator's strategies are functions $D:\backslash Omega\_X\; \backslash to\; [0,\; 1]$.

The objective function is$$L(G,\; E,\; D)\; =\; \backslash mathbb\; E\_\{x\backslash sim\; \backslash mu\_X\}[\backslash ln\; D(x,\; E(x))]\; +\; \backslash mathbb\; E\_\{z\backslash sim\; \backslash mu\_Z\}[\backslash ln\; (1-D(G(z),\; z))]$$

Generator-encoder team aims to minimize the objective, and discriminator aims to maximize it:$$\backslash min\_\{G,\; E\}\; \backslash max\_D\; L(G,\; E,\; D)$$

Applications of bidirectional models include semi-supervised learning,{{cite arXiv |eprint=1606.00704 |class=stat.ML |first1=Vincent |last1=Dumoulin |first2=Ishmael |last2=Belghazi |title=Adversarially Learned Inference |first3=Ben |last3=Poole |first4=Olivier |last4=Mastropietro |first5=Alex |last5=Arjovsky |first6=Aaron |last6=Courville |year=2016}} interpretable machine learning,{{cite arXiv |eprint=1606.03657 |class=cs.LG |author1=Xi Chen |author2=Yan Duan |title=InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets |author3=Rein Houthooft |author4=John Schulman |author5=Ilya Sutskever |author-link5=Ilya Sutskever |author6=Pieter Abeel |author-link6=Pieter Abbeel |year=2016}} and neural machine translation.{{Cite web |author1=Zhirui Zhang |author2=Shujie Liu |author3=Mu Li |author4=Ming Zhou |author5=Enhong Chen |date=October 2018 |title=Bidirectional Generative Adversarial Networks for Neural Machine Translation |url=https://www.aclweb.org/anthology/K18-1019.pdf |pages=190–199 |volume=Proceedings of the 22nd Conference on Computational Natural Language Learning}}

CycleGAN is an architecture for performing translations between two domains, such as between photos of horses and photos of zebras, or photos of night cities and photos of day cities.

The CycleGAN game is defined as follows:{{cite arXiv |last1=Zhu |first1=Jun-Yan |last2=Park |first2=Taesung |last3=Isola |first3=Phillip |last4=Efros |first4=Alexei A. |date=2017 |title=Unpaired Image-To-Image Translation Using Cycle-Consistent Adversarial Networks |pages=2223–2232|class=cs.CV |eprint=1703.10593 }}

There are two probability spaces $(\backslash Omega\_X,\; \backslash mu\_X),\; (\backslash Omega\_Y,\; \backslash mu\_Y)$, corresponding to the two domains needed for translations fore-and-back.

There are 4 players in 2 teams: generators $G\_X:\; \backslash Omega\_X\; \backslash to\; \backslash Omega\_Y,\; G\_Y:\; \backslash Omega\_Y\; \backslash to\; \backslash Omega\_X$, and discriminators $D\_X:\; \backslash Omega\_X\backslash to\; [0,\; 1],\; D\_Y:\backslash Omega\_Y\backslash to\; [0,\; 1]$.

The objective function is$$L(G\_X,\; G\_Y,\; D\_X,\; D\_Y)\; =\; L\_\{GAN\}(G\_X,\; D\_X)\; +L\_\{GAN\}(G\_Y,\; D\_Y)\; +\; \backslash lambda\; L\_\{cycle\}(G\_X,\; G\_Y)$$

where $\backslash lambda$ is a positive adjustable parameter, $L\_\{GAN\}$ is the GAN game objective, and $L\_\{cycle\}$ is the cycle consistency loss:$$L\_\{cycle\}(G\_X,\; G\_Y)\; =\; E\_\{x\backslash sim\; \backslash mu\_X\}\; \backslash |G\_X(G\_Y(x))\; -\; x\backslash |\; +\; E\_\{y\backslash sim\; \backslash mu\_Y\}\; \backslash |G\_Y(G\_X(y))\; -\; y\backslash |$$The generators aim to minimize the objective, and the discriminators aim to maximize it:$$\backslash min\_\{G\_X,\; G\_Y\}\; \backslash max\_\{D\_X,\; D\_Y\}\; L(G\_X,\; G\_Y,\; D\_X,\; D\_Y)$$

The BigGAN is essentially a self-attention GAN trained on a large scale (up to 80 million parameters) to generate large images of ImageNet (up to 512 x 512 resolution), with numerous engineering tricks to make it converge.{{Cite conference |last1=Brock |first1=Andrew |last2=Donahue |first2=Jeff |last3=Simonyan |first3=Karen |date=2018-09-01 |title=Large Scale GAN Training for High Fidelity Natural Image Synthesis |arxiv=1809.11096 |url=https://ui.adsabs.harvard.edu/abs/2018arXiv180911096B |conference=International Conference on Learning Representations 2019}}{{Cite web |last=Brownlee |first=Jason |date=2019-08-22 |title=A Gentle Introduction to BigGAN the Big Generative Adversarial Network |url=https://machinelearningmastery.com/a-gentle-introduction-to-the-biggan/ |access-date=2022-07-15 |website=Machine Learning Mastery |language=en-US}}

When there is insufficient training data, the reference distribution $\backslash mu\_\{\backslash text\{ref\}\}$ cannot be well-approximated by the empirical distribution given by the training dataset. In such cases, data augmentation can be applied, to allow training GAN on smaller datasets. Naïve data augmentation, however, brings its problems.

Consider the original GAN game, slightly reformulated as follows:$$\backslash begin\{cases\}$$

\min_D L_D(D, \mu_G) = -\operatorname E_{x\sim \mu_{\text{ref}}}[\ln D(x)] - \operatorname E_{x\sim \mu_G}[\ln (1-D(x))]\\

\min_G L_G(D, \mu_G) = -\operatorname E_{x\sim \mu_G}[\ln (1-D(x))]

\end{cases}Now we use data augmentation by randomly sampling semantic-preserving transforms $T:\; \backslash Omega\; \backslash to\; \backslash Omega$ and applying them to the dataset, to obtain the reformulated GAN game:$$\backslash begin\{cases\}$$

\min_D L_D(D, \mu_G) = -\operatorname E_{x\sim \mu_{\text{ref}}, T\sim \mu_\text{trans}}[\ln D(T(x))] - \operatorname E_{x\sim \mu_G}[\ln (1-D(x))]\\

\min_G L_G(D, \mu_G) = -\operatorname E_{x\sim \mu_G}[\ln (1-D(x))]

\end{cases}This is equivalent to a GAN game with a different distribution $\backslash mu\_\{\backslash text\{ref\}\}\text{'}$, sampled by $T(x)$, with $x\backslash sim\; \backslash mu\_\{\backslash text\{ref\}\},\; T\backslash sim\; \backslash mu\_\backslash text\{trans\}$. For example, if $\backslash mu\_\{\backslash text\{ref\}\}$ is the distribution of images in ImageNet, and $\backslash mu\_\backslash text\{trans\}$ samples identity-transform with probability 0.5, and horizontal-reflection with probability 0.5, then $\backslash mu\_\{\backslash text\{ref\}\}\text{'}$ is the distribution of images in ImageNet and horizontally-reflected ImageNet, combined.

The result of such training would be a generator that mimics $\backslash mu\_\{\backslash text\{ref\}\}\text{'}$. For example, it would generate images that look like they are randomly cropped, if the data augmentation uses random cropping.

The solution is to apply data augmentation to both generated and real images:$$\backslash begin\{cases\}$$

\min_D L_D(D, \mu_G) = -\operatorname E_{x\sim \mu_{\text{ref}}, T\sim \mu_\text{trans}}[\ln D(T(x))] - \operatorname E_{x\sim \mu_G, T\sim \mu_\text{trans}}[\ln (1-D(T(x)))]\\

\min_G L_G(D, \mu_G) = -\operatorname E_{x\sim \mu_G, T\sim \mu_\text{trans}}[\ln (1-D(T(x)))]

\end{cases}The authors demonstrated high-quality generation using just 100-picture-large datasets.{{Cite journal |last1=Shengyu |first1=Zhao |last2=Zhijian |first2=Liu |last3=Ji |first3=Lin |last4=Jun-Yan |first4=Zhu |last5=Song |first5=Han |date=2020 |title=Differentiable Augmentation for Data-Efficient GAN Training |url=https://proceedings.neurips.cc/paper/2020/hash/55479c55ebd1efd3ff125f1337100388-Abstract.html |journal=Advances in Neural Information Processing Systems |language=en |volume=33|arxiv=2006.10738 }}

The StyleGAN-2-ADA paper points out a further point on data augmentation: it must be invertible. Continue with the example of generating ImageNet pictures. If the data augmentation is "randomly rotate the picture by 0, 90, 180, 270 degrees with equal probability", then there is no way for the generator to know which is the true orientation: Consider two generators $G,\; G\text{'}$, such that for any latent $z$, the generated image $G(z)$ is a 90-degree rotation of $G\text{'}(z)$. They would have exactly the same expected loss, and so neither is preferred over the other.

The solution is to only use invertible data augmentation: instead of "randomly rotate the picture by 0, 90, 180, 270 degrees with equal probability", use "randomly rotate the picture by 90, 180, 270 degrees with 0.1 probability, and keep the picture as it is with 0.7 probability". This way, the generator is still rewarded to keep images oriented the same way as un-augmented ImageNet pictures.

Abstractly, the effect of randomly sampling transformations $T:\; \backslash Omega\; \backslash to\; \backslash Omega$ from the distribution $\backslash mu\_\backslash text\{trans\}$ is to define a Markov kernel $K\_\backslash text\{trans\}:\; \backslash Omega\; \backslash to\; \backslash mathcal\; P\; (\backslash Omega)$. Then, the data-augmented GAN game pushes the generator to find some $\backslash hat\; \backslash mu\_G\backslash in\; \backslash mathcal\; P(\backslash Omega)$, such that $$K\_\backslash text\{trans\}*\backslash mu\_\{\backslash text\{ref\}\}\; =\; K\_\backslash text\{trans\}*\backslash hat\backslash mu\_\{G\}$$where $*$ is the Markov kernel convolution.

A data-augmentation method is defined to be invertible if its Markov kernel $K\_\backslash text\{trans\}$ satisfies$$K\_\backslash text\{trans\}*\backslash mu=\; K\_\backslash text\{trans\}*\backslash mu\text{'}\; \backslash implies\; \backslash mu\; =\; \backslash mu\text{'}\; \backslash quad\; \backslash forall\; \backslash mu,\; \backslash mu\text{'}\; \backslash in\; \backslash mathcal\; P(\backslash Omega)$$Immediately by definition, we see that composing multiple invertible data-augmentation methods results in yet another invertible method. Also by definition, if the data-augmentation method is invertible, then using it in a GAN game does not change the optimal strategy $\backslash hat\; \backslash mu\_G$ for the generator, which is still $\backslash mu\_\{\backslash text\{ref\}\}$.

There are two prototypical examples of invertible Markov kernels:

Discrete case: Invertible stochastic matrices, when $\backslash Omega$ is finite.

For example, if $\backslash Omega\; =\; \backslash \{\backslash uparrow,\; \backslash downarrow,\; \backslash leftarrow,\; \backslash rightarrow\backslash \}$ is the set of four images of an arrow, pointing in 4 directions, and the data augmentation is "randomly rotate the picture by 90, 180, 270 degrees with probability $p$, and keep the picture as it is with probability $(1-3p)$", then the Markov kernel $K\_\backslash text\{trans\}$ can be represented as a stochastic matrix:$$[K\_\backslash text\{trans\}]\; =\; \backslash begin\{bmatrix\}$$

(1-3p) & p & p & p \\

p & (1-3p) & p & p \\

p & p & (1-3p) & p \\

p & p & p & (1-3p)

\end{bmatrix} and $K\_\backslash text\{trans\}$ is an invertible kernel iff $[K\_\backslash text\{trans\}]$ is an invertible matrix, that is, $p\; \backslash neq\; 1/4$.

Continuous case: The gaussian kernel, when $\backslash Omega\; =\; \backslash R^n$ for some $n\; \backslash geq\; 1$.

For example, if $\backslash Omega\; =\; \backslash R^\{256^2\}$ is the space of 256x256 images, and the data-augmentation method is "generate a gaussian noise $z\backslash sim\; \backslash mathcal\; N(0,\; I\_\{256^2\})$, then add $\backslash epsilon\; z$ to the image", then $K\_\backslash text\{trans\}$ is just convolution by the density function of $\backslash mathcal\; N(0,\; \backslash epsilon^2\; I\_\{256^2\})$. This is invertible, because convolution by a gaussian is just convolution by the heat kernel, so given any $\backslash mu\backslash in\backslash mathcal\; P(\backslash R^n)$, the convolved distribution $K\_\backslash text\{trans\}\; *\; \backslash mu$ can be obtained by heating up $\backslash R^n$ precisely according to $\backslash mu$, then wait for time $\backslash epsilon^2/4$. With that, we can recover $\backslash mu$ by running the heat equation backwards in time for $\backslash epsilon^2/4$.

More examples of invertible data augmentations are found in the paper.

SinGAN pushes data augmentation to the limit, by using only a single image as training data and performing data augmentation on it. The GAN architecture is adapted to this training method by using a multi-scale pipeline.

The generator $G$ is decomposed into a pyramid of generators $G\; =\; G\_1\; \backslash circ\; G\_2\; \backslash circ\; \backslash cdots\; \backslash circ\; G\_N$, with the lowest one generating the image $G\_N(z\_N)$ at the lowest resolution, then the generated image is scaled up to $r(G\_N(z\_N))$, and fed to the next level to generate an image $G\_\{N-1\}(z\_\{N-1\}\; +\; r(G\_N(z\_N)))$ at a higher resolution, and so on. The discriminator is decomposed into a pyramid as well.{{Cite book |last1=Shaham |first1=Tamar Rott |last2=Dekel |first2=Tali |last3=Michaeli |first3=Tomer |title=2019 IEEE/CVF International Conference on Computer Vision (ICCV) |chapter=SinGAN: Learning a Generative Model from a Single Natural Image |date=October 2019 |chapter-url=http://dx.doi.org/10.1109/iccv.2019.00467 |pages=4569–4579 |publisher=IEEE |doi=10.1109/iccv.2019.00467|arxiv=1905.01164 |isbn=978-1-7281-4803-8 |s2cid=145052179 }}

{{Main|StyleGAN}}

The StyleGAN family is a series of architectures published by Nvidia's research division.

Progressive GAN{{cite arXiv |last1=Karras |first1=Tero |last2=Aila |first2=Timo |last3=Laine |first3=Samuli |last4=Lehtinen |first4=Jaakko |date=2017-10-01 |title=Progressive Growing of GANs for Improved Quality, Stability, and Variation |class=cs.NE |eprint=1710.10196 }} is a method for training GAN for large-scale image generation stably, by growing a GAN generator from small to large scale in a pyramidal fashion. Like SinGAN, it decomposes the generator as$G\; =\; G\_1\; \backslash circ\; G\_2\; \backslash circ\; \backslash cdots\; \backslash circ\; G\_N$, and the discriminator as $D\; =\; D\_1\; \backslash circ\; D\_2\; \backslash circ\; \backslash cdots\; \backslash circ\; D\_N$.

During training, at first only $G\_N,\; D\_N$ are used in a GAN game to generate 4x4 images. Then $G\_\{N-1\},\; D\_\{N-1\}$ are added to reach the second stage of GAN game, to generate 8x8 images, and so on, until we reach a GAN game to generate 1024x1024 images.

To avoid shock between stages of the GAN game, each new layer is "blended in" (Figure 2 of the paper). For example, this is how the second stage GAN game starts:

• Just before, the GAN game consists of the pair $G\_N,\; D\_N$ generating and discriminating 4x4 images.
• Just after, the GAN game consists of the pair $((1-\backslash alpha)\; +\; \backslash alpha\backslash cdot\; G\_\{N-1\})\backslash circ\; u\; \backslash circ\; G\_N,\; D\_N\; \backslash circ\; d\; \backslash circ\; ((1-\backslash alpha)\; +\; \backslash alpha\backslash cdot\; D\_\{N-1\})$ generating and discriminating 8x8 images. Here, the functions $u,\; d$ are image up- and down-sampling functions, and $\backslash alpha$ is a blend-in factor (much like an alpha in image composing) that smoothly glides from 0 to 1.

File:StyleGAN-1_and_StyleGAN-2.png

StyleGAN-1 is designed as a combination of Progressive GAN with neural style transfer.{{Cite book |last1=Karras |first1=Tero |last2=Laine |first2=Samuli |last3=Aila |first3=Timo |title=2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |chapter=A Style-Based Generator Architecture for Generative Adversarial Networks |date=June 2019 |chapter-url=http://dx.doi.org/10.1109/cvpr.2019.00453 |pages=4396–4405 |publisher=IEEE |doi=10.1109/cvpr.2019.00453|arxiv=1812.04948 |isbn=978-1-7281-3293-8 |s2cid=54482423 }}

The key architectural choice of StyleGAN-1 is a progressive growth mechanism, similar to Progressive GAN. Each generated image starts as a constant $4\backslash times\; 4\; \backslash times\; 512$ array, and repeatedly passed through style blocks. Each style block applies a "style latent vector" via affine transform ("adaptive instance normalization"), similar to how neural style transfer uses Gramian matrix. It then adds noise, and normalize (subtract the mean, then divide by the variance).

At training time, usually only one style latent vector is used per image generated, but sometimes two ("mixing regularization") in order to encourage each style block to independently perform its stylization without expecting help from other style blocks (since they might receive an entirely different style latent vector).

After training, multiple style latent vectors can be fed into each style block. Those fed to the lower layers control the large-scale styles, and those fed to the higher layers control the fine-detail styles.

Style-mixing between two images $x,\; x\text{'}$ can be performed as well. First, run a gradient descent to find $z,\; z\text{'}$ such that $G(z)\backslash approx\; x,\; G(z\text{'})\backslash approx\; x\text{'}$. This is called "projecting an image back to style latent space". Then, $z$ can be fed to the lower style blocks, and $z\text{'}$ to the higher style blocks, to generate a composite image that has the large-scale style of $x$, and the fine-detail style of $x\text{'}$. Multiple images can also be composed this way.

StyleGAN-2 improves upon StyleGAN-1, by using the style latent vector to transform the convolution layer's weights instead, thus solving the "blob" problem.{{Cite book |last1=Karras |first1=Tero |last2=Laine |first2=Samuli |last3=Aittala |first3=Miika |last4=Hellsten |first4=Janne |last5=Lehtinen |first5=Jaakko |last6=Aila |first6=Timo |title=2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |chapter=Analyzing and Improving the Image Quality of StyleGAN |date=June 2020 |chapter-url=http://dx.doi.org/10.1109/cvpr42600.2020.00813 |pages=8107–8116 |publisher=IEEE |doi=10.1109/cvpr42600.2020.00813|arxiv=1912.04958 |isbn=978-1-7281-7168-5 |s2cid=209202273 }}

This was updated by the StyleGAN-2-ADA ("ADA" stands for "adaptive"),{{Cite journal |last1=Tero |first1=Karras |last2=Miika |first2=Aittala |last3=Janne |first3=Hellsten |last4=Samuli |first4=Laine |last5=Jaakko |first5=Lehtinen |last6=Timo |first6=Aila |date=2020 |title=Training Generative Adversarial Networks with Limited Data |url=https://proceedings.neurips.cc/paper/2020/hash/8d30aa96e72440759f74bd2306c1fa3d-Abstract.html |journal=Advances in Neural Information Processing Systems |language=en |volume=33}} which uses invertible data augmentation as described above. It also tunes the amount of data augmentation applied by starting at zero, and gradually increasing it until an "overfitting heuristic" reaches a target level, thus the name "adaptive".

StyleGAN-3{{Cite book |last=Timo |first=Karras, Tero Aittala, Miika Laine, Samuli Härkönen, Erik Hellsten, Janne Lehtinen, Jaakko Aila |url=http://worldcat.org/oclc/1269560084 |title=Alias-Free Generative Adversarial Networks |date=2021-06-23 |oclc=1269560084}} improves upon StyleGAN-2 by solving the "texture sticking" problem, which can be seen in the official videos.{{Cite web |last1=Karras |first1=Tero |last2=Aittala |first2=Miika |last3=Laine |first3=Samuli |last4=Härkönen |first4=Erik |last5=Hellsten |first5=Janne |last6=Lehtinen |first6=Jaakko |last7=Aila |first7=Timo |title=Alias-Free Generative Adversarial Networks (StyleGAN3) |url=https://nvlabs.github.io/stylegan3 |access-date=2022-07-16 |website=nvlabs.github.io}} They analyzed the problem by the Nyquist–Shannon sampling theorem, and argued that the layers in the generator learned to exploit the high-frequency signal in the pixels they operate upon.

To solve this, they proposed imposing strict lowpass filters between each generator's layers, so that the generator is forced to operate on the pixels in a way faithful to the continuous signals they represent, rather than operate on them as merely discrete signals. They further imposed rotational and translational invariance by using more signal filters. The resulting StyleGAN-3 is able to solve the texture sticking problem, as well as generating images that rotate and translate smoothly.

Other than for generative and discriminative modelling of data, GANs have been used for other things.

GANs have been used for transfer learning to enforce the alignment of the latent feature space, such as in deep reinforcement learning.{{cite arXiv |eprint=2102.07097 |class=cs.LG |first1=Bonnie |last1=Li |first2=Vincent |last2=François-Lavet |title=Domain Adversarial Reinforcement Learning |date=2021-02-14 |last3=Doan |first3=Thang |last4=Pineau |first4=Joelle}} This works by feeding the embeddings of the source and target task to the discriminator which tries to guess the context. The resulting loss is then (inversely) backpropagated through the encoder.

• Iteratively reconstruct astronomical images{{cite journal|last1=Schawinski|first1=Kevin|last2=Zhang|first2=Ce|last3=Zhang|first3=Hantian|last4=Fowler|first4=Lucas|last5=Santhanam|first5=Gokula Krishnan|s2cid=7213940|date=2017-02-01|title=Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit|journal=Monthly Notices of the Royal Astronomical Society: Letters|volume=467|issue=1|pages=L110–L114|arxiv=1702.00403|doi=10.1093/mnrasl/slx008|doi-access=free |bibcode=2017MNRAS.467L.110S}}
• Simulate gravitational lensing for dark matter research.{{cite news |last1=Kincade |first1=Kathy |title=Researchers Train a Neural Network to Study Dark Matter |url=https://www.rdmag.com/news/2019/05/researchers-train-neural-network-study-dark-matter |publisher=R&D Magazine}}{{cite news |last1=Kincade |first1=Kathy |title=CosmoGAN: Training a neural network to study dark matter |url=https://phys.org/news/2019-05-cosmogan-neural-network-dark.html |work=Phys.org |date=May 16, 2019}}{{cite web |title=Training a neural network to study dark matter |url=https://www.sciencedaily.com/releases/2019/05/190516145206.htm |website=Science Daily |date=May 16, 2019}}
• Model the distribution of dark matter in a particular direction in space and to predict the gravitational lensing that will occur.{{Cite web|url=https://www.theregister.co.uk/2019/05/20/neural_networks_dark_matter/|title=Cosmoboffins use neural networks to build dark matter maps the easy way|last=at 06:13|first=Katyanna Quach 20 May 2019|website=www.theregister.co.uk|language=en|access-date=2019-05-20}}{{Cite journal|last1=Mustafa|first1=Mustafa|last2=Bard|first2=Deborah|last3=Bhimji|first3=Wahid|last4=Lukić|first4=Zarija|last5=Al-Rfou|first5=Rami|last6=Kratochvil|first6=Jan M.|s2cid=126034204|date=2019-05-06|title=CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networks|journal=Computational Astrophysics and Cosmology|volume=6|issue=1|pages=1|doi=10.1186/s40668-019-0029-9|issn=2197-7909|bibcode=2019ComAC...6....1M|arxiv=1706.02390 |doi-access=free }}
• Model high energy jet formation{{cite journal|first1=Michela|last1=Paganini|first2=Luke|last2=de Oliveira|first3=Benjamin|last3=Nachman|s2cid=88514467|title=Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis|journal=Computing and Software for Big Science|doi= 10.1007/s41781-017-0004-6|year=2017|volume=1|issue=1 |page=4|arxiv=1701.05927|bibcode=2017CSBS....1....4D}} and showers through calorimeters of high-energy physics experiments.{{cite journal|first1=Michela|last1=Paganini|first2=Luke|last2=de Oliveira|first3=Benjamin|last3=Nachman|s2cid=3330974|title=Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multi-Layer Calorimeters|journal= Physical Review Letters|doi= 10.1103/PhysRevLett.120.042003|pmid=29437460|year=2018|volume=120|issue=4|page=042003|arxiv=1705.02355|bibcode=2018PhRvL.120d2003P}}{{cite journal|first1=Michela|last1=Paganini|first2=Luke|last2=de Oliveira|first3=Benjamin|last3=Nachman|s2cid=41265836|title=CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks|journal=Phys. Rev. D|doi=10.1103/PhysRevD.97.014021|year=2018|volume=97|issue=1|page=014021|arxiv=1712.10321|bibcode=2018PhRvD..97a4021P}}{{cite journal|first1=Martin|last1=Erdmann|first2=Jonas|last2=Glombitza|first3=Thorben|last3=Quast|s2cid=54216502|title=Precise Simulation of Electromagnetic Calorimeter Showers Using a Wasserstein Generative Adversarial Network|journal=Computing and Software for Big Science|doi=10.1007/s41781-018-0019-7|year=2019|volume=3|issue=1 |page=4|arxiv=1807.01954|bibcode=2019CSBS....3....4E }}{{cite journal|title=Fast and Accurate Simulation of Particle Detectors Using Generative Adversarial Networks|first1=Pasquale|last1=Musella|first2=Francesco|last2=Pandolfi|s2cid=119474793|year=2018|journal=Computing and Software for Big Science|doi=10.1007/s41781-018-0015-y|volume=2|issue=1 |page=8|arxiv=1805.00850|bibcode=2018CSBS....2....8M}}
• Approximate bottlenecks in computationally expensive simulations of particle physics experiments. Applications in the context of present and proposed CERN experiments have demonstrated the potential of these methods for accelerating simulation and/or improving simulation fidelity.{{cite web|title=Deep generative models for fast shower simulation in ATLAS|year=2018|url=https://atlas.web.cern.ch/Atlas/GROUPS/PHYSICS/PUBNOTES/ATL-SOFT-PUB-2018-001/}}{{cite journal|title=Fast simulation of muons produced at the SHiP experiment using Generative Adversarial Networks|journal=Journal of Instrumentation|volume=14|issue=11|pages=11028|first=Collaboration|last=SHiP|s2cid=202542604|year=2019|arxiv=1909.04451|doi=10.1088/1748-0221/14/11/P11028|bibcode=2019JInst..14P1028A}}
• Reconstruct velocity and scalar fields in turbulent flows.{{Cite journal |last1=Nista |first1=Ludovico |last2=Pitsch |first2=Heinz |last3=Schumann |first3=Christoph D. K. |last4=Bode |first4=Mathis |last5=Grenga |first5=Temistocle |last6=MacArt |first6=Jonathan F. |last7=Attili |first7=Antonio |date=2024-06-04 |title=Influence of adversarial training on super-resolution turbulence reconstruction |url=https://link.aps.org/doi/10.1103/PhysRevFluids.9.064601 |journal=Physical Review Fluids |volume=9 |issue=6 |pages=064601 |arxiv=2308.16015 |bibcode=2024PhRvF...9f4601N |doi=10.1103/PhysRevFluids.9.064601}}{{Cite journal |last1=Nista |first1=L. |last2=Schumann |first2=C. D. K. |last3=Grenga |first3=T. |last4=Attili |first4=A. |last5=Pitsch |first5=H. |date=2023-01-01 |title=Investigation of the generalization capability of a generative adversarial network for large eddy simulation of turbulent premixed reacting flows |url=https://www.sciencedirect.com/science/article/pii/S1540748922002851 |journal=Proceedings of the Combustion Institute |volume=39 |issue=4 |pages=5279–5288 |bibcode=2023PComI..39.5279N |doi=10.1016/j.proci.2022.07.244 |issn=1540-7489}}{{Cite journal |last1=Fukami |first1=Kai |last2=Fukagata |first2=Koji |last3=Taira |first3=Kunihiko |date=2020-08-01 |title=Assessment of supervised machine learning methods for fluid flows |url=https://doi.org/10.1007/s00162-020-00518-y |journal=Theoretical and Computational Fluid Dynamics |language=en |volume=34 |issue=4 |pages=497–519 |arxiv=2001.09618 |bibcode=2020ThCFD..34..497F |doi=10.1007/s00162-020-00518-y |issn=1432-2250}}

GAN-generated molecules were validated experimentally in mice.{{cite journal |last1=Zhavoronkov |first1=Alex |date=2019 |title=Deep learning enables rapid identification of potent DDR1 kinase inhibitors |journal=Nature Biotechnology |volume=37 |issue=9 |pages=1038–1040 |doi=10.1038/s41587-019-0224-x |pmid=31477924 |s2cid=201716327}}{{cite magazine |last=Barber |first=Gregory |title=A Molecule Designed By AI Exhibits "Druglike" Qualities |url=https://www.wired.com/story/molecule-designed-ai-exhibits-druglike-qualities/ |magazine=Wired}}

One of the major concerns in medical imaging is preserving patient privacy. Due to these reasons, researchers often face difficulties in obtaining medical images for their research purposes. GAN has been used for generating synthetic medical images, such as MRI and PET images to address this challenge. {{cite journal | last1 = Moradi | first1 = M | last2 = Demirel | first2 = H | year = 2024 | title = Alzheimer's disease classification using 3D conditional progressive GAN-and LDA-based data selection | journal = Signal, Image and Video Processing | volume = 18 | issue = 2 | pages = 1847–1861 | doi = 10.1007/s11760-023-02878-4 }}

GAN can be used to detect glaucomatous images helping the early diagnosis which is essential to avoid partial or total loss of vision.{{cite journal |last1=Bisneto |first1=Tomaz Ribeiro Viana |last2=de Carvalho Filho |first2=Antonio Oseas |last3=Magalhães |first3=Deborah Maria Vieira |date=February 2020 |title=Generative adversarial network and texture features applied to automatic glaucoma detection |journal=Applied Soft Computing |volume=90 |pages=106165 |doi=10.1016/j.asoc.2020.106165 |s2cid=214571484}}

GANs have been used to create forensic facial reconstructions of deceased historical figures.{{Citation |title=Reconstruction of the Roman Emperors: Interview with Daniel Voshart |date=November 16, 2020 |url=https://www.youtube.com/watch?v=5mr6-JbuLrQ |access-date=2022-06-03 |language=en}}

{{Main|Deepfake}}

File:Woman 1.jpg that looks deceptively like a photograph of a real person. This image was generated by a StyleGAN based on an analysis of portraits.]]

image:GAN deepfake white girl.jpg

Concerns have been raised about the potential use of GAN-based human image synthesis for sinister purposes, e.g., to produce fake, possibly incriminating, photographs and videos.{{cite web|url=https://tech.slashdot.org/story/19/02/14/199200/this-person-does-not-exist-website-uses-ai-to-create-realistic-yet-horrifying-faces|title='This Person Does Not Exist' Website Uses AI To Create Realistic Yet Horrifying Faces|author=msmash|date=2019-02-14|website=Slashdot|access-date=2019-02-16}}

GANs can be used to generate unique, realistic profile photos of people who do not exist, in order to automate creation of fake social media profiles.{{cite news |last1=Doyle |first1=Michael |title=John Beasley lives on Saddlehorse Drive in Evansville. Or does he? |url=https://www.courierpress.com/story/news/crime/2019/05/16/john-beasley-lives-saddlehorse-drive-evansville-does-he/3700111002/ |publisher=Courier and Press |date=May 16, 2019}}

In 2019 the state of California considered{{cite news |last1=Targett |first1=Ed |title=California moves closer to making deepfake pornography illegal |publisher=Computer Business Review |date=May 16, 2019}} and passed on October 3, 2019, the [https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201920200AB602 bill AB-602], which bans the use of human image synthesis technologies to make fake pornography without the consent of the people depicted, and [https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201920200AB730 bill AB-730], which prohibits distribution of manipulated videos of a political candidate within 60 days of an election. Both bills were authored by Assembly member Marc Berman and signed by Governor Gavin Newsom. The laws went into effect in 2020.

{{cite web

| last = Mihalcik

| first = Carrie

| title = California laws seek to crack down on deepfakes in politics and porn

| website = cnet.com

| publisher = CNET

| date = 2019-10-04

| url = https://www.cnet.com/news/california-laws-seek-to-crack-down-on-deepfakes-in-politics-and-porn/

| access-date = 2019-10-13 }}

DARPA's Media Forensics program studies ways to counteract fake media, including fake media produced using GANs.{{cite magazine |last1=Knight |first1=Will |title=The Defense Department has produced the first tools for catching deepfakes |url=https://www.technologyreview.com/s/611726/the-defense-department-has-produced-the-first-tools-for-catching-deepfakes/ |magazine=MIT Technology Review |date=Aug 7, 2018}}

GANs can be used to generate art; The Verge wrote in March 2019 that "The images created by GANs have become the defining look of contemporary AI art."{{cite news |last1=Vincent |first1=James |date=5 March 2019 |title=A never-ending stream of AI art goes up for auction |url=https://www.theverge.com/2019/3/5/18251267/ai-art-gans-mario-klingemann-auction-sothebys-technology |access-date=13 June 2020 |work=The Verge |language=en}} GANs can also be used to

• inpaint photographsYu, Jiahui, et al. "[http://openaccess.thecvf.com/content_cvpr_2018/papers/Yu_Generative_Image_Inpainting_CVPR_2018_paper.pdf Generative image inpainting with contextual attention]." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
• generate fashion models,{{cite web |last1=Wong |first1=Ceecee |date=May 27, 2019 |title=The Rise of AI Supermodels |url=https://www.cdotrends.com/story/14300/rise-ai-supermodels |website=CDO Trends}} shadows,{{cite book |last1=Taif |first1=K. |title=Computational Science – ICCS 2020 |last2=Ugail |first2=H. |last3=Mehmood |first3=I. |year=2020 |isbn=978-3-030-50425-0 |series=Lecture Notes in Computer Science |volume=12141 |pages=481–495 |chapter=Cast Shadow Generation Using Generative Adversarial Networks |doi=10.1007/978-3-030-50426-7_36 |pmc=7302543}} photorealistic renders of interior design, industrial design, shoes, etc.{{Cite web |last=Wei |first=Jerry |date=2019-07-03 |title=Generating Shoe Designs with Machine Learning |url=https://towardsdatascience.com/generating-shoe-designs-with-deep-learning-5dde432a23b8 |access-date=2019-11-06 |website=Medium |language=en}} Such networks were reported to be used by Facebook.{{cite web |last1=Greenemeier |first1=Larry |date=June 20, 2016 |title=When Will Computers Have Common Sense? Ask Facebook |url=https://www.scientificamerican.com/article/when-will-computers-have-common-sense-ask-facebook/ |access-date=July 31, 2016 |website=Scientific American}}

Some have worked with using GAN for artistic creativity, as "creative adversarial network".{{cite arXiv |eprint=1706.07068 |class=cs.AI |first1=Ahmed |last1=Elgammal |first2=Bingchen |last2=Liu |title=CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms |first3=Mohamed |last3=Elhoseiny |first4=Marian |last4=Mazzone |year=2017}}{{cite journal |author1=Mazzone, Marian |author2=Ahmed Elgammal |date=21 February 2019 |title=Art, Creativity, and the Potential of Artificial Intelligence |journal=Arts |volume=8 |pages=26 |doi=10.3390/arts8010026 |doi-access=free}} A GAN, trained on a set of 15,000 portraits from WikiArt from the 14th to the 19th century, created the 2018 painting Edmond de Belamy, which sold for US$432,500.{{cite news |last1=Cohn |first1=Gabe |date=2018-10-25 |title=AI Art at Christie's Sells for $432,500 |url=https://www.nytimes.com/2018/10/25/arts/design/ai-art-sold-christies.html |newspaper=The New York Times}}

GANs were used by the video game modding community to up-scale low-resolution 2D textures in old video games by recreating them in 4k or higher resolutions via image training, and then down-sampling them to fit the game's native resolution (resembling supersampling anti-aliasing).{{cite arXiv |eprint=1809.00219 |class=cs.CV |first1=Xiaoou |last1=Tang |first2=Yu |last2=Qiao |title=ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks |date=2018-09-01 |language=en |last3=Loy |first3=Chen Change |last4=Dong |first4=Chao |last5=Liu |first5=Yihao |last6=Gu |first6=Jinjin |last7=Wu |first7=Shixiang |last8=Yu |first8=Ke |last9=Wang |first9=Xintao}}

In 2020, Artbreeder was used to create the main antagonist in the sequel to the psychological web horror series Ben Drowned. The author would later go on to praise GAN applications for their ability to help generate assets for independent artists who are short on budget and manpower.{{Cite web |last=Allen |first=Eric Van |date=2020-07-08 |title=An Infamous Zelda Creepypasta Saga Is Using Artificial Intelligence to Craft Its Finale |url=https://www.usgamer.net/articles/ben-drowned-zelda-creepypasta-finale-artificial-intelligence |url-status=dead |archive-url=https://web.archive.org/web/20221107030028/https://www.usgamer.net/articles/ben-drowned-zelda-creepypasta-finale-artificial-intelligence |archive-date=November 7, 2022 |access-date=2022-11-07 |website=USgamer |language=en}}{{Cite web |last=arcadeattack |date=2020-09-28 |title=Arcade Attack Podcast – September (4 of 4) 2020 - Alex Hall (Ben Drowned) - Interview |url=https://www.arcadeattack.co.uk/podcast-september-4-2020/ |access-date=2022-11-07 |website=Arcade Attack |language=en-GB}}

In May 2020, Nvidia researchers taught an AI system (termed "GameGAN") to recreate the game of Pac-Man simply by watching it being played.{{cite news |date=2020-05-22 |title=Nvidia's AI recreates Pac-Man from scratch just by watching it being played |url=https://www.theverge.com/2020/5/22/21266251/nvidia-ai-gamegan-recreate-pac-man-virutal-environment |newspaper=The Verge}}{{cite arXiv |eprint=2005.12126 |class=cs.CV |author1=Seung Wook Kim |first2=Yuhao |last2=Zhou |title=Learning to Simulate Dynamic Environments with GameGAN |year=2020 |last3=Philion |first3=Jonah |last4=Torralba |first4=Antonio |last5=Fidler |first5=Sanja}}

In August 2019, a large dataset consisting of 12,197 MIDI songs each with paired lyrics and melody alignment was created for neural melody generation from lyrics using conditional GAN-LSTM (refer to sources at GitHub [https://github.com/yy1lab/Lyrics-Conditioned-Neural-Melody-Generation AI Melody Generation from Lyrics]).{{cite journal |last1=Yu |first1=Yi |last2=Canales |first2=Simon |year=2021 |title=Conditional LSTM-GAN for Melody Generation from Lyrics |journal=ACM Transactions on Multimedia Computing, Communications, and Applications |volume=17 |pages=1–20 |arxiv=1908.05551 |doi=10.1145/3424116 |issn=1551-6857 |s2cid=199668828}}

GANs have been used to

• show how an individual's appearance might change with age.{{cite arXiv |eprint=1702.01983 |class=cs.CV |first1=Grigory |last1=Antipov |first2=Moez |last2=Baccouche |title=Face Aging With Conditional Generative Adversarial Networks |last3=Dugelay |first3=Jean-Luc |year=2017}}
• reconstruct 3D models of objects from images,{{cite web|url=http://3dgan.csail.mit.edu/|title=3D Generative Adversarial Network|website=3dgan.csail.mit.edu}}
• generate novel objects as 3D point clouds,{{cite arXiv|eprint=1707.02392|last1=Achlioptas|first1=Panos|last2=Diamanti|first2=Olga|last3=Mitliagkas|first3=Ioannis|last4=Guibas|first4=Leonidas|title=Learning Representations and Generative Models for 3D Point Clouds|year=2018|class=cs.CV}}
• model patterns of motion in video.{{cite web|url=https://www.cs.columbia.edu/~vondrick/tinyvideo/|title=Generating Videos with Scene Dynamics |website=carlvondrick.com|bibcode=2016arXiv160902612V |last1=Vondrick |first1=Carl |last2=Pirsiavash |first2=Hamed |last3=Torralba |first3=Antonio |year=2016 |arxiv=1609.02612}}
• inpaint missing features in maps, transfer map styles in cartography{{cite journal|last1=Kang|first1=Yuhao|last2=Gao|first2=Song|last3=Roth|first3=Rob|s2cid=146808465|title=Transferring Multiscale Map Styles Using Generative Adversarial Networks|url=https://geods.geography.wisc.edu/archives/1192|journal=International Journal of Cartography|volume=5|issue=2–3|pages=115–141|year=2019|doi=10.1080/23729333.2019.1615729|bibcode=2019IJCar...5..115K|arxiv=1905.02200}} or augment street view imagery.{{cite journal|last1=Wijnands|first1=Jasper|last2=Nice|first2=Kerry|last3=Thompson|first3=Jason|last4=Zhao|first4=Haifeng|last5=Stevenson|first5=Mark|s2cid=155100183|title=Streetscape augmentation using generative adversarial networks: Insights related to health and wellbeing|journal=Sustainable Cities and Society|volume=49|pages=101602|year=2019|doi=10.1016/j.scs.2019.101602|arxiv=1905.06464|bibcode=2019SusCS..4901602W}}
• use feedback to generate images and replace image search systems.{{cite book|last1=Ukkonen|first1=Antti|last2=Joona|first2=Pyry|last3=Ruotsalo|first3=Tuukka|title=Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval |chapter=Generating Images Instead of Retrieving Them |chapter-url=https://doi.org/10.1145/3397271.3401129|series=|pages=1329–1338|year=2020|doi=10.1145/3397271.3401129|s2cid=220730163|hdl=10138/328471|isbn=9781450380164|hdl-access=free}}
• visualize the effect that climate change will have on specific houses.{{cite magazine |title=AI can show us the ravages of climate change |url=https://www.technologyreview.com/f/613547/ai-can-show-us-the-ravages-of-climate-change/ |magazine=MIT Technology Review |date=May 16, 2019}}
• reconstruct an image of a person's face after listening to their voice.{{cite news |last1=Christian |first1=Jon |title=ASTOUNDING AI GUESSES WHAT YOU LOOK LIKE BASED ON YOUR VOICE |url=https://futurism.com/the-byte/ai-guesses-appearance-voice |publisher=Futurism |date=May 28, 2019}}
• produces videos of a person speaking, given only a single photo of that person.{{cite magazine |last1=Kulp |first1=Patrick |date=May 23, 2019 |title=Samsung's AI Lab Can Create Fake Video Footage From a Single Headshot |url=https://www.adweek.com/digital/samsungs-ai-lab-can-create-fake-video-footage-from-a-single-headshot/ |magazine=AdWeek}}
• recurrent sequence generation.{{Cite journal |author1=Mohammad Navid Fekri |author2=Ananda Mohon Ghosh |author3=Katarina Grolinger |date=2020 |title=Generating Energy Data for Machine Learning with Recurrent Generative Adversarial Networks |journal=Energies |volume=13 |issue=1 |page=130 |doi=10.3390/en13010130 |doi-access=free}}

In 1991, Juergen Schmidhuber published "artificial curiosity", neural networks in a zero-sum game.{{cite conference| title = A possibility for implementing curiosity and boredom in model-building neural controllers | last1 = Schmidhuber | first1 = Jürgen | author-link = Juergen Schmidhuber | date = 1991 | publisher = MIT Press/Bradford Books| book-title = Proc. SAB'1991| pages = 222–227}} The first network is a generative model that models a probability distribution over output patterns. The second network learns by gradient descent to predict the reactions of the environment to these patterns. GANs can be regarded as a case where the environmental reaction is 1 or 0 depending on whether the first network's output is in a given set.{{Cite journal |last=Schmidhuber |first=Jürgen |author-link=Juergen Schmidhuber |date=2020 |title=Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991) |journal=Neural Networks |language=en |volume=127 |pages=58–66 |arxiv=1906.04493 |doi=10.1016/j.neunet.2020.04.008 |pmid=32334341 |s2cid=216056336}}

Other people had similar ideas but did not develop them similarly. An idea involving adversarial networks was published in a 2010 blog post by Olli Niemitalo.{{cite web

|title= A method for training artificial neural networks to generate missing data within a variable context

|last1= Niemitalo

|first1= Olli

|date= February 24, 2010

|access-date= February 22, 2019

|newspaper= Internet Archive (Wayback Machine)

|url= http://yehar.com:80/blog/?p=167

|archive-url= https://web.archive.org/web/20120312111546/http://yehar.com/blog/?p=167

|archive-date= March 12, 2012

|url-status= live

}} This idea was never implemented and did not involve stochasticity in the generator and thus was not a generative model. It is now known as a conditional GAN or cGAN.{{cite web | year = 2019

|title = GANs were invented in 2010?

|url = https://www.reddit.com/r/MachineLearning/comments/bnqm0p/d_gans_were_invented_in_2010/

|website = reddit r/MachineLearning | language=en-US|access-date=2019-05-28}} An idea similar to GANs was used to model animal behavior by Li, Gauci and Gross in 2013.{{cite conference

|chapter= A Coevolutionary Approach to Learn Animal Behavior Through Controlled Interaction

|last1= Li

|first1= Wei

|last2= Gauci

|first2= Melvin

|last3= Gross

|first3= Roderich

|title= Proceeding of the fifteenth annual conference on Genetic and evolutionary computation conference - GECCO '13

|date= July 6, 2013

|publisher= ACM

|book-title= Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation (GECCO 2013)

|pages= 223–230

|location= Amsterdam, the Netherlands

|doi= 10.1145/2463372.2465801|isbn= 9781450319638

}}

Another inspiration for GANs was noise-contrastive estimation,{{cite journal |last1=Gutmann |first1=Michael |last2=Hyvärinen |first2=Aapo |title=Noise-Contrastive Estimation |journal=International Conference on AI and Statistics |url=http://proceedings.mlr.press/v9/gutmann10a/gutmann10a.pdf}} which uses the same loss function as GANs and which Goodfellow studied during his PhD in 2010–2014.

Adversarial machine learning has other uses besides generative modeling and can be applied to models other than neural networks. In control theory, adversarial learning based on neural networks was used in 2006 to train robust controllers in a game theoretic sense, by alternating the iterations between a minimizer policy, the controller, and a maximizer policy, the disturbance.{{cite journal|title=Neurodynamic Programming and Zero-Sum Games for Constrained Control Systems|last1=Abu-Khalaf|first1=Murad|last2=Lewis|first2=Frank L.|last3=Huang|first3=Jie|s2cid=15680448|journal=IEEE Transactions on Neural Networks|volume=19|issue=7|pages=1243–1252|date=July 1, 2008|doi=10.1109/TNN.2008.2000204}}{{cite journal|title=Policy Iterations on the Hamilton–Jacobi–Isaacs Equation for H_∞ State Feedback Control With Input Saturation|journal=IEEE Transactions on Automatic Control |last1=Abu-Khalaf|first1=Murad|last2=Lewis|first2=Frank L.|last3=Huang|first3=Jie|s2cid=1338976|date=December 1, 2006|doi=10.1109/TAC.2006.884959}}

In 2017, a GAN was used for image enhancement focusing on realistic textures rather than pixel-accuracy, producing a higher image quality at high magnification.{{cite arXiv|last1=Sajjadi|first1=Mehdi S. M. |last2=Schölkopf|first2=Bernhard|last3=Hirsch|first3=Michael|date=2016-12-23|title=EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis|eprint=1612.07919|class=cs.CV}} In 2017, the first faces were generated.{{cite web|url=https://medium.com/@alagraphy/this-person-does-not-exist-neither-will-anything-if-artificial-intelligence-keeps-learning-1a9fcba728f|title=This Person Does Not Exist: Neither Will Anything Eventually with AI|date=March 20, 2019}} These were exhibited in February 2018 at the Grand Palais.{{cite web|url=https://www.issuewire.com/artificial-intelligence-enters-the-history-of-art-1620667772563815|title=ARTificial Intelligence enters the History of Art|date=December 28, 2018}}{{cite web|url=https://link.medium.com/MYqBrGHIKV|title=Le scandale de l'intelligence ARTificielle|author=Tom Février|date=2019-02-17}} Faces generated by StyleGAN{{cite web|url=https://github.com/NVlabs/stylegan|title=StyleGAN: Official TensorFlow Implementation|date=March 2, 2019|via=GitHub}} in 2019 drew comparisons with Deepfakes.{{cite web|url=https://www.inverse.com/article/53280-this-person-does-not-exist-gans-website=website=Inverse|title=This Person Does Not Exist Is the Best One-Off Website of 2019|last=Paez|first=Danny|date=2019-02-13|access-date=2019-02-16}}{{cite web|url=https://boingboing.net/2019/02/15/this-person-does-not-exist.html|title=This Person Does Not Exist|last=Beschizza|first=Rob|date=2019-02-15|website=Boing-Boing|access-date=2019-02-16}}{{cite web|url=https://www.lyrn.ai/2018/12/26/a-style-based-generator-architecture-for-generative-adversarial-networks/|title=Style-based GANs – Generating and Tuning Realistic Artificial Faces|last=Horev|first=Rani|date=2018-12-26|website=Lyrn.AI|access-date=2019-02-16|archive-date=November 5, 2020|archive-url=https://web.archive.org/web/20201105101517/https://www.lyrn.ai/2018/12/26/a-style-based-generator-architecture-for-generative-adversarial-networks/|url-status=dead}}

• {{annotated link|Artificial intelligence art}}
• {{annotated link|Deepfake}}
• {{annotated link|Deep learning}}
• {{annotated link|Diffusion model}}
• {{annotated link|Generative artificial intelligence}}
• {{annotated link|Synthetic media}}

{{reflist}}

{{Portal|Art}}

• {{cite news|url=https://www.technologyreview.com/s/603216/5-big-predictions-for-artificial-intelligence-in-2017/|title=5 Big Predictions for Artificial Intelligence in 2017|last=Knight|first=Will|newspaper=MIT Technology Review|access-date=2017-01-05}}
• {{cite arXiv|eprint=1812.04948|last1=Karras|first1=Tero|last2=Laine|first2=Samuli|last3=Aila|first3=Timo|title=A Style-Based Generator Architecture for Generative Adversarial Networks|year=2018|class=cs.NE}}
• [https://www.thispersondoesnotexist.com/ This Person Does Not Exist]{{snd}} photorealistic images of people who do not exist, generated by StyleGAN
• [https://thiscatdoesnotexist.com/ This Cat Does Not Exist] {{Webarchive|url=https://web.archive.org/web/20190305040119/https://thiscatdoesnotexist.com/ |date=March 5, 2019 }}{{snd}} photorealistic images of cats who do not exist, generated by StyleGAN
• {{cite arXiv|eprint=1906.01529|last1=Wang|first1=Zhengwei|last2=She|first2=Qi|last3=Ward|first3=Tomas E.|title=Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy|year=2019|class=cs.LG}}

{{Generative AI}}

{{Artificial intelligence navbox}}

Category:Neural network architectures

Category:Cognitive science

Category:Unsupervised learning

Category:Generative artificial intelligence