Polynomial Networks for Generative AI

• Leroy Kang, Electrical Engineering, Virginia Tech

• David Hong, ECE, University of Delaware

Neural networks are widely used for generative AI because of their ability to model complex nonlinear relationships in data. Nonlinearity here is commonly achieved through entrywise application of the ReLU activation function. This poster presents ongoing work on investigaing polynomial networks, which provide an alternative approach to introducing nonlinearity by instead using explicit multiplicative interactions. In particular, we study the ability of polynomial networks to learn to generate points on simple manifolds (circles and spirals) within a Generative Adversarial Network (GAN) framework. To gain insight, we compare their performance with Taylor series approximations that provide a mathematical baseline for studying the effect of polynomial degree. Experiments vary polynomial degree, hidden dimension, and latent dimension to examine how these design choices affect a polynomial network’s ability to reproduce the training distribution.