Example results achieved with the GAN. From left to right: the original SDSS image, the degraded image with a worse PSF and higher noise level (indicating the PSF and noise level used), the image as recovered by the GAN, and for comparison, the result of a deconvolution. This figure visually illustrates the GAN’s ability to recover features which conventional deconvolutions cannot.

Abstract: Observations of astrophysical objects such as galaxies are limited by various sources of random and systematic noise from the sky background, the optical system of the telescope and the detector used to record the data. Conventional deconvolution techniques are limited in their ability to recover features in imaging data by the Shannon-Nyquist sampling theorem. Here we train a generative adversarial network (GAN) on a sample of 4,550 images of nearby galaxies at 0.01<z<0.02 from the Sloan Digital Sky Survey and conduct 10X cross validation to evaluate the results. We present a method using a GAN trained on galaxy images that can recover features from artificially degraded images with worse seeing and higher noise than the original with a performance which far exceeds simple deconvolution. The ability to better recover detailed features such as galaxy morphology from low-signal-to-noise and low angular resolution imaging data significantly increases our ability to study existing data sets of astrophysical objects as well as future observations with observatories such as the Large Synoptic Sky Telescope (LSST) and the Hubble and James Webb space telescopes.

Selected Press Coverage

Weakly Supervised Neural Network with Astrophysics Knowledge

One limiting factor in training deep neural networks is often the number of training examples one can get. The folklore often tells us “the more, the better.” In GalaxyGAN, one training example is a pair of two images: one clear image without space noise and one that we can observe in the presence of space noise.

Direct acquisition of such training pairs could be difficult because our ability to get clear images is limited by the throughput of a few high-quality telescopes. Instead, we choose to weakly supervise a neural network to generate training images automatically. The observations are simple; astrophysicists have been studying sky noises for hundreds of years, and they already have some physical models to describe their current understanding of such noises. Why not take advantage of such noise models to automatically degrade a relatively clear image as the training set? This observation allows us to generate a large-enough training set in just a couple of hours to train our generative adversarial network. You can see in our paper that the results obtained by this weak supervision mechanism: they are quite good!

Applications to astrophysics data sets

The GAN method presented here opens up the possibility of recovering more information from existing and future imaging data. Applying this GAN to images of galaxies as demonstarted in the paper de-noises data and effectively improves the seeing. The method is thus similar to obtaining a deeper image with better seeing. The reliability of the method relies heavily on the training set: the GAN generally can only produce useful improvements for features it has been trained on. It can attempt to reconstruct previously unknown features though with a lower degree of reliability. We propose that the GAN presented here, and methods based on it can be applied to a wide range of existing and future sky surveys (such as Pan-STARRS, LSST, HSC and others).

Team Members


Lucas Fowler

Hantian Zhang

Gokula Santhanam

Kevin Schawinski

Ce Zhang