DeepDream is a computer vision program created by Google engineer Alexander Mordvintsev which uses a convolutional neural network to find and enhance patterns in images via algorithmic pareidolia, thus creating a dream-like hallucinogenic appearance in the deliberately over-processed images.
Google's program popularized the term (deep) "dreaming" to refer to the generation of images that produce desired activations in a trained deep network, and the term now refers to a collection of related approaches.
The DeepDream software originates in a deep convolutional network codenamed "Inception" after the film of the same name, was developed for the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) in 2014 and released in July 2015.
The software is designed to detect faces and other patterns in images, with the aim of automatically classifying images. However, once trained, the network can also be run in reverse, being asked to adjust the original image slightly so that a given output neuron (e.g. the one for faces or certain animals) yields a higher confidence score. This can be used for visualizations to understand the emergent structure of the neural network better, and is the basis for the DeepDream concept. However, after enough reiterations, even imagery initially devoid of the sought features will be adjusted enough that a form of pareidolia results, by which psychedelic and surreal images are generated algorithmically. The optimization resembles Backpropagation, however instead of adjusting the network weights, the weights are held fixed and the input is adjusted.
For example, an existing image can be altered so that it is "more cat-like", and the resulting enhanced image can be again input to the procedure. This usage resembles the activity of looking for animals or other patterns in clouds.
Applying gradient descent independently to each pixel of the input produces images in which adjacent pixels have little relation and thus the image has too much high frequency information. The generated images can be greatly improved by including a prior or regularizer that prefers inputs that have natural image statistics (without a preference for any particular image), or are simply smooth. For example, Mahendran et al. used the total variation regularizer that prefers images that are piecewise constant. Various regularizers are discussed further in. An in-depth, visual exploration of feature visualization and regularization techniques was published more recently.
The dreaming idea and name became popular on the internet in 2015 thanks to Google's DeepDream program. The idea dates from early in the history of neural networks, and similar methods have been used to synthesize visual textures. Related visualization ideas were developed (prior to Google's work) by several research groups.
After Google published their techniques and made their code open source, a number of tools in the form of web services, mobile applications, and desktop software appeared on the market to enable users to transform their own photos.
The dreaming idea can be applied to hidden (internal) neurons other than those in the output, which allows exploration of the roles and representations of various parts of the network. It is also possible to optimize the input to satisfy either a single neuron (this usage is sometimes called Activity Maximization) or an entire layer of neurons.
While dreaming is most often used for visualizing networks or producing computer art, it has recently been proposed that adding "dreamed" inputs to the training set can improve training times for abstractions in Computer Science.
The cited resemblance of the imagery to LSD- and psilocybin-induced hallucinations is suggestive of a functional resemblance between artificial neural networks and particular layers of the visual cortex.
- Mordvintsev, Alexander; Olah, Christopher; Tyka, Mike (2015). "DeepDream - a code example for visualizing Neural Networks". Google Research. Archived from the original on 2015-07-08.
- Mordvintsev, Alexander; Olah, Christopher; Tyka, Mike (2015). "Inceptionism: Going Deeper into Neural Networks". Google Research. Archived from the original on 2015-07-03.
- Szegedy, Christian; Liu, Wei; Jia, Yangqing; Sermanet, Pierre; Reed, Scott; Anguelov, Dragomir; Erhan, Dumitru; Vanhoucke, Vincent; Rabinovich, Andrew (2014). "Going Deeper with Convolutions". Computing Research Repository. arXiv:1409.4842. Bibcode:2014arXiv1409.4842S.
- Rich McCormick (7 July 2015). "Fear and Loathing in Las Vegas is terrifying through the eyes of a computer". The Verge. Retrieved 2015-07-25.
- Simonyan, Karen; Vedaldi, Andrea; Zisserman, Andrew (2014). Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. International Conference on Learning Representations Workshop.
- Mahendran, Aravindh; Vedaldi, Andrea (2015). Understanding Deep Image Representations by Inverting Them. IEEE Conference on Computer Vision and Pattern Recognition. arXiv:1412.0035. doi:10.1109/CVPR.2015.7299155.
- Yosinski, Jason; Clune, Jeff; Nguyen, Anh; Fuchs, Thomas (2015). Understanding Neural Networks Through Deep Visualization. Deep Learning Workshop, International Conference on Machine Learning (ICML) Deep Learning Workshop.
- Olah, Chris; Mordvintsev, Alexander; Schubert, Ludwig (2017-11-07). "Feature Visualization". Distill. 2 (11). doi:10.23915/distill.00007. ISSN 2476-0757.
- Lewis, J.P. (1988). Creation by refinement: a creativity paradigm for gradient descent learning networks. IEEE International Conference on Neural Networks. doi:10.1109/ICNN.1988.23933.
- Portilla, J; Simoncelli, Eero (2000). "A parametric texture model based on joint statistics of complex wavelet coefficients". International Journal of Computer Vision. doi:10.1023/A:1026553619983.
- Erhan, Dumitru. (2009). Visualizing Higher-Layer Features of a Deep Network (PDF). International Conference on Machine Learning Workshop on Learning Feature Hierarchies.
- deepdream on GitHub
- Daniel Culpan (2015-07-03). "These Google "Deep Dream" Images Are Weirdly Mesmerising". Wired. Retrieved 2015-07-25.
- Nguyen, Anh; Dosovitskiy, Alexey; Yosinski, Jason; Brox, Thomas (2016). Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. arxiv. arXiv:1605.09304. Bibcode:2016arXiv160509304N.
- Arora, Sanjeev; Liang, Yingyu; Tengyu, Ma (2016). Why are deep nets reversible: A simple theory, with implications for training. arxiv. arXiv:1511.05653. Bibcode:2015arXiv151105653A.
- Spratt, Emily L. (2017). "Dream Formulations and Deep Neural Networks: Humanistic Themes in the Iconology of the Machine-Learned Image" (PDF). Kunsttexte. Humboldt-Universität zu Berlin. 4.
- LaFrance, Adrienne. "When Robots Hallucinate". The Atlantic. Retrieved 24 September 2015.
- fosterthepeopleVEVO (2017-08-11), Foster The People - Doing It for the Money, retrieved 2017-08-15