An Introduction to Deep Learning

Topics

This post covers the first lecture in the course: “An Introduction to Deep Learning.”

This lecture will begin by setting the stage for why deep learning is necessary. When curating data at scale, there are generally two approaches: human-designed rules and deep learning. While we’ll see throughout the course that the judicious use of rules has its place in many big data endeavors, generally the performance of rule-based methods is disappointing in comparison to deep learning. The lecture will provide a couple of case studies of deep learning applied to economic data. We will then turn to an overview of neural networks.

For those who need to brush up on deep learning methods, the online text Neural Networks and Deep Learning provides an introduction to the main concepts and how they relate. Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville is the standard reference in the field. It is best suited to those who have some basic familiarity with deep learning. Part I provides a solid introduction to the concepts in applied math that are required to understand deep learning (students in the course should be familiar with this material, but the book is a good refresher). I recommend the book Deep Learning with PyTorch to anyone who does not already have extensive familiarity with PyTorch. There are also numerous lectures available for free online.

Lecture Video

Lecture notes

References Cited in Lecture 1: Introduction to Deep Learning

Academic Papers

• LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep learning.” Nature 521, no. 7553 (2015): 436-444.

Other Resources

Textbooks

• Nielsen, Michael. Neural Networks and Deep Learning, 2019

• Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning, 2016, MIT Press

• Stevens, Eli, Luca Antiga, and Thomas Viehmann. Deep learning with PyTorch. Manning Publications Company, 2020.

Other Resources and Videos

• The Decade of Deep Learning

• This blog is a classic for neural networks background: https://colah.github.io/

• A basic introduction to neural nets: https://www.3blue1brown.com/topics/neural-networks

• Weights & biases tutorials: https://wandb.ai/site/tutorials

Code Bases

• PyTorch official examples: https://github.com/pytorch/examples. Often very helpful to review relevant examples once one has a use case in mind.

Image Source: https://www.freecodecamp.org/news/want-to-know-how-deep-learning-works-heres-a-quick-guide-for-everyone-1aedeca88076/