The Science of Deep Learning

“From the available books on deep learning, this textbook is outstanding. Drori has provided an extensive overview of the field including reinforcement learning – in its technical meaning and in his successful, common-sense approach to teaching and understanding.” Gilbert Strang, Professor of Mathematics, Massachusetts Institute of Technology

“This book covers an impressive breadth of foundational concepts and algorithms behind modern deep learning. By reading this book, readers will quickly but thoroughly learn and appreciate foundations and advances of modern deep learning.” Kyunghyun Cho, Associate Professor of Computer Science and Data Science, New York University

“This book offers a fascinating tour of the field of deep learning, which in only ten years has come to revolutionize almost every area of computing. Drori provides concise descriptions of many of the most important developments, combining unified mathematical notation and ample figures to form an essential resource for students and practitioners alike.” Jonathan Ventura, Assistant Professor of Computer Science, Cal Poly

“Drori’s textbook goes under the hood of deep learning, covering a broad swath of modern techniques in optimization that are useful for efficiently training neural networks. The book also covers regularization methods to avoid overfitting, a common issue when working with deep learning models. Overall, this is an excellent textbook for students and practitioners who want to gain a deeper understanding of deep learning.” Madeleine Udell, Assistant Professor of Management Science and Engineering, Stanford University

"This new book by Prof. Drori brings fresh insights from his experience teaching thousands of students at Columbia, MIT, and NYU during the past several years. The book is a unique resource and opportunity for educators and researchers worldwide to build on his highly successful deep learning course.", Claudio Silva, Professor of Computer Science and Engineering, New York University

“Drori's book covers deep learning, from fundamentals to applications. The fundamentals are covered with clear figures and examples, making the underlying algorithms easy to understand for non-specialists. The multidisciplinary applications are thoughtfully selected to illustrate the broad applications of deep neural networks to specialized domains while highlighting the common themes and architectures between them.” Tonio Buonassisi, Professor of Mechanical Engineering, Massachusetts Institute of Technology

“This textbook provides an excellent introduction to contemporary methods and models in deep learning. I expect this book to become a key resource in data science education for students and researchers.” Nakul Verma, Lecturer of Computer Science, Columbia University

“Drori's textbook makes the learning curve for deep learning a whole lot easier to climb. It follows a rigid scientific narrative, accompanied by a trove of code examples and visualizations. These enable a truly multi-modal approach to learning that will allow many students to understand the material better and sets them on a path of exploration.” Joaquin Vanschoren, Assistant Professor of Machine Learning, Eindhoven University of Technology

The Science of Deep Learning emerged from courses taught by the author that have provided thousands of students with training and experience for their academic studies, and prepared them for jobs in deep learning, machine learning, and artificial intelligence in leading companies in industry and academia. The book begins by covering the foundations of deep learning, followed by key deep learning architectures. Subsequent parts on generative models and reinforcement learning may be used either as part of a deep learning course or as part of a course on each topic. The book includes state-of-the-art topics such as Transformers, graph neural networks, variational autoencoders, and deep reinforcement learning, with a broad range of applications. Appendices provide equations for computing gradients in backpropagation and optimization, and best practices in scientific writing and reviewing. The text presents an up-to-date guide to the field built upon clear visualizations using a unified notation and equations, lowering the barrier to entry for the reader. The accompanying web site provides complementary code and hundreds of exercises with solutions.

Contents

Part I: Foundations