This book chapter is published open access.
What lies at the heart of modern neural network-based machine learning is the ability to approximate very high dimensional functions with good accuracy. This opens up two major avenues of research. The first is to develop machine learning-based algorithms for scientific problems that suffer from the curse of dimensionality. The second is to build a theoretical framework that helps us to form a better foundation for machine learning. For the latter, the most important questions that need to be addressed include: Why do neural network models work so well in high dimension? Why does their performance depend so sensitively on the choice of the hyperparameters? Can we develop more robust and equally accurate new machine learning models and algorithms? In this article, we review some of the major progresses made in these directions.