High-Dimensional Statistics: A Non-Asymptotic Viewpoint

Description

Recent years have witnessed an explosion in the volume and variety of data collected in all scientific disciplines and industrial settings. Such massive data sets present a number of challenges to researchers in statistics and machine learning. This book provides a self-contained introduction to the area of high-dimensional statistics, aimed at the first-year graduate level. It includes chapters that are focused on core methodology and theory - including tail bounds, concentration inequalities, uniform laws and empirical process, and random matrices - as well as chapters devoted to in-depth exploration of particular model classes - including sparse linear models, matrix models with rank constraints, graphical models, and various types of non-parametric models. With hundreds of worked examples and exercises, this text is intended both for courses and for self-study by graduate students and researchers in statistics, machine learning, and related fields who must understand, apply, and adapt modern statistical methods suited to large-scale data.

Martin J. Wainwright is a Chancellor's Professor at the University of California, Berkeley, with a joint appointment between the Department of Statistics and the Department of Electrical Engineering and Computer Sciences. His research lies at the nexus of statistics, machine learning, optimization, and information theory, and he has published widely in all of these disciplines. He has written two other books, one on graphical models together with Michael I. Jordan, and one on sparse learning together with Trevor Hastie and Robert Tibshirani. Among other awards, he has received the COPSS Presdients' Award, has been a Medallion Lecturer and Blackwell Lecturer for the Institute of Mathematical Statistics, and has received Best Paper Awards from the Neural Information Processing Systems (NIPS), the International Conference on Machine Learning (ICML), and the Uncertainty in Artificial Intelligence (UAI) conferences, as well as from the Institute of Electrical and Electronics Engineers (IEEE) Information Theory Society.

'Non-asymptotic, high-dimensional theory is critical for modern statistics and machine learning. This book is unique in providing a crystal clear, complete and unified treatment of the area. With topics ranging from concentration of measure to graphical models, the author weaves together probability theory and its applications to statistics. Ideal for graduate students and researchers. This will surely be the standard reference on the topic for many years.' Larry Wasserman, Carnegie Mellon University, Pennsylvania 'Martin J. Wainwright brings his large box of analytical power tools to bear on the problems of the day - the analysis of models for wide data. A broad knowledge of this new area combines with his powerful analytical skills to deliver this impressive and intimidating work - bound to be an essential reference for all the brave souls that try their hand.' Trevor Hastie, Stanford University, California 'This book provides an excellent treatment of perhaps the fastest growing area within high-dimensional theoretical statistics - non-asymptotic theory that seeks to provide probabilistic bounds on estimators as a function of sample size and dimension. It offers the most thorough, clear, and engaging coverage of this area to date, and is thus poised to become the definitive reference and textbook on this topic.' Genevera Allen, William Marsh Rice University, Texas 'Statistical theory and practice have undergone a renaissance in the past two decades, with intensive study of high-dimensional data analysis. No researcher has deepened our understanding of high-dimensional statistics more than Martin Wainwright. This book brings the signature clarity and incisiveness of his published research into book form. It will be a fantastic resource for both beginning students and seasoned researchers, as the field continues to make exciting breakthroughs.' John Lafferty, Yale University, Connecticut 'This is an outstanding book on high-dimensional statistics, written by a creative and celebrated researcher in the field. It gives comprehensive treatments on many important topics in statistical machine learning and, furthermore, is self-contained, from introductory materials to most updated results on various research frontiers. This book is a must-read for those who wish to learn and to develop modern statistical machine theory, methods and algorithms.' Jianqing Fan, Princeton University, New Jersey 'This book provides an in-depth mathematical treatment and methodological intuition of high-dimensional statistics. The main technical tools from probability theory are carefully developed and the construction and analysis of statistical methods and algorithms for high-dimensional problems is presented in an outstandingly clear way. Martin J. Wainwright has written a truly exceptional, inspiring and beautiful masterpiece!' Peter Buhlmann, Eidgenoessische Technische Hochschule Zurich 'This new book by Martin J. Wainwright covers modern topics in high-dimensional statistical inference, and focuses primarily on explicit non-asymptotic results related to sparsity and non-parametric estimation. This is a must-read for all graduate students in mathematical statistics and theoretical machine learning, both for the breadth of recent advances it covers and the depth of results which are presented. The exposition is outstandingly clear, starting from the first introductory chapters on the necessary probabilistic tools. Then, the book covers state-of-the-art advances in high-dimensional statistics, with always a clever choice of results which have the perfect mix of significance and mathematical depth.' Francis Bach, INRIA Paris 'Wainwright's book on those parts of probability theory and mathematical statistics critical to understanding of the new phenomena encountered in high dimensions is marked by the clarity of its presentation and the depth to which it travels. In every chapter he starts with intuitive examples and simulations which are systematically developed either into powerful mathematical tools or complete answers to fundamental questions of inference. It is not easy, but elegant and rewarding whether read systematically or dipped into as a reference.' Peter Bickel, University of California, Berkeley '... this is a very valuable book, covering a variety of important topics, self-contained and nicely written.' Fabio Mainardi, MAA Reviews 'This is an excellent book. It provides a lucid, accessible and in-depth treatment of nonasymptotic high-dimensional statistical theory, which is critical as the underpinning of modern statistics and machine learning. It succeeds brilliantly in providing a self-contained overview of high-dimensional statistics, suitable for use in formal courses or for self-study by graduate-level students or researchers. The treatment is outstandingly clear and engaging, and the production is first-rate. It will quickly become essential reading and the key reference text in the field.' G. Alastair Young, International Statistical Review 'Martin Wainwright takes great care to polish every sentence of each part of the book. He introduces state-of-the-art theory in every chapter, as should probably be expected from an acknowledged specialist of the field. But it is certainly an enormous amount of work to organize all these results in a complete, coherent, rigorous yet easy-to-follow theory. I am simply amazed by the quality of the writing. The explanations on the motivations (Chapter 1) and on the core of the theory are extremely pedagogical. The proofs of the main results are rigorous and complete, but most of them are also built in a way that makes them seem easier to the reader than they actually are. This is the kind of magic only a few authors are capable of.' Pierre Alquier, MatSciNet '... provides a masterful exposition of various mathematical tools that are becoming increasingly common in the analysis of contemporary statistical problems. In addition to providing a rigorous and comprehensive overview of these tools, the author delves into the details of many illustrative examples to provide a convincing case for the general usefulness of the methods that are introduced.' Po-Ling Lo, Bulletin of the American Mathematical Society 'An excellent statistical masterpiece is in the hands of the reader, which is a must read book for all graduate students in both mathematical statistics and mathematical machine learning.' Rozsa Horvath-Bokor, ZB Math Reviews