Statistical Inference and Machine Learning for Big Data

This book initially presents a variety of advanced statistical methods at a level suitable for advanced undergraduate and graduate students as well as others interested in familiarizing themselves with this important subject. Later, it proceeds to illustrate these methods in the context of real life applications. The non specialist seldom gets to see the main focus of modern statistics. Through the presentation of several real life applications in a variety of areas such as genetics and environmental problems one begins to gain an appreciation of the challenges and the utility of statistics.
The book begins in Part I by outlining various data types and by indicating how these are normally represented graphically and subsequently analyzed. In Part II, Chapters 2 and 3 we introduce the basic tools in probability and statistics. Here, we have retained the most useful and relevant results pertinent to this book. In Chapter 4, we proceed with an introduction to multivariate methods and to copula methods. We illustrate a number of applications by presenting real life examples. In Chapter 5 we introduce nonparametric methods which are particularly useful in the analysis of BIG DATA when the underlying distributions are often unknown. Some emphasis is placed on the use of ranking methods. We continue with a discussion of exponential tilting and its applications in Chapter 6. There we discuss the subject of empirical Bayes and its application to micro-array data. In Chapter 7, we touch on counting data analysis and survival analysis. In Chapter 8, time series methods are briefly described both from the usual classical as well as from the state space modeling approaches. Estimating equations and empirical likelihood are discussed in Chapter 9. We present their application in nonparametric testing. Symbolic data analysis is a relatively new field which aims to reduce the dimension of the data through a process of aggregation. It forms the subject of Chapter 10 wherein traditional statistical methods are applied to aggregated medical data. In Part III we focus first on the subject of regression through the lens of machine learning. In Chapter 11 we describe regression methods from the machine learning point of view along with support vector machines often used to study interactions and classification. We then continue in Chapter 12 with the important topics of neural networks and text analytics. We conclude with Part IV by presenting the computational aspects of BIG DATA with special attention devoted to Markov Chain Monte Carlo methods and to Bayesian nonparametric statistics.

This book was written for two key audiences. It would serve as a handy desk reference for statistical methods at the undergraduate and graduate level. It would also be useful in courses which aim to provide an overview of modern statistics and its applications.