Computational Statistics Handbook with MATLAB by Wendy L. Martinez, Angel R. Martinez.pdf

【实例截图】

【核心代码】

Table of Contents
Preface to the Third Edition............................................................................... xix
Preface to the Second Edition............................................................................ xxi
Preface to the First Edition................................................................................ xxv
Chapter 1
Introduction
1.1 What Is Computational Statistics? .................................................................1
1.2 An Overview of the Book ...............................................................................3
Philosophy..................................................................................................... 3
What Is Covered ........................................................................................... 4
A Word About Notation.............................................................................. 5
1.3 MATLAB® Code ...............................................................................................6
Computational Statistics Toolbox .............................................................. 8
Internet Resources ........................................................................................ 8
1.4 Further Reading ...............................................................................................9
Chapter 2
Probability Concepts
2.1 Introduction ....................................................................................................11
2.2 Probability .......................................................................................................12
Background ................................................................................................. 12
Probability ................................................................................................... 14
Axioms of Probability................................................................................ 16
2.3 Conditional Probability and Independence ...............................................17
Conditional Probability ............................................................................. 17
Independence.............................................................................................. 18
Bayes’ Theorem........................................................................................... 19
2.4 Expectation ......................................................................................................21
Mean and Variance .................................................................................... 21
Skewness...................................................................................................... 23
Kurtosis........................................................................................................ 23
2.5 Common Distributions ..................................................................................24
Binomial....................................................................................................... 24
Poisson ......................................................................................................... 26
Uniform........................................................................................................ 29
Normal ......................................................................................................... 30
CompStats3.book Page ix Monday, November 16, 2015 1:55 PM
x Computational Statistics Handbook with MATLAB®, Third Edition
Exponential.................................................................................................. 34
Gamma......................................................................................................... 36
Chi-Square................................................................................................... 37
Weibull......................................................................................................... 38
Beta ............................................................................................................... 40
Student’s t Distribution ............................................................................. 42
Multivariate Normal .................................................................................. 43
Multivariate t Distribution........................................................................ 47
2.6 MATLAB® Code .............................................................................................48
2.7 Further Reading .............................................................................................49
Exercises ................................................................................................................51
Chapter 3
Sampling Concepts
3.1 Introduction ....................................................................................................55
3.2 Sampling Terminology and Concepts ........................................................55
Sample Mean and Sample Variance ........................................................ 57
Sample Moments ........................................................................................ 58
Covariance................................................................................................... 60
3.3 Sampling Distributions .................................................................................63
3.4 Parameter Estimation ....................................................................................65
Bias................................................................................................................ 66
Mean Squared Error................................................................................... 66
Relative Efficiency ...................................................................................... 67
Standard Error ............................................................................................ 67
Maximum Likelihood Estimation ............................................................ 68
Method of Moments................................................................................... 71
3.5 Empirical Distribution Function ..................................................................73
Quantiles...................................................................................................... 74
3.6 MATLAB® Code .............................................................................................78
3.7 Further Reading .............................................................................................79
Exercises ................................................................................................................80
Chapter 4
Generating Random Variables
4.1 Introduction ....................................................................................................83
4.2 General Techniques for Generating Random Variables ...........................83
Uniform Random Numbers...................................................................... 83
Inverse Transform Method ....................................................................... 86
Acceptance-Rejection Method .................................................................. 90
4.3 Generating Continuous Random Variables ...............................................94
Normal Distribution .................................................................................. 94
Exponential Distribution........................................................................... 94
Gamma......................................................................................................... 96
Chi-Square................................................................................................... 97
CompStats3.book Page x Monday, November 16, 2015 1:55 PM
Table of Contents xi
Beta ............................................................................................................... 99
Multivariate Normal ................................................................................ 101
Multivariate Student’s t Distribution .................................................... 103
Generating Variates on a Sphere............................................................ 105
4.4 Generating Discrete Random Variables ...................................................106
Binomial..................................................................................................... 106
Poisson ....................................................................................................... 108
Discrete Uniform ...................................................................................... 110
4.5 MATLAB® Code ...........................................................................................112
4.6 Further Reading ...........................................................................................114
Exercises ..............................................................................................................115
Chapter 5
Exploratory Data Analysis
5.1 Introduction ..................................................................................................117
5.2 Exploring Univariate Data ..........................................................................119
Histograms ................................................................................................ 119
Stem-and-Leaf........................................................................................... 122
Quantile-Based Plots — Continuous Distributions............................. 124
Quantile Plots — Discrete Distributions............................................... 132
Box Plots .................................................................................................... 138
5.3 Exploring Bivariate and Trivariate Data ...................................................143
Scatterplots ................................................................................................ 145
Surface Plots .............................................................................................. 147
Contour Plots ............................................................................................ 148
Bivariate Histogram ................................................................................. 149
3D Scatterplot............................................................................................ 155
5.4 Exploring Multi-Dimensional Data ...........................................................156
Scatterplot Matrix..................................................................................... 157
Slices and Isosurfaces............................................................................... 159
Glyphs ........................................................................................................ 164
Andrews Curves....................................................................................... 167
Parallel Coordinates................................................................................. 172
5.5 MATLAB® Code ...........................................................................................178
5.6 Further Reading ...........................................................................................180
Exercises ..............................................................................................................182
Chapter 6
Finding Structure
6.1 Introduction ..................................................................................................185
6.2 Projecting Data .............................................................................................186
6.3 Principal Component Analysis ..................................................................188
6.4 Projection Pursuit EDA ...............................................................................192
Projection Pursuit Index .......................................................................... 195
Finding the Structure ............................................................................... 196
CompStats3.book Page xi Monday, November 16, 2015 1:55 PM
xii Computational Statistics Handbook with MATLAB®, Third Edition
Structure Removal.................................................................................... 197
6.5 Independent Component Analysis ...........................................................202
6.6 Grand Tour ...................................................................................................209
6.7 Nonlinear Dimensionality Reduction .......................................................213
Multidimensional Scaling ....................................................................... 214
Isometric Feature Mapping (ISOMAP) ................................................. 217
6.8 MATLAB® Code ...........................................................................................222
6.9 Further Reading ...........................................................................................224
Exercises ..............................................................................................................227
Chapter 7
Monte Carlo Methods for Inferential Statistics
7.1 Introduction ..................................................................................................229
7.2 Classical Inferential Statistics .....................................................................230
Hypothesis Testing................................................................................... 230
Confidence Intervals ................................................................................ 239
7.3 Monte Carlo Methods for Inferential Statistics ........................................241
Basic Monte Carlo Procedure ................................................................. 242
Monte Carlo Hypothesis Testing ........................................................... 243
Monte Carlo Assessment of Hypothesis Testing................................. 248
7.4 Bootstrap Methods .......................................................................................252
General Bootstrap Methodology............................................................ 252
Bootstrap Estimate of Standard Error ................................................... 254
Bootstrap Estimate of Bias....................................................................... 257
Bootstrap Confidence Intervals.............................................................. 258
7.5 MATLAB® Code ...........................................................................................264
7.6 Further Reading ...........................................................................................265
Exercises ..............................................................................................................266
Chapter 8
Data Partitioning
8.1 Introduction ..................................................................................................269
8.2 Cross-Validation ...........................................................................................270
8.3 Jackknife ........................................................................................................277
8.4 Better Bootstrap Confidence Intervals ......................................................285
8.5 Jackknife-After-Bootstrap ...........................................................................289
8.6 MATLAB® Code ...........................................................................................292
8.7 Further Reading ...........................................................................................293
Exercises ..............................................................................................................293
Chapter 9
Probability Density Estimation
9.1 Introduction ..................................................................................................297
9.2 Histograms ....................................................................................................299
CompStats3.book Page xii Monday, November 16, 2015 1:55 PM
Table of Contents xiii
1D Histograms .......................................................................................... 299
Multivariate Histograms ......................................................................... 306
Frequency Polygons................................................................................. 307
Averaged Shifted Histograms ................................................................ 312
9.3 Kernel Density Estimation ..........................................................................318
Univariate Kernel Estimators ................................................................. 318
Multivariate Kernel Estimators .............................................................. 323
9.4 Finite Mixtures .............................................................................................325
Univariate Finite Mixtures ...................................................................... 327
Visualizing Finite Mixtures..................................................................... 329
Multivariate Finite Mixtures................................................................... 331
EM Algorithm for Estimating the Parameters ..................................... 334
Adaptive Mixtures ................................................................................... 339
9.5 Generating Random Variables ...................................................................344
9.6 MATLAB® Code ...........................................................................................352
9.7 Further Reading ...........................................................................................352
Exercises ..............................................................................................................354
Chapter 10
Supervised Learning
10.1 Introduction ................................................................................................359
10.2 Bayes Decision Theory ..............................................................................361
Estimating Class-Conditional Probabilities: Parametric Method ..... 363
Naive Bayes Classifiers............................................................................ 365
Estimating Class-Conditional Probabilities: Nonparametric............. 365
Bayes Decision Rule ................................................................................. 367
Likelihood Ratio Approach..................................................................... 372
10.3 Evaluating the Classifier ...........................................................................376
Independent Test Sample........................................................................ 376
Cross-Validation....................................................................................... 378
Receiver Operating Characteristic (ROC) Curve................................. 381
10.4 Classification Trees ....................................................................................387
Growing the Tree...................................................................................... 390
Pruning the Tree ....................................................................................... 394
Choosing the Best Tree ............................................................................ 398
Other Tree Methods ................................................................................. 407
10.5 Combining Classifiers ...............................................................................410
Bagging ...................................................................................................... 410
Boosting ..................................................................................................... 413
Arcing Classifiers ..................................................................................... 416
Random Forests ........................................................................................ 418
10.6 Nearest Neighbor Classifier .....................................................................419
10.7 Support Vector Machines .........................................................................422
Maximal Margin Classifier...................................................................... 422
Support Vector Classifier ........................................................................ 426
CompStats3.book Page xiii Monday, November 16, 2015 1:55 PM
xiv Computational Statistics Handbook with MATLAB®, Third Edition
Support Vector Machines........................................................................ 427
10.8 MATLAB® Code .........................................................................................433
10.9 Further Reading .........................................................................................436
Exercises ..............................................................................................................437
Chapter 11
Unsupervised Learning
11.1 Introduction ................................................................................................441
11.2 Measures of Distance .................................................................................442
11.3 Hierarchical Clustering .............................................................................444
11.4 K-Means Clustering ...................................................................................452
11.5 Model-Based Clustering ............................................................................455
Finite Mixture Models and the EM Algorithm .................................... 456
Model-Based Agglomerative Clustering .............................................. 460
Bayesian Information Criterion.............................................................. 463
Model-Based Clustering Procedure....................................................... 463
11.6 Assessing Cluster Results .........................................................................468
Mojena – Upper Tail Rule ....................................................................... 468
Silhouette Statistic .................................................................................... 469
Other Methods for Evaluating Clusters................................................ 472
11.7 MATLAB® Code .........................................................................................475
11.8 Further Reading .........................................................................................477
Exercises ..............................................................................................................478
Chapter 12
Parametric Models
12.1 Introduction ................................................................................................481
12.2 Spline Regression Models .........................................................................487
12.3 Logistic Regression ....................................................................................492
Creating the Model .................................................................................. 492
Interpreting the Model Parameters........................................................ 496
12.4 Generalized Linear Models ......................................................................498
Exponential Family Form........................................................................ 499
Generalized Linear Model ...................................................................... 504
Model Checking........................................................................................ 509
12.5 Model Selection and Regularization .......................................................517
Best Subset Selection ................................................................................ 518
Stepwise Regression................................................................................. 519
Ridge Regression ...................................................................................... 521
Lasso—Least Absolute Shrinkage and Selection Operator................ 527
Elastic Net.................................................................................................. 529
12.6 Partial Least Squares Regression .............................................................532
Principal Component Regression .......................................................... 533
Partial Least Squares Regression............................................................ 535
12.7 MATLAB® Code .........................................................................................538
CompStats3.book Page xiv Monday, November 16, 2015 1:55 PM
Table of Contents xv
12.8 Further Reading .........................................................................................540
Exercises ..............................................................................................................540
Chapter 13
Nonparametric Models
13.1 Introduction ................................................................................................543
13.2 Some Smoothing Methods ........................................................................544
Bin Smoothing........................................................................................... 545
Running Mean .......................................................................................... 547
Running Line............................................................................................. 548
Local Polynomial Regression – Loess.................................................... 549
Robust Loess ............................................................................................. 555
13.3 Kernel Methods ..........................................................................................558
Nadaraya–Watson Estimator.................................................................. 561
Local Linear Kernel Estimator................................................................ 562
13.4 Smoothing Splines .....................................................................................565
Natural Cubic Splines.............................................................................. 565
Reinsch Method for Finding Smoothing Splines................................. 567
Values for a Cubic Smoothing Spline.................................................... 569
Weighted Smoothing Spline ................................................................... 570
13.5 Nonparametric Regression – Other Details ...........................................572
Choosing the Smoothing Parameter...................................................... 572
Estimation of the Residual Variance...................................................... 577
Variability of Smooths ............................................................................. 577
13.6 Regression Trees .........................................................................................581
Growing a Regression Tree..................................................................... 583
Pruning a Regression Tree ...................................................................... 585
Selecting a Tree ......................................................................................... 587
13.7 Additive Models ........................................................................................591
13.8 Multivariate Adaptive Regression Splines .............................................597
13.9 MATLAB® Code .........................................................................................605
13.10 Further Reading .......................................................................................608
Exercises ..............................................................................................................610
Chapter 14
Markov Chain Monte Carlo Methods
14.1 Introduction ................................................................................................613
14.2 Background .................................................................................................614
Bayesian Inference.................................................................................... 614
Monte Carlo Integration.......................................................................... 615
Markov Chains ......................................................................................... 617
Analyzing the Output.............................................................................. 618
14.3 Metropolis-Hastings Algorithms .............................................................618
Metropolis-Hastings Sampler................................................................. 619
Metropolis Sampler.................................................................................. 621
CompStats3.book Page xv Monday, November 16, 2015 1:55 PM
xvi Computational Statistics Handbook with MATLAB®, Third Edition
Independence Sampler ............................................................................ 626
Autoregressive Generating Density ...................................................... 627
14.4 The Gibbs Sampler .....................................................................................630
14.5 Convergence Monitoring ..........................................................................640
Gelman and Rubin Method .................................................................... 642
Raftery and Lewis Method...................................................................... 645
14.6 MATLAB® Code .........................................................................................647
14.7 Further Reading .........................................................................................648
Exercises ..............................................................................................................649
Appendix A
MATLAB® Basics
A.1 Desktop Environment ................................................................................653
A.2 Getting Help and Other Documentation .................................................655
A.3 Data Import and Export .............................................................................656
Data Import and Export in Base MATLAB .......................................... 656
Data Import and Export with the Statistics Toolbox........................... 658
A.4 Data in MATLAB® ......................................................................................659
Data Objects in Base MATLAB............................................................... 659
Accessing Data Elements ........................................................................ 662
Object-Oriented Programming............................................................... 665
A.5 Workspace and Syntax ...............................................................................665
File and Workspace Management.......................................................... 666
Syntax in MATLAB .................................................................................. 667
Functions in MATLAB............................................................................. 669
A.6 Basic Plot Functions ....................................................................................670
Plotting 2D Data ....................................................................................... 670
Plotting 3D Data ....................................................................................... 673
Scatterplots ................................................................................................ 674
Scatterplot Matrix..................................................................................... 675
GUIs for Graphics..................................................................................... 675
A.7 Summary and Further Reading ................................................................677
Appendix B
Projection Pursuit Indexes
B.1 Friedman-Tukey Index ...............................................................................681
B.2 Entropy Index ..............................................................................................682
B.3 Moment Index ..............................................................................................682
B.4 Distances .......................................................................................................683
Appendix C
Data Sets
C.1 Introduction .................................................................................................685
C.2 Descriptions .................................................................................................685
CompStats3.book Page xvi Monday, November 16, 2015 1:55 PM
Table of Contents xvii
Appendix D
Notation
D.1 Observed Data .............................................................................................695
D.2 Greek Letters ................................................................................................696
D.3 Functions and Distributions ......................................................................696
D.4 Matrix Notation ...........................................................................................696
D.5 Statistics ........................................................................................................697
References ........................................................................................................... 699