Advanced Data Analysis from an Elementary Point of View

【实例简介】
Advanced Data Analysis from an Elementary Point of View
17: 09 Monday 3oth January, 2017 Copy right Cusma Rohilla Shalizi; do not distribute without per mission updates at ht tp: //wwwstat. cmu. edu/-cshalizi/ADAf aEPov/ Contents in Brief Regression and Its generalizations 26 Regression Basics The Truth about Linear Regression 3 Model evaluation 4 Smoothing in regression 97 s Simulatie 127 6 The Bootstrap 7 Weighting and Variance 168 8 Spline 9 Additive Models 210 10 Testing Regression Specifications 234 11 Logistic regressio 251 12 GLMs and GAMs rees 289 II Distributions and Latent Structure 313 14 Density Estimation 314 CONTENTS IN BRIEF 15 Relative distributions and Smooth Test 339 16 Principal Components analysis 368 17F 393 18 Nonlinear Dimensionality Reduction 424 19Mⅸ Xture Model图 50 20 Gra p hical models 81 皿 Causal Inference 502 21 Graphical Causal Models 503 2 Identifying causal Effects 23 Experimental Causal Inference 532 24 Estimating causal Effects 538 5 Discovering Causal Structure 550 Ⅳ 7 Dependent Data 568 6 Time series 569 27 Spatial and Network Data 619 8 Simulation-Based Inference 629 Appendices 642 Data-Analysis Problem Sets 642 Linear Algebra Reminders 727 C BigO and Little o Notation 736 Taylor Expansions 738 E Multivariate Distributions 740 F Algebra with Expectations and variances 752 17: 09 Monday 30th January, 2017 CONTENTS IN BRIEF G Propagation of Error 5 H Optimization 757 and Likelihood Ratios 777 Proof of the gauss-Markoy Theorem 78 K Rudimentary Graph Theory 782 L Information Theoryl 785 M More about Hypothesis Testing 786 N Programming 787 o Generating Random variables 82 Cknowledgments 831 Bibliography 833 17: 09 Monday 30th January, 2017 CONTENTS IN BRIEF 17: 09 Monday 30th January, 2017 17: 09 Monday 3oth January, 2017 Copy right Cusma Rohilla Shalizi; do not distribute without per mission updatesatht:tp://www.stat.cmuedu/-cshalizi/adafaepov/ Contents Introducti To the read 23 Concepts You Should Know 24 I Regression and Its generalizations 26 1 Regression Basics 1 Statistics, Data Analysis, Regression 1.2 Guessing the value of a Random variable 28 1.2.1 Estimating the Expected Value .. The Regression Function 2 1.3.1 Somc Disclaimers ,,,30 1.4 Estimating the Regression Function 1.4.1 The Bias-Variance Tradeoff 4.2 The Bias-Variance Trade-Off in Action 35 1.4.3 Ordinary least squares linear regression as Smoothing 35 5 Lincar Smoothers 1.5.1 k-Nearest- Neighbor Regression 1.5.2 Kernel Smoothers 1.5.3 Some General Theory for Linear Smoothers ............46 1.5.3.1 Standard error of predicted mean values 46 1.5.3.2(Effective) Degrees of Freedom 16 5.3.3 Prediction Errors 48 1.5.3.4 Inferential Statistics ,49 1.6 Further Reading 50 2 The Truth about Linear regression 52 2. 1 Optimal Linear Prediction: Multiple variables 52 2.1.1 Collinearity 2.1.2 The Prediction and Its Error 2.1.3 Estimating the Optimal Linear predictor CONTENTS 1.3.1 Unbiasedness and Variance of Ordinary Least Squares Estimates 2.2 Shifting Distributions, Omitted Variables, and Transformations..57 2.2.1 Changing Slopes 2.2. 1 1 R2: Distraction or Nuisance 2.2.2 Omitted Variables and Shifting distributions 2.2.3 Errors in Variables. 2.2.4 Transformation 翻潘番 64 2.3 Adding Probabilistic Assumptions 67 2.3.1 Examine the Residuals 68 2.3.2 On Significant Coefficients 69 .4 Linear Regression Is Not the Philosophers Stone 2.5 Further Readins 71 3 Model Evaluation 73 3.1 What Are Statistical Models For? 73 3.2 Errors, In and Out of sample 74 3.3 Over-Fitting and Model Selection 78 3.4 Cross- validat ···· 83 3.4.1 Data Splitting 3. 4.2 k-Fold Cross-Validation(CV) 3.4.3 Lcavc-onc-out Cross-Validaioll 85 3.5 Warnin 91 3.5.1 Inference after Selection 3.5.2 Parameter Interpretation 93 3.6 Further Reading 94 3.7 Exercise 95 4 Smoothing in Regression 97 4.1 How Much Should We Smooth? 7 4.2 Adapting to Unknown Roughness 100 4.2.1 Bandwidth Selection by Cross-Validation 106 4.2.2 Convergence of Kernel Smoothing and Bandwidth Scaling.. 108 4.2.3 Summary on Kernel Smoothing in ID 111 3 Kernel Regression with Multiple Input 4.4 Interpreting Smoothers: Plots 4.5 Average Predictive Comparisons 118 4.6 Computational Advice: npreg 120 4.7 Further Reading 126 4.8 Exerc 17: 09 Monday 30th January, 2017 CONTENTS 5 Simulation 127 5.1 What Is a Simulation: ,,,,,,,,,127 5.2 How Do We Simulate Stochastic Models 128 5.2.1 Chaining together Random Variables 128 5.2.2 Random Variable Generation 5.2.2.1 Built-in Random Number generat ,128 5.2.2.2 Transformations 5.2.2.3 Quantile Method 129 5.2.3 Samplin 130 5.2.3.2 Multinomials and Multinoul es, 5.2.3.1 Sampling Rows from Data Frames 131 131 5.2.3.3 Probabilities of Observation 5.3 Rcpcating Simulations 132 5.4 Why simulate ...133 5.4.1 Undcrstanding thc Modcl; Montc Carlo ,,,133 5.4.2 Checking the Mode 134 ensitivity analyS 5.4.2.1“ Explorato Analysis of simulation 5.4.3 136 5.5 Further Reading ,,,,139 5.6 Exercises ,,,,,139 6 The Bootstrap 6.1 Stochastic Models, Uncertainty, Sampling Distributions.. 6.2 The Bootstrap Principle. 142 6.2.1 Variances and Standard Errors 6.2.2 Bias Correction 144 6.2.3 Confidence Intervals 144 6.2.3.1 Other Bootstrap Confidence Intervals .,,,,146 6.2.4 Hypothesis Testing ..,149 6.2.4.1 Double bootstrap hypothesis testing 149 6.2.5 Model-Based Bootstrapping Example: Pareto's Law of Wealth Inequality Resampling… ,,,154 6.3.1 Model-Based vs Resampling Bootstraps 156 6.4 Bootstrapping Regression Models ,,,,,156 6.4.2 Re-sampling points: Non-parametric Model ExamPls> 6.4.1 Re-sampling Points: Parametric Model Example 157 ..,159 6.4.3 Re-sampling Residuals: Example. :········· 6.5 Bootstrap with Dependent datal 6.6 Things Bootstrapping Does Poorly 164 6.7 Which Bootstrap When 165 6.8 Further Reading ,,,,,,,,,166 6.9 Exercises 166 17: 09 Monday 30th January, 2017 CONTENTS 7 Weighting and Variance 168 Weighted Least squares ..168 7.2 Hctcroskcdasticity. 170 7. 2.1 Weighted Least Squares as a Solution to heteroskedasticity..174 7.2.2 Some Explanations for Weighted Least Squares. 174 7.2.3 Finding thc variance and Weights 178 3 Conditional Variance Function Estimation 178 7.3.1 Iterative Refinement of Mean and Variance: An Example 180 7.3.2 Real Data Example: Old Heteroskedastic 183 7.4 Re-sampling Residuals with Heteroskedasticity 187 5 Local Linear regression 187 7.5.1 For and Against Locally Linear Regression 189 7.5.2 Lowess 鲁垂 192 7. 6 Further Readin 192 7.7 Exercises 193 plines 194 8.1 Smoothing by Penalizing Curve Flexibility 194 8.1.1 The Meaning of the Splines 195 8. 2 Computational Example: Splines for Stock Returns ,,,,,,196 8.2.1 Confidence Bands for Splines 8.3 Basis Functions and Degrees of Freedom 202 18.3. 1 Basis Functions ,,,,,,,,,,,202 8.4 Splines in multiple dimensions 204 8.5 Smoothing Splines versus Kernel Regression .,......205 8.6 Somc of thc Math Bchind Splines ..205 8. 7 Further Reading 207 8.8 Exercises 208 9 Additive Models 210 9.1 Additive Models · 210 .2 Partial Residuals and Back-fitting . . 9.2.1 Back-fitting for Linear Models 9.2.2 Backfitting Additive Models 9.3 The Curse of dimensionalit .213 9.4 Example: California House Prices Revisited 9.5 Interaction Terms and Expansions 227 9.6 Closing modeling advi 231 19.7 Further Reading 232 9.8 Exercise 10 Testing Regression Specifications 234 10.1 Testing functional forms 10.1.1 Examples of Testing a Parametric Model 10.1.2 Remarks 241 10.2 Why Use Parametric Models At AP 246 17: 09 Monday 30th January, 2017 【实例截图】
【核心代码】