高斯过程回归与分类学习的经典书籍Gaussian_Processes_for_Machine_Learning

【实例简介】
高斯过程 回归 分类的经典书籍还有相关的工具包 Gaussian Processes for Machine Learning Carl Edward Rasmussen Christopher K. I. Williams The MIT Press
Adaptive Computation and Machine Learning Thomas dietterich editor Christopher Bishop, David Heckerman, Michael Jordan, and Michael Kearns, Associate Editors Bioinformatics: The Machine Learning Approach, Pierre Baldi and Soren brunak Reinforcement Learning: An Introduction Richard s. Sutton and andrew G. barto Graphical Models for Machine learning and Digital Communication, Brendan J. Frey Learning in graphical Model Michael I. Jordan Causation. Prediction. and search secon d edition, Peter Spirtes, Clark Glymour, and Richard scheines Principles of Data Mining David Hand. Heikki Mannila and Padhraic Smyth Bioinformatics: The Machine Learning Approach, second edition Pierre Baldi and soren brunak Learning Kernel Classifiers: Theory and Algorithms Ralf herbrich Learning with Kernels: Support vector Machines, Regularization, Optimization, and Beyond, Bernhard Scholkopf and Alexander J smola Introduction to Machine Learning Ethem Alpaydin Gaussian Processes for Machine learning, Carl edward rasmussen and Christopher K. I. williams Gaussian processes for Machine learning Carl edward rasmussen Christopher k. I. Williams The mit p Cambridge. Massachusetts London, England C 2006 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means(including photocopying recording or information storage and retrieval) without permission in writing from the publisher MIT Press books may be purchased at special quantity discounts for business or sales promotional use For information, please email special_sales@mitpress. mit. edu or write to Special Sales Department The MIT Press, 55 Hayward Street, Cambridge, MA 02142 Typeset by the authors using IATEX 2E This book printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Rasmussen. Carl Edward Gaussian processes for machine learning/Carl Edward Rasmussen, Christopher K I. Williams (Adaptive computation and machine learning Includes bibliographical references and indexes ISBN0-262-18253-X 1. Gaussian processes--Data processing. 2. Machine learning-Mathematical models Williams, Christopher K. I. II. Title. III Series QA2744R372006 519.223dc22 2005053433 10987654321 The actual science of logic is conversant at present only with things either certain, impossible, or entirely doubtful, none of which(fortunately)we have to reason on. Therefore the true logic for this world is the calculus of probabilitic which takes account of the magnitude of the probability which is, or ought be in a reasonable mans mind James Clerk Maxwell [1850 Contents Series foreword XI Preface Symbols and notation 1 Introduction 1.1 A Pictorial Introduction to Bayesian Modelling 3 1.2 Roadmap 2 Regression 2.1 Weight-space View 2.1.1 The Standard Linear Model 2.1.2 Projections of Inputs into Feature Space 2.2 Function-space View 13 2.3 Varying the Hyperparameters 19 2.4 Decision Theory for Regression 2.5 An Example application 2.6 Smoothing, Weight Functions and equivalent Kernels 24 k 2.7 Incorporating explicit Basis Functions 27 2.7.1 Marginal Likelihood 2.8 History and Related Work 29 2.9 Exercises 3 Classification 33 3.1 Classification Problems .34 3.1.1 Decision Theory for Classification 35 3.2 Linear models for Classification 3.3 Gaussian Process classification 39 3.4 The Laplace Approximation for the Binary GP Classifier 41 3.4.1 Posterior 42 3.4.2 Predictions 44 3.4.3 Implementation 45 3.4.4 Marginal Likelihood k3.5 Multi-class Laplace approximation 48 3.5.1 Implementation 51 3.6 Expectation Propagation 52 3.6.1 Predictions 3.6.2 Marginal Likelihood 57 3.6.3 Implementation 57 3.7 Experiments 60 3.7.1 A Toy Problem 60 3.7.2 One-dimensional Example 3.7.3 Binary Handwritten Digit Classification Example 3.7.4 10-class Handwritten Digit Classification Example 3.8 Discussion 72 Sections marked by an asterisk contain advanced material that may be omitted on a first reading Contents *3.9 Appendix: Moment Derivations 74 3.10 Exercises 75 4 Covariance functions 79 4.1 Preliminaries 79 4.1.1 Mean Square Continuity and Differentiability 81 4.2 Examples of Covariance Functions 81 4.2.1 Stationary Covariance Functions 82 4.2.2 Dot Product Covariance Functions 89 4.2.3 Other Non-stationary Covariance Functions 4.2.4 Making new Kernels from old 94 4.3 Eigenfunction Analysis of Kernels 96 4.3.1 An Analytic Example 97 4.3.2 Numerical Approximation of Eigenfunctions 98 4.4 Kernels for Non-vectorial Inputs 99 4.4.1 String Kernels 4.4.2 Fisher Kernels 101 4.5 Exercises 102 5 Model selection and Adaptation of Hyperparameters 105 5.1 The Model Selection Problem 106 5.2 Bayesian Model Selection 108 5.3Cr alidation 1 5.4 Model Selection for GP Regression 112 5.4.1 Marginal Likelihood 112 5.4.2 Cross-validation 116 5.4.3 Examples and Discussion 118 5.5 Model selection for GP Classification .124 5.5. 1 Derivatives of the Marginal Likelihood for Laplace's approximation 125 5.5.2 Derivatives of the Marginal Likelihood for EP 127 5.5.3 Cross-validation 127 5.5.4 Example 128 5.6 Exercises 128 6 Relationships between GPs and other models 129 6.1 Reproducing Kernel Hilbert Spaces 129 6.2 Regularization 132 6.2. 1 Regularization Defined by Differential Operators 133 6.2.2 Obtaining the Regularized Solution .135 6.2.3 The Relationship of the Regularization View to Gaussian Process Prediction 135 6.3 Spline models .136 6.3.1 A l-d Gaussian Process Spline Construction 138 k 6.4 Support Vector Machines 141 4.1 Support Vector Classification 141 6.4.2 Support Vector Regression .145 k6.5 Least -Squares classification 146 6.5.1 Probabilistic Least-Squares Classification 147 Contents 6.6 Relevance vector machines 149 6.7 Exercises 150 7 Theoretical Perspectives 151 7. 1 The Equivalent Kernel .151 7.1.1 Some Specific Examples of equivalent Kernels 153 k7.2 Asymptotic analysis 155 7.2.1 Consistency 155 7.2.2 Equivalence and Orthogonality 157 k 7.3 Average-Case Learning Curves 159 *7.4 PAC-Bayesian Analysis 161 7.4.1 The PAC Framework 162 7.4.2 PAC-Bayesian Analysis 163 7.4.3 PAC-Bayesian Analysis of GP Classification 164 7.5 Comparison with Other Supervised Learning Methods 165 7.6 Appendix: Learning Curve for the Ornstein-Uhlenbeck Process 168 7. 7 Exercises .169 8 Approximation Methods for Large Datasets 171 8.1 Reduced-rank Approximations of the gram Matrix 2 Greedy Approximation 174 8.3 Approximations for GPR with Fixed Hyperparameters 8.3.1 Subset of Regressors 8.3.2 The Nystrom Method 177 8.3.3 Subset of Datapoints 177 8.3.4 Projected Process Approximation 8.3.5 Bayesian Committee Machine 180 8.3.6 Iterative Solution of Linear Systems 181 8.3.7 Comparison of Approximate gPR methods 8.4 Approximations for GPC with Fixed Hyperparameters k8.5 Approximating the Marginal Likelihood and its Derivatives 8 k8.6 Appendix: Equivalence of SR and gPR using the Nystrom Approximate Kernel 187 8.7 Exercises 187 9 Further issues and Conclusions 189 9.1 Multiple Outputs 190 9.2 Noise Models with Dependencies 190 9.3 Non-Gaussian Likelihoods .191 9. 4 Derivative Observations 191 9.5 Prediction with Uncertain Inputs 192 9.6 Mixtures of Gaussian processes 192 9.7 Global Optimization 193 9.8 Evaluation of Integrals 9.9 Student’ st Proces 194 9.10 Invariances .194 9.11 Latent Variable Models 196 9.12 Conclusions and Future Directions 196 【实例截图】
【核心代码】