实例介绍
【实例简介】
解决任何机器学习问题，本质上都是在解一个限制条件下的优化问题。这是我最近看到的一本关于 "Optimization for Machine Learning" 的书， 非常清晰和系统。
Neural Information processing series hael I. Jordan and Thomas Dietterich. editors Advances in Large margin classifiers, Alexander J Smola, Peter L. Bartlett Bernhard Scholkopf, and Dale Schuurmans, eds, 2000 Advanced Mean Field Methods: Theory and Practice, Manfred Opper and David saad. eds.. 2001 P.N. Rao, Bruno A Olshausen, and Michael S. Lewicki, eds,, 2002 Rajesh Probabilistic Models of the Brain: Perception and Neural Function, Rajesh Exploratory Analysis and Data Modeling in Functional Neuroimaging Friedrich T. Sommer and Andrzej Wichert, eds, 2003 Advances in Minimum Description Length Theory and Applications, Peter D. Grunwald, In Jae Myung, and Mark A. Pitt, eds, 2005 Nearest-Neighbor Methods in Learning and Vision: Theory and Practice, Gregory Shakhnarovich, Piotr Indyk, and Trevor Darrell, eds, 2006 New Directions in Statistical Signal Processing: From Systems to Brains, Si mon Haykin, Jose C. Principe. Terrence J Sejnowski, and John McWhirter ds..2007 Predicting Structured Dala, Gokhan BakIr, Thomas Hofmann, Bernhard Scholkopf, Alexander J. Smola, Ben Taskar, and S. V.N. Vishwanathan eds.,2007 Toward brain-Computer Interfacing, Guido Dornhege, Jose del r. millan Thilo Hinterberger, Dennis J. McFarland, and Klaus-Robert Miller, eds 2007 Large-Scale Kernel Machines, Leon bottou, Olivier Chapelle, Denis De d a ds..2007 Learning Machine Translation. Cyril Goutte, Nicola Cancedda, Marc Letman, and George Foster, eds, 2009 Dataset Shift in Machine Learning, Joaquin Quinonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence, eds, 2009 Optimization for Machine Learning, Suvrit Sra, Sebastian Nowozin, and Stephen Wright, eds, 2012 Optimization for Machine Learning Edited by Suvrit Sra, Sebastian Nowozin, and Stephen J. Wright The mit press Cambridge Massachusetts London, england C2012 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval)without permission in writing from the publisher For information about special quantity discounts, please email special sales(amitpress. mit. edu This book was set in late by the authors and editors Printed and bound in the United States of america Library of Congress Cataloging-in-Publication Data Optimization for machine learning/edited by Suvrit Sra, Sebastian Nowozin, and Stephen J. Wright p c (Neural information processing series) Includes bibliographical references IsBN978-0-262-01646-9(hardcover: alk. paper) 1. Machine learning- Mathematical models. 2. Mathematical optimization. I Sra, Suvrit, 1976-II Nowozin, Sebastian, 1980-IIL. Wright, Stephen J., 1960 Q325.50652012 006.3'1c22 2011002059 1098765432 Contents Series foreword P reface 1 Introduction: Optimization and Machine Learning S. Sra, S. Nowozin, and S.d. Wright 1.1 Support Vector Machines 1.2 Regularized Optimization 7 1.3 Summary of the Chapters 1.4 References 15 2 Convex Optimization with Sparsity-Inducing norms F. Bach.R. Jenatton.. Mairal. and g. Obozinski 19 2.1I 2.2 Generic Methods 26 2.3 Proximal methods 2.4(Block) Coordinate Descent Algorithms 32 2.5 Reweighted-e, algorithms 2.6 Working-Set Methods 36 2.7 Quantitative Evaluation 2. 8 Extensions 47 2.9 Conclusion 48 2.10 References 3 Interior-Point Methods for Large-Scale Cone Programming M. Andersen, Dahl, Z Liu, and L. Vandenberghe 55 3.1 Introduction 6 3.2 Primal-Dual Interior-Point methods 3.3 Linear and Quadratic Programming 64 3.1 Second-Order Cone Programming 3.5 Semidefinite Programming 3.6 Conclusion 3.7 References 7 4 Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey D. P. Bertsekas 85 4.1 Introduct 86 4.2 Incremental Subgradient-Proximal Methods 4.3 Convergence for Methods with Cyclic Order 98 102 4.4 Convergence for Methods with Randomized Order 108 4.5 Some Applications 111 4.6 Conclusions 114 4.7 References 115 5 First-Order Methods for Nonsmooth Convex Large-Scale Optimization, I: General Purpose methods A. Judilsky und A. Nenirouski 121 5.1 Introduction 121 5. 2 Mirror Descent Algorithm: Minimizing over a Simple Set 126 5.3 Problems with Functional Constraints 130 5.4 Minimizing Strongly Convex Functions 131 5.5 Mirror Descent Stochastic Approximation 134 5.6 Mirror descent for Convex-Concave Saddle-Point problems, 135 5.7 Sctting up a Mirror Descent Mcthod 139 5.8 Notes and remarks ..145 5.9 References 146 6 First-Order Methods for Nonsmooth Convex Large-Scale Optimization, Il: Utilizing Problem's Structure A./ and. A. Nemirovski 149 6.1 Introduction 149 6.2 Saddle-Point reformulations of convex Minimization problems 151 6.3 Mirror-Prox Algorithm 154 6.4 Accelerating the Mirror-Prox Algorithm 160 6.5 Accelerating first -Order Methods by randomization 171 6.6 Notes and remarks 6.7 References 181 7 Cutting-Plane Methods in Machine Learning V. Franc. S. Sonnen ndT. werne 185 7. 1 Introduction to Cutting-plane Methods 187 7.2 Regularized Risk Minimization 191 7.3 Multiple Kernel Learning 197 7. 4 MAP Inference in Graphical Models 203 7.5 Rcfcrcnccs 214 8 Introduction to Dual Decomposition for Inference D. Sontag. A. Globerson. and T. Jaakkola 219 8. 1 Introduction 220 8.2 Motivating Applications 8.3 Dual Decomposition and Lagrangian Relaxation 224 8. 4 Subgradient Algorithm 229 8.5 Block Coordinate Descent Algorithms 232 8. 6 Relations to linear progra mming relaxations 8.7 Decoding: Finding the MAP Assignment 8.8 Discussion 8.10 References 252 9 Augmented Lagrangian Methods for Learning, Selecting, and Combining Features R. Tomioka, T. Suzuki, and M. Sugiyama 255 9. 1 Introducti 256 9.2 Background 9.3 Proximal Minimization Algorithm 258 63 9.4 Dual Augmented Lagrangian(DAL) Algorithm 265 9.5 Connections 272 9.6 Application 276 9. 7 Summary 9.9 References 10 The Convex Optimization Approach to Regret Minimization E. Hazan 287 10.1 Introducti 87 10.2 The RFTL Algorithm and Its Analysis 291 10.3The“ Prima-Dua” Approach 294 10.1 Convexity of Loss Functions 298 10.5 Recent Applications 300 10.6 References 302 11 Projected Newton-type Methods in Machine Learning M. Schmidt. D. Kim. and s. sra 305 11.1 Introduction 11.2 Projected Newton-type Methods 306 11.3 Two-Metric Projection Methods 312 11. 4 Inexact Projection methods 316 11.5 Toward Nonsmooth Objcctivcs 320 11.6 Summary and discussion 326 11.7 References 327 12 Interior-Point Methods in Machine Learning Gondxio 331 12.1 Introduction 31 12.2 Interior-Point methods: Background 333 12.3 Polynomial Complexity Result 337 12.4 Interior-Point Methods for Machine Learning 38 12.5 Accelerating Interior-Point Methods 344 12. 6 Conclusions 347 12.7 References 347 13 The Tradeoffs of Large-Scale Learning L. Bottou and o. bousquet 351 13.1 Introduction ...351 13.2 Approximate Optimization 352 13.3 Asymptotic Analysis 355 13. 4 Experiments 363 13.5 Conclusion 366 13.6 Rcfcrcnccs 367 14 Robust Optimization in Machine Learning C. Caramanis s Mannor and h. Xu 369 14.1 Introduction 370 11.2 Background on Robust Optimization 371 4.3 Robust, Optimization and Adversary resistant Learning 373 14.4 Robust Optimization and regularization 377 11.5 Robustness and Consistency 390 14.6 Robustness and generalization 394 14.7 Conclusion 399 11.8 Reference 399 15 Improving First and Second-Order Methods by Modeling Uncertaint N. Le roux, Y. Bengio, and A. Fitzgibbon 403 15.1 Introduction 403 15.2 Optinization Versus Learning 404 15.3 Building a model of the gradients 406 15. 4 The Relative Roles of the Covariance and the hessian 409 15.5 A Second-Order Model of the gradients 412 15.6 An Efficient Implementation of Online Consensus Gradicnt TONGA 414 15.7 Experiments 419 15.8 Conclusion 427 15.9 References 429 16 Bandit View on Noisy Optimization -Y. Audibert. S. Bubeck. and R. Munos 431 16.1 Introduction 431 16.2 Concentration Inequalities 433 16.3 Discrete Optimization 434 16.4 Online Optimization 443 16.5 References 452 17 Optimization methods for Sparse Inverse Covariance Selection K. Scheinberg and s. Ma 455 7.1 Introduction 455 17.2 Block Coordinatc Dcsccnt Mcthods 461 17.3 Alternating Linearization Method 469 17.4 Remarks on Numerical Performance 4 17.5 Rcfcrcnccs 476 18 A Pathwise Algorithm for Covariance Selection V. Krishnamurthy, S. D. Ahipasaoglu, and A. d'Aspremont 479 18.1 Introduction 479 18.2 Covariance Selection 181 18.3 Algorithm 482 18.4 Numerical results 487 18.5 Online Covariance Selection 491 18.6 References 494 【实例截图】
【核心代码】
解决任何机器学习问题，本质上都是在解一个限制条件下的优化问题。这是我最近看到的一本关于 "Optimization for Machine Learning" 的书， 非常清晰和系统。
Neural Information processing series hael I. Jordan and Thomas Dietterich. editors Advances in Large margin classifiers, Alexander J Smola, Peter L. Bartlett Bernhard Scholkopf, and Dale Schuurmans, eds, 2000 Advanced Mean Field Methods: Theory and Practice, Manfred Opper and David saad. eds.. 2001 P.N. Rao, Bruno A Olshausen, and Michael S. Lewicki, eds,, 2002 Rajesh Probabilistic Models of the Brain: Perception and Neural Function, Rajesh Exploratory Analysis and Data Modeling in Functional Neuroimaging Friedrich T. Sommer and Andrzej Wichert, eds, 2003 Advances in Minimum Description Length Theory and Applications, Peter D. Grunwald, In Jae Myung, and Mark A. Pitt, eds, 2005 Nearest-Neighbor Methods in Learning and Vision: Theory and Practice, Gregory Shakhnarovich, Piotr Indyk, and Trevor Darrell, eds, 2006 New Directions in Statistical Signal Processing: From Systems to Brains, Si mon Haykin, Jose C. Principe. Terrence J Sejnowski, and John McWhirter ds..2007 Predicting Structured Dala, Gokhan BakIr, Thomas Hofmann, Bernhard Scholkopf, Alexander J. Smola, Ben Taskar, and S. V.N. Vishwanathan eds.,2007 Toward brain-Computer Interfacing, Guido Dornhege, Jose del r. millan Thilo Hinterberger, Dennis J. McFarland, and Klaus-Robert Miller, eds 2007 Large-Scale Kernel Machines, Leon bottou, Olivier Chapelle, Denis De d a ds..2007 Learning Machine Translation. Cyril Goutte, Nicola Cancedda, Marc Letman, and George Foster, eds, 2009 Dataset Shift in Machine Learning, Joaquin Quinonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence, eds, 2009 Optimization for Machine Learning, Suvrit Sra, Sebastian Nowozin, and Stephen Wright, eds, 2012 Optimization for Machine Learning Edited by Suvrit Sra, Sebastian Nowozin, and Stephen J. Wright The mit press Cambridge Massachusetts London, england C2012 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval)without permission in writing from the publisher For information about special quantity discounts, please email special sales(amitpress. mit. edu This book was set in late by the authors and editors Printed and bound in the United States of america Library of Congress Cataloging-in-Publication Data Optimization for machine learning/edited by Suvrit Sra, Sebastian Nowozin, and Stephen J. Wright p c (Neural information processing series) Includes bibliographical references IsBN978-0-262-01646-9(hardcover: alk. paper) 1. Machine learning- Mathematical models. 2. Mathematical optimization. I Sra, Suvrit, 1976-II Nowozin, Sebastian, 1980-IIL. Wright, Stephen J., 1960 Q325.50652012 006.3'1c22 2011002059 1098765432 Contents Series foreword P reface 1 Introduction: Optimization and Machine Learning S. Sra, S. Nowozin, and S.d. Wright 1.1 Support Vector Machines 1.2 Regularized Optimization 7 1.3 Summary of the Chapters 1.4 References 15 2 Convex Optimization with Sparsity-Inducing norms F. Bach.R. Jenatton.. Mairal. and g. Obozinski 19 2.1I 2.2 Generic Methods 26 2.3 Proximal methods 2.4(Block) Coordinate Descent Algorithms 32 2.5 Reweighted-e, algorithms 2.6 Working-Set Methods 36 2.7 Quantitative Evaluation 2. 8 Extensions 47 2.9 Conclusion 48 2.10 References 3 Interior-Point Methods for Large-Scale Cone Programming M. Andersen, Dahl, Z Liu, and L. Vandenberghe 55 3.1 Introduction 6 3.2 Primal-Dual Interior-Point methods 3.3 Linear and Quadratic Programming 64 3.1 Second-Order Cone Programming 3.5 Semidefinite Programming 3.6 Conclusion 3.7 References 7 4 Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey D. P. Bertsekas 85 4.1 Introduct 86 4.2 Incremental Subgradient-Proximal Methods 4.3 Convergence for Methods with Cyclic Order 98 102 4.4 Convergence for Methods with Randomized Order 108 4.5 Some Applications 111 4.6 Conclusions 114 4.7 References 115 5 First-Order Methods for Nonsmooth Convex Large-Scale Optimization, I: General Purpose methods A. Judilsky und A. Nenirouski 121 5.1 Introduction 121 5. 2 Mirror Descent Algorithm: Minimizing over a Simple Set 126 5.3 Problems with Functional Constraints 130 5.4 Minimizing Strongly Convex Functions 131 5.5 Mirror Descent Stochastic Approximation 134 5.6 Mirror descent for Convex-Concave Saddle-Point problems, 135 5.7 Sctting up a Mirror Descent Mcthod 139 5.8 Notes and remarks ..145 5.9 References 146 6 First-Order Methods for Nonsmooth Convex Large-Scale Optimization, Il: Utilizing Problem's Structure A./ and. A. Nemirovski 149 6.1 Introduction 149 6.2 Saddle-Point reformulations of convex Minimization problems 151 6.3 Mirror-Prox Algorithm 154 6.4 Accelerating the Mirror-Prox Algorithm 160 6.5 Accelerating first -Order Methods by randomization 171 6.6 Notes and remarks 6.7 References 181 7 Cutting-Plane Methods in Machine Learning V. Franc. S. Sonnen ndT. werne 185 7. 1 Introduction to Cutting-plane Methods 187 7.2 Regularized Risk Minimization 191 7.3 Multiple Kernel Learning 197 7. 4 MAP Inference in Graphical Models 203 7.5 Rcfcrcnccs 214 8 Introduction to Dual Decomposition for Inference D. Sontag. A. Globerson. and T. Jaakkola 219 8. 1 Introduction 220 8.2 Motivating Applications 8.3 Dual Decomposition and Lagrangian Relaxation 224 8. 4 Subgradient Algorithm 229 8.5 Block Coordinate Descent Algorithms 232 8. 6 Relations to linear progra mming relaxations 8.7 Decoding: Finding the MAP Assignment 8.8 Discussion 8.10 References 252 9 Augmented Lagrangian Methods for Learning, Selecting, and Combining Features R. Tomioka, T. Suzuki, and M. Sugiyama 255 9. 1 Introducti 256 9.2 Background 9.3 Proximal Minimization Algorithm 258 63 9.4 Dual Augmented Lagrangian(DAL) Algorithm 265 9.5 Connections 272 9.6 Application 276 9. 7 Summary 9.9 References 10 The Convex Optimization Approach to Regret Minimization E. Hazan 287 10.1 Introducti 87 10.2 The RFTL Algorithm and Its Analysis 291 10.3The“ Prima-Dua” Approach 294 10.1 Convexity of Loss Functions 298 10.5 Recent Applications 300 10.6 References 302 11 Projected Newton-type Methods in Machine Learning M. Schmidt. D. Kim. and s. sra 305 11.1 Introduction 11.2 Projected Newton-type Methods 306 11.3 Two-Metric Projection Methods 312 11. 4 Inexact Projection methods 316 11.5 Toward Nonsmooth Objcctivcs 320 11.6 Summary and discussion 326 11.7 References 327 12 Interior-Point Methods in Machine Learning Gondxio 331 12.1 Introduction 31 12.2 Interior-Point methods: Background 333 12.3 Polynomial Complexity Result 337 12.4 Interior-Point Methods for Machine Learning 38 12.5 Accelerating Interior-Point Methods 344 12. 6 Conclusions 347 12.7 References 347 13 The Tradeoffs of Large-Scale Learning L. Bottou and o. bousquet 351 13.1 Introduction ...351 13.2 Approximate Optimization 352 13.3 Asymptotic Analysis 355 13. 4 Experiments 363 13.5 Conclusion 366 13.6 Rcfcrcnccs 367 14 Robust Optimization in Machine Learning C. Caramanis s Mannor and h. Xu 369 14.1 Introduction 370 11.2 Background on Robust Optimization 371 4.3 Robust, Optimization and Adversary resistant Learning 373 14.4 Robust Optimization and regularization 377 11.5 Robustness and Consistency 390 14.6 Robustness and generalization 394 14.7 Conclusion 399 11.8 Reference 399 15 Improving First and Second-Order Methods by Modeling Uncertaint N. Le roux, Y. Bengio, and A. Fitzgibbon 403 15.1 Introduction 403 15.2 Optinization Versus Learning 404 15.3 Building a model of the gradients 406 15. 4 The Relative Roles of the Covariance and the hessian 409 15.5 A Second-Order Model of the gradients 412 15.6 An Efficient Implementation of Online Consensus Gradicnt TONGA 414 15.7 Experiments 419 15.8 Conclusion 427 15.9 References 429 16 Bandit View on Noisy Optimization -Y. Audibert. S. Bubeck. and R. Munos 431 16.1 Introduction 431 16.2 Concentration Inequalities 433 16.3 Discrete Optimization 434 16.4 Online Optimization 443 16.5 References 452 17 Optimization methods for Sparse Inverse Covariance Selection K. Scheinberg and s. Ma 455 7.1 Introduction 455 17.2 Block Coordinatc Dcsccnt Mcthods 461 17.3 Alternating Linearization Method 469 17.4 Remarks on Numerical Performance 4 17.5 Rcfcrcnccs 476 18 A Pathwise Algorithm for Covariance Selection V. Krishnamurthy, S. D. Ahipasaoglu, and A. d'Aspremont 479 18.1 Introduction 479 18.2 Covariance Selection 181 18.3 Algorithm 482 18.4 Numerical results 487 18.5 Online Covariance Selection 491 18.6 References 494 【实例截图】
【核心代码】
标签：
好例子网口号：伸出你的我的手 — 分享！
小贴士
感谢您为本站写下的评论，您的评论对其它用户来说具有重要的参考价值，所以请认真填写。
- 类似“顶”、“沙发”之类没有营养的文字，对勤劳贡献的楼主来说是令人沮丧的反馈信息。
- 相信您也不想看到一排文字/表情墙，所以请不要反馈意义不大的重复字符，也请尽量不要纯表情的回复。
- 提问之前请再仔细看一遍楼主的说明，或许是您遗漏了。
- 请勿到处挖坑绊人、招贴广告。既占空间让人厌烦，又没人会搭理，于人于己都无利。
关于好例子网
本站旨在为广大IT学习爱好者提供一个非营利性互相学习交流分享平台。本站所有资源都可以被免费获取学习研究。本站资源来自网友分享，对搜索内容的合法性不具有预见性、识别性、控制性，仅供学习研究，请务必在下载后24小时内给予删除，不得用于其他任何用途，否则后果自负。基于互联网的特殊性，平台无法对用户传输的作品、信息、内容的权属或合法性、安全性、合规性、真实性、科学性、完整权、有效性等进行实质审查；无论平台是否已进行审查，用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。本站所有资源不代表本站的观点或立场，基于网友分享，根据中国法律《信息网络传播权保护条例》第二十二与二十三条之规定，若资源存在侵权或相关问题请联系本站客服人员，点此联系我们。关于更多版权及免责申明参见 版权及免责申明
网友评论
我要评论