Loading...
Please wait, while we are loading the content...
Similar Documents
Efficient Optimization Algorithms for Learning Abstract Efficient Optimization Algorithms for Learning
| Content Provider | Semantic Scholar |
|---|---|
| Author | Salakhutdinov, Ruslan |
| Copyright Year | 2003 |
| Abstract | Efficient Optimization Algorithms for Learning Ruslan Salakhutdinov Master’s Degree Graduate Department of Computer Science University of Toronto 2003 Many problems in machine learning and pattern recognition ultimately reduce to the optimization of a scalar valued function. A variety of general techniques exist for optimizing such objective functions. We study the general class of bound optimization algorithms – including Expectation-Maximization, Iterative Scaling, Non-negative Matrix Factorization, Concave-Convex Procedure – and their relationship to direct optimization algorithms such as gradient-based methods for parameter learning. We also provide a theoretical analysis of the convergence properties of bound optimization algorithms and identify analytic conditions under which these optimizers exhibit quasi-Newton behavior, and conditions under which they possess poor, first-order convergence. Motivated by these analyses, we interpret and analyze their convergence properties and provide some recipes for preprocessing input to these algorithms to yield faster convergence behavior. Our presented analysis also allows us to design several algorithms for practical optimization, that possess superior convergence over standard existing methods. |
| File Format | PDF HTM / HTML |
| Alternate Webpage(s) | http://www.cs.nyu.edu/~roweis/papers/Ruslan_msc_thesis.pdf |
| Alternate Webpage(s) | http://www.cs.toronto.edu/~roweis/papers/Ruslan_msc_thesis.pdf |
| Alternate Webpage(s) | http://cs.nyu.edu/~roweis/papers/Ruslan_msc_thesis.pdf |
| Language | English |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
