Provably Efficient Methods for Large-scale Learning PDF Download
Are you looking for read ebook online? Search for your book and save it on your Kindle device, PC, phones or tablets. Download Provably Efficient Methods for Large-scale Learning PDF full book. Access full book title Provably Efficient Methods for Large-scale Learning by Shuo Yang (Ph. D.). Download full books in PDF and EPUB format.
Author: Shuo Yang (Ph. D.) Publisher: ISBN: Category : Languages : en Pages : 0
Book Description
The scale of machine learning problems grows rapidly in recent years and calls for efficient methods. In this dissertation, we propose simple and efficient methods for various large-scale learning problems. We start with a standard supervised learning problem of solving quadratic regression. In Chapter 2, we show that by utilizing the quadratic structure and a novel gradient estimation algorithm, we can solve sparse quadratic regression with sub-quadratic time complexity and near-optimal sample complexity. We then move to online learning problems. In Chapter 3, we identify a weak assumption and theoretically prove that the standard UCB algorithm efficiently learns from inconsistent human preferences with nearly optimal regret; in Chapter 4 we propose an approximate maximum inner product search data structure for adaptive queries and present two efficient algorithms that achieve sublinear time complexity for linear bandits, which is especially desirable for extremely large and slowly changing action sets. In Chapter 5, we study how to efficiently use privileged features with deep learning models. We present an efficient learning algorithm to exploit privileged features that are not available during testing time. We conduct comprehensive empirical evaluations and present rigorous analysis for linear models to build theoretical insights. It provides a general algorithmic paradigm that can be integrated with many other machine learning methods
Author: Shuo Yang (Ph. D.) Publisher: ISBN: Category : Languages : en Pages : 0
Book Description
The scale of machine learning problems grows rapidly in recent years and calls for efficient methods. In this dissertation, we propose simple and efficient methods for various large-scale learning problems. We start with a standard supervised learning problem of solving quadratic regression. In Chapter 2, we show that by utilizing the quadratic structure and a novel gradient estimation algorithm, we can solve sparse quadratic regression with sub-quadratic time complexity and near-optimal sample complexity. We then move to online learning problems. In Chapter 3, we identify a weak assumption and theoretically prove that the standard UCB algorithm efficiently learns from inconsistent human preferences with nearly optimal regret; in Chapter 4 we propose an approximate maximum inner product search data structure for adaptive queries and present two efficient algorithms that achieve sublinear time complexity for linear bandits, which is especially desirable for extremely large and slowly changing action sets. In Chapter 5, we study how to efficiently use privileged features with deep learning models. We present an efficient learning algorithm to exploit privileged features that are not available during testing time. We conduct comprehensive empirical evaluations and present rigorous analysis for linear models to build theoretical insights. It provides a general algorithmic paradigm that can be integrated with many other machine learning methods
Author: Prateek Jain Publisher: ISBN: Category : Languages : en Pages : 410
Book Description
A large number of machine learning algorithms are critically dependent on the underlying distance/metric/similarity function. Learning an appropriate distance function is therefore crucial to the success of many methods. The class of distance functions that can be learned accurately is characterized by the amount and type of supervision available to the particular application. In this thesis, we explore a variety of such distance learning problems using different amounts/types of supervision and provide efficient and scalable algorithms to learn appropriate distance functions for each of these problems. First, we propose a generic regularized framework for Mahalanobis metric learning and prove that for a wide variety of regularization functions, metric learning can be used for efficiently learning a kernel function incorporating the available side-information. Furthermore, we provide a method for fast nearest neighbor search using the learned distance/kernel function. We show that a variety of existing metric learning methods are special cases of our general framework. Hence, our framework also provides a kernelization scheme and fast similarity search scheme for such methods. Second, we consider a variation of our standard metric learning framework where the side-information is incremental, streaming and cannot be stored. For this problem, we provide an efficient online metric learning algorithm that compares favorably to existing methods both theoretically and empirically. Next, we consider a contrasting scenario where the amount of supervision being provided is extremely small compared to the number of training points. For this problem, we consider two different modeling assumptions: 1) data lies on a low-dimensional linear subspace, 2) data lies on a low-dimensional non-linear manifold. The first assumption, in particular, leads to the problem of matrix rank minimization over polyhedral sets, which is a problem of immense interest in numerous fields including optimization, machine learning, computer vision, and control theory. We propose a novel online learning based optimization method for the rank minimization problem and provide provable approximation guarantees for it. The second assumption leads to our geometry-aware metric/kernel learning formulation, where we jointly model the metric/kernel over the data along with the underlying manifold. We provide an efficient alternating minimization algorithm for this problem and demonstrate its wide applicability and effectiveness by applying it to various machine learning tasks such as semi-supervised classification, colored dimensionality reduction, manifold alignment etc. Finally, we consider the task of learning distance functions under no supervision, which we cast as a problem of learning disparate clusterings of the data. To this end, we propose a discriminative approach and a generative model based approach and we provide efficient algorithms with convergence guarantees for both the approaches.
Author: Guanghui Lan Publisher: Springer Nature ISBN: 3030395685 Category : Mathematics Languages : en Pages : 591
Book Description
This book covers not only foundational materials but also the most recent progresses made during the past few years on the area of machine learning algorithms. In spite of the intensive research and development in this area, there does not exist a systematic treatment to introduce the fundamental concepts and recent progresses on machine learning algorithms, especially on those based on stochastic optimization methods, randomized algorithms, nonconvex optimization, distributed and online learning, and projection free methods. This book will benefit the broad audience in the area of machine learning, artificial intelligence and mathematical programming community by presenting these recent developments in a tutorial style, starting from the basic building blocks to the most carefully designed and complicated algorithms for machine learning.
Author: Suvrit Sra Publisher: MIT Press ISBN: 026201646X Category : Computers Languages : en Pages : 509
Book Description
An up-to-date account of the interplay between optimization and machine learning, accessible to students and researchers in both communities. The interplay between optimization and machine learning is one of the most important developments in modern computational science. Optimization formulations and methods are proving to be vital in designing algorithms to extract essential knowledge from huge volumes of data. Machine learning, however, is not simply a consumer of optimization technology but a rapidly evolving field that is itself generating new optimization ideas. This book captures the state of the art of the interaction between optimization and machine learning in a way that is accessible to researchers in both fields. Optimization approaches have enjoyed prominence in machine learning because of their wide applicability and attractive theoretical properties. The increasing complexity, size, and variety of today's machine learning models call for the reassessment of existing assumptions. This book starts the process of reassessment. It describes the resurgence in novel contexts of established frameworks such as first-order methods, stochastic approximations, convex relaxations, interior-point methods, and proximal methods. It also devotes attention to newer themes such as regularized optimization, robust optimization, gradient and subgradient methods, splitting techniques, and second-order methods. Many of these techniques draw inspiration from other fields, including operations research, theoretical computer science, and subfields of optimization. The book will enrich the ongoing cross-fertilization between the machine learning community and these other fields, and within the broader optimization community.
Author: Shai Shalev-Shwartz Publisher: Cambridge University Press ISBN: 1107057132 Category : Computers Languages : en Pages : 415
Book Description
Introduces machine learning and its algorithmic paradigms, explaining the principles behind automated learning approaches and the considerations underlying their usage.
Author: Johannes Fürnkranz Publisher: Springer Science & Business Media ISBN: 354045375X Category : Computers Languages : en Pages : 873
Book Description
This book constitutes the refereed proceedings of the 17th European Conference on Machine Learning, ECML 2006, held, jointly with PKDD 2006. The book presents 46 revised full papers and 36 revised short papers together with abstracts of 5 invited talks, carefully reviewed and selected from 564 papers submitted. The papers present a wealth of new results in the area and address all current issues in machine learning.
Author: Sirakov Boyan Publisher: World Scientific ISBN: 9813272899 Category : Mathematics Languages : en Pages : 5396
Book Description
The Proceedings of the ICM publishes the talks, by invited speakers, at the conference organized by the International Mathematical Union every 4 years. It covers several areas of Mathematics and it includes the Fields Medal and Nevanlinna, Gauss and Leelavati Prizes and the Chern Medal laudatios.
Author: Longbo Huang Publisher: Springer Nature ISBN: 3031315979 Category : Technology & Engineering Languages : en Pages : 80
Book Description
This book introduces the Learning-Augmented Network Optimization (LANO) paradigm, which interconnects network optimization with the emerging AI theory and algorithms and has been receiving a growing attention in network research. The authors present the topic based on a general stochastic network optimization model, and review several important theoretical tools that are widely adopted in network research, including convex optimization, the drift method, and mean-field analysis. The book then covers several popular learning-based methods, i.e., learning-augmented drift, multi-armed bandit and reinforcement learning, along with applications in networks where the techniques have been successfully applied. The authors also provide a discussion on potential future directions and challenges.
Author: Shuo Yang (Ph. D.)
Author: Shuo Yang (Ph. D.)
Author: Prateek Jain
Author: Guanghui Lan
Author: Yichuan Zhao
Author: Suvrit Sra
Author: Shai Shalev-Shwartz
Author: Johannes Fürnkranz
Author: Sirakov Boyan
Author: Longbo Huang
Author: Jure Leskovec