Distributed Optimization: Advances in Theories, Methods, and Applications PDF Download

Author: Huaqing Li
Publisher: Springer Nature
ISBN: 9811561095
Category : Technology & Engineering
Languages : en
Pages : 243

View

Book Description
This book offers a valuable reference guide for researchers in distributed optimization and for senior undergraduate and graduate students alike. Focusing on the natures and functions of agents, communication networks and algorithms in the context of distributed optimization for networked control systems, this book introduces readers to the background of distributed optimization; recent developments in distributed algorithms for various types of underlying communication networks; the implementation of computation-efficient and communication-efficient strategies in the execution of distributed algorithms; and the frameworks of convergence analysis and performance evaluation. On this basis, the book then thoroughly studies 1) distributed constrained optimization and the random sleep scheme, from an agent perspective; 2) asynchronous broadcast-based algorithms, event-triggered communication, quantized communication, unbalanced directed networks, and time-varying networks, from a communication network perspective; and 3) accelerated algorithms and stochastic gradient algorithms, from an algorithm perspective. Finally, the applications of distributed optimization in large-scale statistical learning, wireless sensor networks, and for optimal energy management in smart grids are discussed.

Author: Qingguo Lü
Publisher: Springer Nature
ISBN: 9811985596
Category : Computers
Languages : en
Pages : 282

View

Book Description
This book focuses on improving the performance (convergence rate, communication efficiency, computational efficiency, etc.) of algorithms in the context of distributed optimization in networked systems and their successful application to real-world applications (smart grids and online learning). Readers may be particularly interested in the sections on consensus protocols, optimization skills, accelerated mechanisms, event-triggered strategies, variance-reduction communication techniques, etc., in connection with distributed optimization in various networked systems. This book offers a valuable reference guide for researchers in distributed optimization and for senior undergraduate and graduate students alike.

Author: Pontus Giselsson
Publisher: Springer
ISBN: 3319974785
Category : Mathematics
Languages : en
Pages : 412

View

Book Description
This book presents tools and methods for large-scale and distributed optimization. Since many methods in "Big Data" fields rely on solving large-scale optimization problems, often in distributed fashion, this topic has over the last decade emerged to become very important. As well as specific coverage of this active research field, the book serves as a powerful source of information for practitioners as well as theoreticians. Large-Scale and Distributed Optimization is a unique combination of contributions from leading experts in the field, who were speakers at the LCCC Focus Period on Large-Scale and Distributed Optimization, held in Lund, 14th–16th June 2017. A source of information and innovative ideas for current and future research, this book will appeal to researchers, academics, and students who are interested in large-scale optimization.

Author: Stephen Boyd
Publisher: Now Publishers Inc
ISBN: 160198460X
Category : Computers
Languages : en
Pages : 138

View

Book Description
Surveys the theory and history of the alternating direction method of multipliers, and discusses its applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others.

Author: Mingxi Zhu (Researcher in optimization algorithms)
Publisher:
ISBN:
Category :
Languages : en
Pages : 0

View

Book Description
This dissertation focuses on the large-scale optimization algorithm and its application in revenue management. It comprises three chapters. Chapter 1, Managing Randomization in the Multi-Block Alternating Direction Method of Multipliers for Quadratic Optimization, provides theoretical foundations for managing randomization in the multi-block alternating direction method of multipliers (ADMM) method for quadratic optimization. Chapter 2, How a Small Amount of Data Sharing Benefits Distributed Optimization and Learning, presents both the theoretical and practical evidences on sharing a small amount of data could hugely benefit distributed optimization and learning. Chapter 3, Dynamic Exploration and Exploitation: The Case of Online Lending, studies exploration/ exploitation trade-offs, and the value of dynamic extracting information in the context of online lending. The first chapter is a joint work with Kresimir Mihic and Yinyu Ye. The Alternating Direction Method of Multipliers (ADMM) has gained a lot of attention for solving large-scale and objective-separable constrained optimization. However, the two-block variable structure of the ADMM still limits the practical computational efficiency of the method, because one big matrix factorization is needed at least once even for linear and convex quadratic programming. This drawback may be overcome by enforcing a multi-block structure of the decision variables in the original optimization problem. Unfortunately, the multi-block ADMM, with more than two blocks, is not guaranteed to be convergent. On the other hand, two positive developments have been made: first, if in each cyclic loop one randomly permutes the updating order of the multiple blocks, then the method converges in expectation for solving any system of linear equations with any number of blocks. Secondly, such a randomly permuted ADMM also works for equality-constrained convex quadratic programming even when the objective function is not separable. The goal of this paper is twofold. First, we add more randomness into the ADMM by developing a randomly assembled cyclic ADMM (RAC-ADMM) where the decision variables in each block are randomly assembled. We discuss the theoretical properties of RAC-ADMM and show when random assembling helps and when it hurts, and develop a criterion to guarantee that it converges almost surely. Secondly, using the theoretical guidance on RAC-ADMM, we conduct multiple numerical tests on solving both randomly generated and large-scale benchmark quadratic optimization problems, which include continuous, and binary graph-partition and quadratic assignment, and selected machine learning problems. Our numerical tests show that the RAC-ADMM, with a variable-grouping strategy, could significantly improve the computation efficiency on solving most quadratic optimization problems. The second chapter is a joint work with Yinyu Ye. Distributed optimization algorithms have been widely used in machine learning and statistical estimation, especially under the context where multiple decentralized data centers exist and the decision maker is required to perform collaborative learning across those centers. While distributed optimization algorithms have the merits in parallel processing and protecting local data security, they often suffer from slow convergence compared with centralized optimization algorithms. This paper focuses on how small amount of data sharing could benefit distributed optimization and learning for more advanced optimization algorithms. Specifically, we consider how data sharing could benefit distributed multi-block alternating direction method of multipliers (ADMM) and preconditioned conjugate gradient method (PCG) with application in machine learning tasks of linear and logistic regression. These algorithms are commonly known as algorithms between the first and the second order methods, and we show that data share could hugely boost the convergence speed for this class of the algorithms. Theoretically, we prove that a small amount of data share leads to improvements from near-worst to near-optimal convergence rate when applying ADMM and PCG methods to machine learning tasks. A side theory product is the tight upper bound of linear convergence rate for distributed ADMM applied in linear regression. We further propose a meta randomized data-sharing scheme and provide its tailored applications in multi-block ADMM and PCG methods in order to enjoy both the benefit from data-sharing and from the efficiency of distributed computing. From the numerical evidences, we are convinced that our algorithms provide good quality of estimators in both the least square and the logistic regressions within much fewer iterations by only sharing 5% of pre-fixed data, while purely distributed optimization algorithms may take hundreds more times of iterations to converge. We hope that the discovery resulted from this paper would encourage even small amount of data sharing among different regions to combat difficult global learning problems. The third chapter is a joint work with Haim Mendelson. This paper studies exploration and exploitation tradeoffs in the context of online lending. Unlike traditional contexts where the cost of exploration is an opportunity cost of lost revenue or some other implicit cost, in the case of unsecured online lending, the lender effectively gives away money in order to learn about the borrower's ability to repay. In our model, the lender maximizes the expected net present value of the cash flow she receives by dynamically adjusting the loan amounts and the interest (discount) rate as she learns about the borrower's unknown income. The lender has to carefully balance the trade-offs between earning more interest when she lends more and the risk of default, and we provided the optimal dynamic policy for the lender. The optimal policy support the classic "lean experimentation" in certain regime, while challenge such concept in other regime. When the demand elasticity is zero (the discount rate is set exogenously), or the elasticity a decreasing function of the discount rate, the optimal policy is characterized by a large number of small experiments with increasing repayment amounts. When the demand elasticity is constant or when it is an increasing function of the discount rate, we obtain a two-step optimal policy: the lender performs a single experiment and then, if the borrower repays the loan, offers the same loan amount and discount rate in each subsequent period without any further experimentation. This result sheds light in how to take into account the market churn measured by elasticity, in the dynamic experiment design under uncertain environment. We further provide the implications under the optimal policies, including the impact of the income variability, the value of information and the consumer segmentation. Lastly, we extend the methodology to analyze the Buy-Now-Pay-Later business model and provide the policy suggestions.

Author: De-Shuang Huang
Publisher: Springer
ISBN: 3540742050
Category : Computers
Languages : en
Pages : 1377

View

Book Description
This volume, in conjunction with the two volumes CICS 0002 and LNCS 4681, constitutes the refereed proceedings of the Third International Conference on Intelligent Computing held in Qingdao, China, in August 2007. The 139 full papers published here were carefully reviewed and selected from among 2,875 submissions. These papers offer important findings and insights into the field of intelligent computing.

Author: Aboul Ella Hassanien
Publisher: Springer
ISBN: 3319746901
Category : Technology & Engineering
Languages : en
Pages : 717

View

Book Description
This book presents the refereed proceedings of the third International Conference on Advanced Machine Learning Technologies and Applications, AMLTA 2018, held in Cairo, Egypt, on February 22–24, 2018, and organized by the Scientific Research Group in Egypt (SRGE). The papers cover current research in machine learning, big data, Internet of Things, biomedical engineering, fuzzy logic, security, and intelligence swarms and optimization.

Author: Yunmook Nah
Publisher: Springer Nature
ISBN: 3030594130
Category : Computers
Languages : en
Pages : 296

View

Book Description
The LNCS 12115 constitutes the workshop papers which were held also online in conjunction with the 25th International Conference on Database Systems for Advanced Applications in September 2020. The complete conference includes 119 full papers presented together with 19 short papers plus 15 demo papers and 4 industrial papers in this volume were carefully reviewed and selected from a total of 487 submissions. DASFAA 2020 presents this year following five workshops: The 7th International Workshop on Big Data Management and Service (BDMS 2020) The 6th International Symposium on Semantic Computing and Personalization (SeCoP 2020) The 5th Big Data Quality Management (BDQM 2020) The 4th International Workshop on Graph Data Management and Analysis (GDMA 2020) The 1st International Workshop on Artificial Intelligence for Data Engineering (AIDE 2020)

Author: Rohit Anand
Publisher: John Wiley & Sons
ISBN: 1394188080
Category : Computers
Languages : en
Pages : 469

View

Book Description
META-HEURISTIC ALGORITHMS FOR ADVANCED DISTRIBUTED SYSTEMS Discover a collection of meta-heuristic algorithms for distributed systems in different application domains Meta-heuristic techniques are increasingly gaining favor as tools for optimizing distributed systems—generally, to enhance the utility and precision of database searches. Carefully applied, they can increase system effectiveness, streamline operations, and reduce cost. Since many of these techniques are derived from nature, they offer considerable scope for research and development, with the result that this field is growing rapidly. Meta-Heuristic Algorithms for Advanced Distributed Systems offers an overview of these techniques and their applications in various distributed systems. With strategies based on both global and local searching, it covers a wide range of key topics related to meta-heuristic algorithms. Those interested in the latest developments in distributed systems will find this book indispensable. Meta-Heuristic Algorithms for Advanced Distributed Systems readers will also find: Analysis of security issues, distributed system design, stochastic optimization techniques, and more Detailed discussion of meta-heuristic techniques such as the genetic algorithm, particle swam optimization, and many others Applications of optimized distribution systems in healthcare and other key??industries Meta-Heuristic Algorithms for Advanced Distributed Systems is ideal for academics and researchers studying distributed systems, their design, and their applications.

Author: Bin Shi
Publisher: Springer
ISBN: 3030170764
Category : Technology & Engineering
Languages : en
Pages : 133

View

Book Description
This book studies mathematical theories of machine learning. The first part of the book explores the optimality and adaptivity of choosing step sizes of gradient descent for escaping strict saddle points in non-convex optimization problems. In the second part, the authors propose algorithms to find local minima in nonconvex optimization and to obtain global minima in some degree from the Newton Second Law without friction. In the third part, the authors study the problem of subspace clustering with noisy and missing data, which is a problem well-motivated by practical applications data subject to stochastic Gaussian noise and/or incomplete data with uniformly missing entries. In the last part, the authors introduce an novel VAR model with Elastic-Net regularization and its equivalent Bayesian model allowing for both a stable sparsity and a group selection.