Entropic Vector Optimization and Simulated Entropy PDF Download

Author: Asaad M. Abu Asaad-Sultan
Publisher:
ISBN:
Category :
Languages : en
Pages :

View

Book Description

Author: Shu-Cherng Fang
Publisher: Springer Science & Business Media
ISBN: 1461561310
Category : Business & Economics
Languages : en
Pages : 350

View

Book Description
Entropy optimization is a useful combination of classical engineering theory (entropy) with mathematical optimization. The resulting entropy optimization models have proved their usefulness with successful applications in areas such as image reconstruction, pattern recognition, statistical inference, queuing theory, spectral analysis, statistical mechanics, transportation planning, urban and regional planning, input-output analysis, portfolio investment, information analysis, and linear and nonlinear programming. While entropy optimization has been used in different fields, a good number of applicable solution methods have been loosely constructed without sufficient mathematical treatment. A systematic presentation with proper mathematical treatment of this material is needed by practitioners and researchers alike in all application areas. The purpose of this book is to meet this need. Entropy Optimization and Mathematical Programming offers perspectives that meet the needs of diverse user communities so that the users can apply entropy optimization techniques with complete comfort and ease. With this consideration, the authors focus on the entropy optimization problems in finite dimensional Euclidean space such that only some basic familiarity with optimization is required of the reader.

Author: Yunshu Liu
Publisher:
ISBN:
Category : Electrical engineering
Languages : en
Pages : 198

View

Book Description
Entropy and conditional mutual information are the key quantities information theory provides to measure uncertainty of and independence relations between random variables. While these measures are key to diverse areas such as physics, communication, signal processing, and machine learning, surprisingly there is still much about them that is yet unknown. This thesis explores some of this unknown territory, ranging from tackling fundamental questions involving the interdependence between entropies of different subsets of random variables via the characterization of the region of entropic vectors, to applied questions involving how conditional independences can be harnessed to improve the efficiency of supervised learning in discrete valued datasets. The region of entropic vectors is a convex cone that has been shown to be at the core of many fundamental limits for problems in multiterminal data compression, network coding, and multimedia transmission. This cone has been shown to be non-polyhedral for four or more random variables, however its boundary remains unknown for four or more discrete random variables. We prove that only one form of nonlinear non-shannon inequality is necessary to fully characterize the region for four random variables. We identify this inequality in terms of a function that is the solution to an optimization problem. We also give some symmetry and convexity properties of this function which rely on the structure of the region of entropic vectors and Ingleton inequalities. Methods for specifying probability distributions that are in faces and on the boundary of the convex cone are derived, then utilized to map optimized inner bounds to the unknown part of the entropy region. The first method utilizes tools and algorithms from abstract algebra to efficiently determine those supports for the joint probability mass functions for four or more random variables that can, for some appropriate set of non-zero probabilities, yield entropic vectors in the gap between the best known inner and outer bounds. These supports are utilized, together with numerical optimization over non-zero probabilities, to provide inner bounds to the unknown part of the entropy region. Next, information geometry is utilized to parameterize and study the structure of probability distributions on these supports yielding entropic vectors in the faces of entropy and in the unknown part of the entropy region. In the final section of the thesis, we propose score functions based on entropy and conditional mutual information as components in partition strategies for supervised learning of datasets with discrete valued features. Partitioning the data enables a reduction in the complexity of training and testing on large datasets. We demonstrate that such partition strategies can also be efficient in the sense that when the training and testing datasets are split according to them, and the blocks in the partition are processed separately, the classification performance is comparable to, or better than, the performance when the data are not partitioned at all.

Author: Mehrdad Tamiz
Publisher: Springer Science & Business Media
ISBN: 3642875610
Category : Business & Economics
Languages : en
Pages : 365

View

Book Description
Most real-life problems involve making decisions to optimally achieve a number of criteria while satisfying some hard or soft constraints. In this book several methods for solving such problems are presented by the leading experts in the area. The book also contains a number of very interesting application papers which demonstrate theoretical modelling, analysing and solution of real-life problems.

Author: Christophe Goupil
Publisher: MDPI
ISBN: 3036501142
Category : Science
Languages : en
Pages : 222

View

Book Description
Beyond its identification with the second law of thermodynamics, entropy is a formidable tool for describing systems in their relationship with their environment. This book proposes to go through some of these situations where the formulation of entropy, and more precisely, the production of entropy in out-of-equilibrium processes, makes it possible to forge an approach to the behavior of very different systems. Whether for dimensioning structures; influencing parameter variability; or optimizing power, efficiency, or waste heat reduction, simulations based on entropy production offer a tool that is both compact and reliable. In the case of systems marked by complexity, it appears to be the only way. In that sense, realistic optimization can be carried out, integrating within the same framework both the system and all the constraints and boundary conditions that define it. Simulations based on entropy give the researcher a powerful analytical framework that crosses the disciplines of physics and links them together.

Author: Jagat Narain Kapur
Publisher:
ISBN:
Category : Computers
Languages : en
Pages : 440

View

Book Description
This senior-level textbook on entropy provides a conceptual framework for the study of probabilistic systems with its elucidation of three key concepts - Shannon's information theory, Jaynes' maximum entropy principle and Kullback's minimum cross-entropy principle.

Author:
Publisher:
ISBN:
Category : Dissertations, Academic
Languages : en
Pages : 886

View

Book Description

Author: H.-S. J. Tsao
Publisher:
ISBN: 9781461561323
Category :
Languages : en
Pages : 356

View

Book Description

Author: Reuven Y. Rubinstein
Publisher: John Wiley & Sons
ISBN: 1118210522
Category : Mathematics
Languages : en
Pages : 331

View

Book Description
This accessible new edition explores the major topics in Monte Carlo simulation Simulation and the Monte Carlo Method, Second Edition reflects the latest developments in the field and presents a fully updated and comprehensive account of the major topics that have emerged in Monte Carlo simulation since the publication of the classic First Edition over twenty-five years ago. While maintaining its accessible and intuitive approach, this revised edition features a wealth of up-to-date information that facilitates a deeper understanding of problem solving across a wide array of subject areas, such as engineering, statistics, computer science, mathematics, and the physical and life sciences. The book begins with a modernized introduction that addresses the basic concepts of probability, Markov processes, and convex optimization. Subsequent chapters discuss the dramatic changes that have occurred in the field of the Monte Carlo method, with coverage of many modern topics including: Markov Chain Monte Carlo Variance reduction techniques such as the transform likelihood ratio method and the screening method The score function method for sensitivity analysis The stochastic approximation method and the stochastic counter-part method for Monte Carlo optimization The cross-entropy method to rare events estimation and combinatorial optimization Application of Monte Carlo techniques for counting problems, with an emphasis on the parametric minimum cross-entropy method An extensive range of exercises is provided at the end of each chapter, with more difficult sections and exercises marked accordingly for advanced readers. A generous sampling of applied examples is positioned throughout the book, emphasizing various areas of application, and a detailed appendix presents an introduction to exponential families, a discussion of the computational complexity of stochastic programming problems, and sample MATLAB programs. Requiring only a basic, introductory knowledge of probability and statistics, Simulation and the Monte Carlo Method, Second Edition is an excellent text for upper-undergraduate and beginning graduate courses in simulation and Monte Carlo techniques. The book also serves as a valuable reference for professionals who would like to achieve a more formal understanding of the Monte Carlo method.

Author:
Publisher:
ISBN:
Category : Dissertations, Academic
Languages : en
Pages : 626

View

Book Description
Theses on any subject submitted by the academic libraries in the UK and Ireland.