Acceleration Methods for Monte Carlo Integration in Bayesian Inference PDF Download

Author: John Geweke
Publisher:
ISBN:
Category : Bayesian statistical decision theory
Languages : en
Pages : 32

View

Book Description

Author: John Geweke
Publisher:
ISBN:
Category : Monte Carlo method
Languages : en
Pages : 22

View

Book Description

Author: Johan Dahlin
Publisher: Linköping University Electronic Press
ISBN: 9176857972
Category :
Languages : sv
Pages : 139

View

Book Description
Making decisions and predictions from noisy observations are two important and challenging problems in many areas of society. Some examples of applications are recommendation systems for online shopping and streaming services, connecting genes with certain diseases and modelling climate change. In this thesis, we make use of Bayesian statistics to construct probabilistic models given prior information and historical data, which can be used for decision support and predictions. The main obstacle with this approach is that it often results in mathematical problems lacking analytical solutions. To cope with this, we make use of statistical simulation algorithms known as Monte Carlo methods to approximate the intractable solution. These methods enjoy well-understood statistical properties but are often computational prohibitive to employ. The main contribution of this thesis is the exploration of different strategies for accelerating inference methods based on sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC). That is, strategies for reducing the computational effort while keeping or improving the accuracy. A major part of the thesis is devoted to proposing such strategies for the MCMC method known as the particle Metropolis-Hastings (PMH) algorithm. We investigate two strategies: (i) introducing estimates of the gradient and Hessian of the target to better tailor the algorithm to the problem and (ii) introducing a positive correlation between the point-wise estimates of the target. Furthermore, we propose an algorithm based on the combination of SMC and Gaussian process optimisation, which can provide reasonable estimates of the posterior but with a significant decrease in computational effort compared with PMH. Moreover, we explore the use of sparseness priors for approximate inference in over-parametrised mixed effects models and autoregressive processes. This can potentially be a practical strategy for inference in the big data era. Finally, we propose a general method for increasing the accuracy of the parameter estimates in non-linear state space models by applying a designed input signal. Borde Riksbanken höja eller sänka reporäntan vid sitt nästa möte för att nå inflationsmålet? Vilka gener är förknippade med en viss sjukdom? Hur kan Netflix och Spotify veta vilka filmer och vilken musik som jag vill lyssna på härnäst? Dessa tre problem är exempel på frågor där statistiska modeller kan vara användbara för att ge hjälp och underlag för beslut. Statistiska modeller kombinerar teoretisk kunskap om exempelvis det svenska ekonomiska systemet med historisk data för att ge prognoser av framtida skeenden. Dessa prognoser kan sedan användas för att utvärdera exempelvis vad som skulle hända med inflationen i Sverige om arbetslösheten sjunker eller hur värdet på mitt pensionssparande förändras när Stockholmsbörsen rasar. Tillämpningar som dessa och många andra gör statistiska modeller viktiga för många delar av samhället. Ett sätt att ta fram statistiska modeller bygger på att kontinuerligt uppdatera en modell allteftersom mer information samlas in. Detta angreppssätt kallas för Bayesiansk statistik och är särskilt användbart när man sedan tidigare har bra insikter i modellen eller tillgång till endast lite historisk data för att bygga modellen. En nackdel med Bayesiansk statistik är att de beräkningar som krävs för att uppdatera modellen med den nya informationen ofta är mycket komplicerade. I sådana situationer kan man istället simulera utfallet från miljontals varianter av modellen och sedan jämföra dessa mot de historiska observationerna som finns till hands. Man kan sedan medelvärdesbilda över de varianter som gav bäst resultat för att på så sätt ta fram en slutlig modell. Det kan därför ibland ta dagar eller veckor för att ta fram en modell. Problemet blir särskilt stort när man använder mer avancerade modeller som skulle kunna ge bättre prognoser men som tar för lång tid för att bygga. I denna avhandling använder vi ett antal olika strategier för att underlätta eller förbättra dessa simuleringar. Vi föreslår exempelvis att ta hänsyn till fler insikter om systemet och därmed minska antalet varianter av modellen som behöver undersökas. Vi kan således redan utesluta vissa modeller eftersom vi har en bra uppfattning om ungefär hur en bra modell ska se ut. Vi kan också förändra simuleringen så att den enklare rör sig mellan olika typer av modeller. På detta sätt utforskas rymden av alla möjliga modeller på ett mer effektivt sätt. Vi föreslår ett antal olika kombinationer och förändringar av befintliga metoder för att snabba upp anpassningen av modellen till observationerna. Vi visar att beräkningstiden i vissa fall kan minska ifrån några dagar till någon timme. Förhoppningsvis kommer detta i framtiden leda till att man i praktiken kan använda mer avancerade modeller som i sin tur resulterar i bättre prognoser och beslut.

Author: Ronald W. Shonkwiler
Publisher: Springer Nature
ISBN: 3031559649
Category :
Languages : en
Pages : 290

View

Book Description

Author: Christian Robert
Publisher: Springer Science & Business Media
ISBN: 1441915753
Category : Computers
Languages : en
Pages : 297

View

Book Description
This book covers the main tools used in statistical simulation from a programmer’s point of view, explaining the R implementation of each simulation technique and providing the output for better understanding and comparison.

Author: Christian Andersson Naesseth
Publisher: Linköping University Electronic Press
ISBN: 9176851613
Category :
Languages : en
Pages : 39

View

Book Description
Automatic decision making and pattern recognition under uncertainty are difficult tasks that are ubiquitous in our everyday life. The systems we design, and technology we develop, requires us to coherently represent and work with uncertainty in data. Probabilistic models and probabilistic inference gives us a powerful framework for solving this problem. Using this framework, while enticing, results in difficult-to-compute integrals and probabilities when conditioning on the observed data. This means we have a need for approximate inference, methods that solves the problem approximately using a systematic approach. In this thesis we develop new methods for efficient approximate inference in probabilistic models. There are generally two approaches to approximate inference, variational methods and Monte Carlo methods. In Monte Carlo methods we use a large number of random samples to approximate the integral of interest. With variational methods, on the other hand, we turn the integration problem into that of an optimization problem. We develop algorithms of both types and bridge the gap between them. First, we present a self-contained tutorial to the popular sequential Monte Carlo (SMC) class of methods. Next, we propose new algorithms and applications based on SMC for approximate inference in probabilistic graphical models. We derive nested sequential Monte Carlo, a new algorithm particularly well suited for inference in a large class of high-dimensional probabilistic models. Then, inspired by similar ideas we derive interacting particle Markov chain Monte Carlo to make use of parallelization to speed up approximate inference for universal probabilistic programming languages. After that, we show how we can make use of the rejection sampling process when generating gamma distributed random variables to speed up variational inference. Finally, we bridge the gap between SMC and variational methods by developing variational sequential Monte Carlo, a new flexible family of variational approximations.

Author: Dani Gamerman
Publisher: CRC Press
ISBN: 9781584885870
Category : Mathematics
Languages : en
Pages : 352

View

Book Description
While there have been few theoretical contributions on the Markov Chain Monte Carlo (MCMC) methods in the past decade, current understanding and application of MCMC to the solution of inference problems has increased by leaps and bounds. Incorporating changes in theory and highlighting new applications, Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Second Edition presents a concise, accessible, and comprehensive introduction to the methods of this valuable simulation technique. The second edition includes access to an internet site that provides the code, written in R and WinBUGS, used in many of the previously existing and new examples and exercises. More importantly, the self-explanatory nature of the codes will enable modification of the inputs to the codes and variation on many directions will be available for further exploration. Major changes from the previous edition: · More examples with discussion of computational details in chapters on Gibbs sampling and Metropolis-Hastings algorithms · Recent developments in MCMC, including reversible jump, slice sampling, bridge sampling, path sampling, multiple-try, and delayed rejection · Discussion of computation using both R and WinBUGS · Additional exercises and selected solutions within the text, with all data sets and software available for download from the Web · Sections on spatial models and model adequacy The self-contained text units make MCMC accessible to scientists in other disciplines as well as statisticians. The book will appeal to everyone working with MCMC techniques, especially research and graduate statisticians and biostatisticians, and scientists handling data and formulating models. The book has been substantially reinforced as a first reading of material on MCMC and, consequently, as a textbook for modern Bayesian computation and Bayesian inference courses.

Author: Nancy Flournoy
Publisher: American Mathematical Soc.
ISBN: 0821851225
Category : Mathematics
Languages : en
Pages : 290

View

Book Description
High dimensional integration arises naturally in two major sub-fields of statistics: multivariate and Bayesian statistics. Indeed, the most common measures of central tendency, variation, and loss are defined by integrals over the sample space, the parameter space, or both. Recent advances in computational power have stimulated significant new advances in both Bayesian and classical multivariate statistics. In many statistical problems, however, multiple integration can be the major obstacle to solutions. This volume contains the proceedings of an AMS-IMS-SIAM Joint Summer Research Conference on Statistical Multiple Integration, held in June 1989 at Humboldt State University in Arcata, California. The conference represents an attempt to bring together mathematicians, statisticians, and computational scientists to focus on the many important problems in statistical multiple integration. The papers document the state of the art in this area with respect to problems in statistics, potential advances blocked by problems with multiple integration, and current work directed at expanding the capability to integrate over high dimensional surfaces.

Author: Dani Gamerman
Publisher: CRC Press
ISBN: 9780412818202
Category : Mathematics
Languages : en
Pages : 264

View

Book Description
Bridging the gap between research and application, Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference provides a concise, and integrated account of Markov chain Monte Carlo (MCMC) for performing Bayesian inference. This volume, which was developed from a short course taught by the author at a meeting of Brazilian statisticians and probabilists, retains the didactic character of the original course text. The self-contained text units make MCMC accessible to scientists in other disciplines as well as statisticians. It describes each component of the theory in detail and outlines related software, which is of particular benefit to applied scientists.

Author: Neal Noah Madras
Publisher: American Mathematical Soc.
ISBN: 0821819925
Category : Mathematics
Languages : en
Pages : 238

View

Book Description
This volume contains the proceedings of the Workshop on Monte Carlo Methods held at The Fields Institute for Research in Mathematical Sciences (Toronto, 1998). The workshop brought together researchers in physics, statistics, and probability. The papers in this volume - of the invited speakers and contributors to the poster session - represent the interdisciplinary emphasis of the conference. Monte Carlo methods have been used intensively in many branches of scientific inquiry. Markov chain methods have been at the forefront of much of this work, serving as the basis of many numerical studies in statistical physics and related areas since the Metropolis algorithm was introduced in 1953. Statisticians and theoretical computer scientists have used these methods in recent years, working on different fundamental research questions, yet using similar Monte Carlo methodology. This volume focuses on Monte Carlo methods that appear to have wide applicability and emphasizes new methods, practical applications and theoretical analysis. It will be of interest to researchers and graduate students who study and/or use Monte Carlo methods in areas of probability, statistics, theoretical physics, or computer science.