Automatic Differentiation of Parallel Programs PDF Download

Author: Paul Dennis Hovland
Publisher:
ISBN:
Category : Coding theory
Languages : en
Pages : 124

View

Book Description
Abstract: "There are many areas of computational science in which it is necessary or desirable to compute derivatives. Automatic differentiation (AD) tools such as ADIFOR and ADIC have proven very useful for developing derivative code for programs written in Fortran and C. However, many scientific applications are written for or ported to parallel platforms to maximize performance. We have developed tools and techniques for applying AD to parallel programs, paying special attention to message- passing parallel programs. We list several potential problems that arise in differentiating parallel programs and present solutions for each of them. Some of the issues concern the correctness of the generated code, whereas others concern performance. While many of the issues have analogues in sequential programs, the solution is often quite different. In addition, some new concerns arise that are unique to parallel programs. We also describe how the tools and techniques developed to enable AD of parallel programs were applied to a variety of applications, ranging from a simple test problem to a parallel molecular dynamics application. The results confirm the need for and efficacy of several techniques. They also verify the prediction that the program generated by AD will generally demonstrate better speedup and scalability than the original program. We conclude with some brief remarks on how AD can be applied to other types of parallel programs and a description of how this work relates to other research in the areas of AD and scientific computing."

Author:
Publisher:
ISBN:
Category :
Languages : en
Pages : 10

View

Book Description
Many applications require the derivatives of functions defined by computer programs. Automatic differentiation (AD) is a means of developing code to compute the derivatives of complicated functions accurately and efficiently, without the difficulties associated with developing correct code by hand. We discuss some of the issues involved in developing automatic differentiation tools for parallel programming environments.

Author: Michael Förster
Publisher: Springer
ISBN: 365807597X
Category : Computers
Languages : en
Pages : 411

View

Book Description
Numerical programs often use parallel programming techniques such as OpenMP to compute the program's output values as efficient as possible. In addition, derivative values of these output values with respect to certain input values play a crucial role. To achieve code that computes not only the output values simultaneously but also the derivative values, this work introduces several source-to-source transformation rules. These rules are based on a technique called algorithmic differentiation. The main focus of this work lies on the important reverse mode of algorithmic differentiation. The inherent data-flow reversal of the reverse mode must be handled properly during the transformation. The first part of the work examines the transformations in a very general way since pragma-based parallel regions occur in many different kinds such as OpenMP, OpenACC, and Intel Phi. The second part describes the transformation rules of the most important OpenMP constructs.

Author: George Corliss
Publisher: Springer Science & Business Media
ISBN: 1461300754
Category : Computers
Languages : en
Pages : 431

View

Book Description
A survey book focusing on the key relationships and synergies between automatic differentiation (AD) tools and other software tools, such as compilers and parallelizers, as well as their applications. The key objective is to survey the field and present the recent developments. In doing so the topics covered shed light on a variety of perspectives. They reflect the mathematical aspects, such as the differentiation of iterative processes, and the analysis of nonsmooth code. They cover the scientific programming aspects, such as the use of adjoints in optimization and the propagation of rounding errors. They also cover "implementation" problems.

Author:
Publisher:
ISBN:
Category :
Languages : en
Pages : 7

View

Book Description
The ADIC and ADIFOR automatic differentiation tools have proven useful for obtaining the derivatives needed in many scientific applications written in Fortran 77 or ANSI C. But many new scientific programs are written for or ported to parallel platforms to achieve maximal performance. The authors provide an overview of the approach to the complex task of applying automatic differentiation techniques to parallel programming environments, especially as applied to a parallel molecular dynamics application written in C++ with PVM message passing.

Author: Christian H. Bischof
Publisher: Springer Science & Business Media
ISBN: 3540689427
Category : Computers
Languages : en
Pages : 366

View

Book Description
The Fifth International Conference on Automatic Differentiation held from August 11 to 15, 2008 in Bonn, Germany, is the most recent one in a series that began in Breckenridge, USA, in 1991 and continued in Santa Fe, USA, in 1996, Nice, France, in 2000 and Chicago, USA, in 2004. The 31 papers included in these proceedings re?ect the state of the art in automatic differentiation (AD) with respect to theory, applications, and tool development. Overall, 53 authors from institutions in 9 countries contributed, demonstrating the worldwide acceptance of AD technology in computational science. Recently it was shown that the problem underlying AD is indeed NP-hard, f- mally proving the inherently challenging nature of this technology. So, most likely, no deterministic “silver bullet” polynomial algorithm can be devised that delivers optimum performance for general codes. In this context, the exploitation of doma- speci?c structural information is a driving issue in advancing practical AD tool and algorithm development. This trend is prominently re?ected in many of the pub- cations in this volume, not only in a better understanding of the interplay of AD and certain mathematical paradigms, but in particular in the use of hierarchical AD approaches that judiciously employ general AD techniques in application-speci?c - gorithmic harnesses. In this context, the understanding of structures such as sparsity of derivatives, or generalizations of this concept like scarcity, plays a critical role, in particular for higher derivative computations.

Author: Matthias S. Müller
Publisher: Springer
ISBN: 3540685553
Category : Computers
Languages : en
Pages : 446

View

Book Description
This book constitutes the thoroughly refereed post-workshop proceedings of the First and the Second International Workshop on OpenMP, IWOMP 2005 and IWOMP 2006, held in Eugene, OR, USA, and in Reims, France, in June 2005 and 2006 respectively. The first part of the book presents 16 revised full papers carefully reviewed and selected from the IWOMP 2005 program and organized in topical sections on performance tools, compiler technology, run-time environment, applications, as well as the OpenMP language and its evaluation. In the second part there are 19 papers of IWOMP 2006, fully revised and grouped thematically in sections on advanced performance tuning aspects of code development applications, and proposed extensions to OpenMP.

Author: Thomas F. Coleman
Publisher: SIAM
ISBN: 1611974356
Category : Science
Languages : en
Pages : 114

View

Book Description
The calculation of partial derivatives is a fundamental need in scientific computing. Automatic differentiation (AD) can be applied straightforwardly to obtain all necessary partial derivatives (usually first and, possibly, second derivatives) regardless of a code?s complexity. However, the space and time efficiency of AD can be dramatically improved?sometimes transforming a problem from intractable to highly feasible?if inherent problem structure is used to apply AD in a judicious manner. Automatic Differentiation in MATLAB using ADMAT with Applications discusses the efficient use of AD to solve real problems, especially multidimensional zero-finding and optimization, in the MATLAB environment. This book is concerned with the determination of the first and second derivatives in the context of solving scientific computing problems with an emphasis on optimization and solutions to nonlinear systems. The authors focus on the application rather than the implementation of AD, solve real nonlinear problems with high performance by exploiting the problem structure in the application of AD, and provide many easy to understand applications, examples, and MATLAB templates.

Author: H. Martin Bücker
Publisher: Springer Science & Business Media
ISBN: 3540284389
Category : Computers
Languages : en
Pages : 370

View

Book Description
Covers the state of the art in automatic differentiation theory and practice. Intended for computational scientists and engineers, this book aims to provide insight into effective strategies for using automatic differentiation for design optimization, sensitivity analysis, and uncertainty quantification.

Author: Christian Bischof
Publisher: IOS Press
ISBN: 158603796X
Category : Computers
Languages : en
Pages : 824

View

Book Description
ParCo2007 marks a quarter of a century of the international conferences on parallel computing that started in Berlin in 1983. The aim of the conference is to give an overview of the developments, applications and future trends in high-performance computing for various platforms.