Towards Practical Stochastic Computing Architectures for Emerging Applications PDF Download
Are you looking for read ebook online? Search for your book and save it on your Kindle device, PC, phones or tablets. Download Towards Practical Stochastic Computing Architectures for Emerging Applications PDF full book. Access full book title Towards Practical Stochastic Computing Architectures for Emerging Applications by Vincent T. Lee. Download full books in PDF and EPUB format.
Author: Vincent T. Lee Publisher: ISBN: Category : Languages : en Pages : 148
Book Description
The end of Dennard scaling and demands for energy efficient, low power, and high density computing solutions over the past decade has forced exploration of new computing technologies. Stochastic computing is one of these alternative computing technologies which has enjoyed renewed interest and is the primary focus of this dissertation. Stochastic computing is a form of approximate computing which encodes values as probabilistic bitstreams where the ratio of 1s and 0s determines the encoded value. This representation allows stochastic computing to achieve lower operating power, higher computational density, and better error resilience compared to conventional binary-encoded circuits. In its current form, stochastic computing presents a number of challenges before it can become a practical replacement for conventional binary-encoded computing. First, there is little prior work detailing design methodologies to guide effective implementation and integration of stochastic computing into accelerator architectures. Second, the application space where stochastic computing yields compelling gains is far from obvious and has only seen limited exploration. Third, stochastic arithmetic circuits are unintuitive to design because they require careful consideration of correlation and quantization effects. This thesis focuses on new circuit components, applications, architectural considerations, and design techniques to improve the practicality of stochastic computing accelerators. I first propose novel stochastic circuits to improve the accuracy of stochastic computations and augment the range of implementable functions. I then evaluate the viability of stochastic computing with a design space exploration of end-to-end stochastic computing accelerator architectures. In this exploration, I evaluate under what design parameters and conditions stochastic computing accelerators are competitive alternatives to their binary-encoded counterparts. Using these guidelines, I use these results to establish a set of architecture design guidelines to help designers identify when and why they should consider stochastic computing. I then evaluate codesign opportunities and empirically measuring power, area, and energy efficiency for emerging applications. I also propose borrowing techniques from program synthesis such as stochastic synthesis and mixed integer linear programming to automatically synthesize novel stochastic circuits. Finally, I conclude with future directions for further improving the practicality of stochastic computing as well as additional research directions beyond stochastic computing.
Author: Vincent T. Lee Publisher: ISBN: Category : Languages : en Pages : 148
Book Description
The end of Dennard scaling and demands for energy efficient, low power, and high density computing solutions over the past decade has forced exploration of new computing technologies. Stochastic computing is one of these alternative computing technologies which has enjoyed renewed interest and is the primary focus of this dissertation. Stochastic computing is a form of approximate computing which encodes values as probabilistic bitstreams where the ratio of 1s and 0s determines the encoded value. This representation allows stochastic computing to achieve lower operating power, higher computational density, and better error resilience compared to conventional binary-encoded circuits. In its current form, stochastic computing presents a number of challenges before it can become a practical replacement for conventional binary-encoded computing. First, there is little prior work detailing design methodologies to guide effective implementation and integration of stochastic computing into accelerator architectures. Second, the application space where stochastic computing yields compelling gains is far from obvious and has only seen limited exploration. Third, stochastic arithmetic circuits are unintuitive to design because they require careful consideration of correlation and quantization effects. This thesis focuses on new circuit components, applications, architectural considerations, and design techniques to improve the practicality of stochastic computing accelerators. I first propose novel stochastic circuits to improve the accuracy of stochastic computations and augment the range of implementable functions. I then evaluate the viability of stochastic computing with a design space exploration of end-to-end stochastic computing accelerator architectures. In this exploration, I evaluate under what design parameters and conditions stochastic computing accelerators are competitive alternatives to their binary-encoded counterparts. Using these guidelines, I use these results to establish a set of architecture design guidelines to help designers identify when and why they should consider stochastic computing. I then evaluate codesign opportunities and empirically measuring power, area, and energy efficiency for emerging applications. I also propose borrowing techniques from program synthesis such as stochastic synthesis and mixed integer linear programming to automatically synthesize novel stochastic circuits. Finally, I conclude with future directions for further improving the practicality of stochastic computing as well as additional research directions beyond stochastic computing.
Author: Warren J. Gross Publisher: Springer ISBN: 3030037304 Category : Technology & Engineering Languages : en Pages : 215
Book Description
This book covers the history and recent developments of stochastic computing. Stochastic computing (SC) was first introduced in the 1960s for logic circuit design, but its origin can be traced back to von Neumann's work on probabilistic logic. In SC, real numbers are encoded by random binary bit streams, and information is carried on the statistics of the binary streams. SC offers advantages such as hardware simplicity and fault tolerance. Its promise in data processing has been shown in applications including neural computation, decoding of error-correcting codes, image processing, spectral transforms and reliability analysis. There are three main parts to this book. The first part, comprising Chapters 1 and 2, provides a history of the technical developments in stochastic computing and a tutorial overview of the field for both novice and seasoned stochastic computing researchers. In the second part, comprising Chapters 3 to 8, we review both well-established and emerging design approaches for stochastic computing systems, with a focus on accuracy, correlation, sequence generation, and synthesis. The last part, comprising Chapters 9 and 10, provides insights into applications in machine learning and error-control coding.
Author: Warren J. Gross Publisher: ISBN: 9783030037314 Category : Probabilistic automata Languages : en Pages : 215
Book Description
This book covers the history and recent developments of stochastic computing. Stochastic computing (SC) was first introduced in the 1960s for logic circuit design, but its origin can be traced back to von Neumann's work on probabilistic logic. In SC, real numbers are encoded by random binary bit streams, and information is carried on the statistics of the binary streams. SC offers advantages such as hardware simplicity and fault tolerance. Its promise in data processing has been shown in applications including neural computation, decoding of error-correcting codes, image processing, spectral transforms and reliability analysis. There are three main parts to this book. The first part, comprising Chapters 1 and 2, provides a history of the technical developments in stochastic computing and a tutorial overview of the field for both novice and seasoned stochastic computing researchers. In the second part, comprising Chapters 3 to 8, we review both well-established and emerging design approaches for stochastic computing systems, with a focus on accuracy, correlation, sequence generation, and synthesis. The last part, comprising Chapters 9 and 10, provides insights into applications in machine learning and error-control coding.
Author: Hassnaa El-Derhalli Publisher: ISBN: Category : Languages : en Pages : 0
Book Description
Approximate computing (AC) is an emerging computing approach that allows to trade off design energy efficiency with computing accuracy. It targets error resilient applications, such as image processing, where energy consumption is of major concern. Stochastic computing (SC) is an approximate computing paradigm that leads to energy efficient and reduced hardware complexity designs. In this approach, data is represented as probabilities in bit streams format. The main drawback of this computing paradigm is the intrinsic serial processing of bit streams, which negatively impacts the processing time. Nanophotonics technology is characterized by high bandwidth and high signals propagation speed, which has the potential to support the electrical domain in computations to speed up the processing rate. The major issues in optical computing (OC) remain the large size of silicon photonics devices, which impact the design scalability. In this thesis, we propose, for the first time, an optical stochastic computing (OSC) approach, where we aim to design SC architectures using integrated optics. For this purpose, we propose a methodology that has libraries for optical processing and interfaces, e.g., bit stream generator. We design all-optical gates for the computation and develop transmission models for the architectures. The methodology allows for design space exploration of technological and system-level parameters to optimize design performance, i.e., energy efficiency, computing accuracy, and latency, for the targeted application. This exploration leads to multiple design options that satisfy different design requirements for the selected application. The optical processing libraries include designing a polynomial architecture that can execute any arbitrary single input function. We explore the design parameters by implementing a Gamma correction application for image processing. Results show a 4.5x increase in the errors, which leads to 47x energy saving and 16x faster processing speed. We propose a reconfigurable polynomial architecture to adapt design order at run-time. The design allows the execution of high order polynomial functions for better accuracy or multiple low order functions to increase throughput and energy efficiency. Finally, we propose the design of combinational filters. The purpose is to investigate the design of cascaded gates architectures using photonic crystal (PhC) nanocavities. We use this device to design a Sobel edge detection filter for image processing. The resulting architecture shows 0.85nJ/pixel energy consumption and 51.2ns/pixel processing time. The optical interface libraries include designing different architectures of stochastic number generators (SNG) that are either electrical-optical or all-optical to generate the bit streams. We compare these SNGs in terms of computing accuracy and energy efficiency. The results show that all implementations can lead to the same level of computing accuracy. Moreover, using an all-optical SNG to design a fully optical 8-bit adder results in 98% reduction in hardware complexity and 70% energy saving compared to a conventional optical design.
Author: Massimo D'Elia Publisher: Chapman & Hall/CRC ISBN: 9781498796330 Category : Computers Languages : en Pages : 0
Book Description
This book introduces the reader to advanced concepts in stochastic modelling, rooted in an intuitive yet rigorous presentation of the underlying mathematical concepts. The reader will find valuable insights into topics ranging from Social Sciences and Particle Physics to modern-day Computer Science with Machine Learning and AI in focus.
Author: Soroosh Khoram Publisher: ISBN: Category : Languages : en Pages : 0
Book Description
Machine learning systems often need to be able to evaluate their own predictive uncertainty, especially in high-risk areas, to activate safety measures when necessary. For example, a medical diagnosis model needs to inform the doctor if it is uncertain of a diagnosis so the doctor can account for that in their decision making. Alternatively, an autonomous vehicle that is unsure if it is detecting a pedestrian on the road it can transfer control to the driver. Essentially these models need to be able to say "I don't know." Unfortunately, conventional deep learning models are often overconfident in their predictions, even when encountering out-of-distribution inputs. Recent studies have addressed this challenge using Bayesian Neural Networks (BNNs), a type of neural network model that provides a prediction as well as its confidence (or more commonly uncertainty) in that prediction. However, the additional computations for BNNs compared to conventional neural networks can increase the energy cost of inference. Yet, in many edge applications where BNNs' Uncertainty Quantification (UQ) capability is crucial, power and energy can be constrained. Previous works have discussed challenges of deploying deep learning models at the edge through various model compression techniques in DNNs. However, these studies usually do not consider the changes in the loss function when performing quantization, nor do they take the different importances of DNN model parameters to the accuracy into account. We address these issues in this paper by proposing a new method, called adaptive quantization. We further approach these challenges in the context of BNNs by proposing the Bayesian Bitstream Processor (BBP) which uses Bitstream Computing (BC) to achieve low cost, power, and energy. BC substrates perform computations serially and approximately. This allows for much simpler architecture design compared to architectures that perform precise computations using ALUs. They can this way offer smaller energy consumption. Furthermore, they inherently incorporate Random Number Generators (RNG). These RNGs can be further utilized during the random sampling steps of the BNN computations. We first evaluate a basic design of BBP for an audio classification task and use it as a test case. Our results show that our approach can outperform a micro-controller baseline in energy by two orders of magnitude and delay by an order of magnitude, all while operating at lower power. While our results are promising, we identify a key challenge for deploying BNNs on bitstream computing substrates - designing random number generators that are suitable for this hardware. To overcome this challenge, we present a novel bitstream generation architecture that can significantly reduce costs and, more importantly, allow for early decision making. Since bitstream computing substrates spread computations temporally, improving precision over time, early decision making can significantly improve delay. Our results show that the majority of sample data can often be evaluated quickly, allowing for uncertainty quantification at the edge at low costs. In conclusion, our proposed approach of using stochastic bitstream computing substrates for deploying BNNs can significantly reduce power and costs while providing much-needed uncertainty quantification in machine learning applications. The novel bitstream generation architecture we present in this work is a crucial step towards making this approach more practical and efficient for edge devices.
Author: Julian M. Kunkel Publisher: Springer ISBN: 331967630X Category : Computers Languages : en Pages : 754
Book Description
This book constitutes revised selected papers from 10 workshops that were held as the ISC High Performance 2017 conference in Frankfurt, Germany, in June 2017. The 59 papers presented in this volume were carefully reviewed and selected for inclusion in this book. They stem from the following workshops: Workshop on Virtualization in High-Performance Cloud Computing (VHPC) Visualization at Scale: Deployment Case Studies and Experience Reports International Workshop on Performance Portable Programming Models for Accelerators (P^3MA) OpenPOWER for HPC (IWOPH) International Workshop on Data Reduction for Big Scientific Data (DRBSD) International Workshop on Communication Architectures for HPC, Big Data, Deep Learning and Clouds at Extreme Scale Workshop on HPC Computing in a Post Moore's Law World (HCPM) HPC I/O in the Data Center ( HPC-IODC) Workshop on Performance and Scalability of Storage Systems (WOPSSS) IXPUG: Experiences on Intel Knights Landing at the One Year Mark International Workshop on Communication Architectures for HPC, Big Data, Deep Learning and Clouds at Extreme Scale (ExaComm)
Author: Keerthana Pamidimukkala Publisher: ISBN: Category : Languages : en Pages : 53
Book Description
"Stochastic computing as a computing paradigm is currently undergoing revival as the advancements in technology make it applicable especially in the wake of the need for higher computing power for emerging applications. Recent research in stochastic computing exploits the benefits of approximate computing, called Approximate Stochastic Computing (ASC), which further reduces the operational overhead in implementing stochastic circuits. A mathematical model is proposed to analyze the efficiency and error involved in ASC. Using this mathematical model, a new generalized adaptive method improving on ASC is proposed in the current thesis. The proposed method has been discussed with two possible implementation variants - Area efficient and Time efficient. The proposed method has also been implemented in Matlab to compare against ASC and is shown to perform better than previous approaches for error-tolerant applications"--Abstract, page iii.
Author: Paul Hudak Publisher: Springer Science & Business Media ISBN: 3540774416 Category : Computers Languages : en Pages : 342
Book Description
This book, complete with online files and updates, covers a hugely important area of study in computing. It constitutes the refereed proceedings of the 10th International Symposium on Practical Aspects of Declarative Languages, PADL 2008, held in San Francisco, CA, USA, in January 2008. The 20 revised full papers along with the abstract of 1 invited talk were carefully reviewed and selected from 44 submissions. The papers address all current aspects of declarative programming.
Author: Vincent T. Lee
Author: Vincent T. Lee
Author: Warren J. Gross
Author: Warren J. Gross
Author: Hassnaa El-Derhalli
Author: Massimo D'Elia
Author: Soroosh Khoram
Author: Lei Deng
Author: Julian M. Kunkel
Author: Keerthana Pamidimukkala
Author: Paul Hudak