Author: Shuang Song (Ph. D.) Publisher: ISBN: Category : Languages : en Pages : 294
Book Description
The amount of data generated every day is growing exponentially in the big data era. A significant portion of this data is stored as graphs in various domains, such as online retail and social networks. Analyzing large-scale graphs provides important insights that are highly utilized in many areas, such as recommendation systems, banking systems, and medical diagnosis. To accommodate analysis on large-scale graphs, developers from industry and academia design the distributed graph processing systems. However, processing graphs in a distributed manner suffers performance inefficiencies caused by workload imbalance and redundant computations. For instance, while data centers are trending towards a large amount of heterogeneous processing machines, graph partitioners still operate under the assumption of uniform computing resources. This leads to load imbalance that degrades the overall performance. Even with a balanced data distribution, the irregularity of graph applications can result in different amounts of dynamic operations on each machine in the cluster. Such imbalanced work distribution slows down the execution speed. Besides these, redundancy also impacts the performance of distributed graph analysis. To utilize the available parallelism of computing clusters, distributed graph systems deploy the repeated-relaxing computation model (e.g., Bellman-Ford algorithm variants) rather than in a sequential but work-optimal order. Studies performed in this dissertation show that redundant computations pervasively exist and significantly impact the performance efficiency of distributed graph processing. This dissertation explores novel techniques to reduce the workload imbalance and redundant computations of analyzing large-scale graphs in a distributed setup. It evaluates proposed techniques on both pre-processing and execution modules to enable fair data distribution, lightweight workload balancing, and redundancy optimization for future distributed graph processing systems. The first contribution of this dissertation is the Heterogeneity-aware Partitioning (HAP) that aims to balance load distribution of distributed graph processing in heterogeneous clusters. HAP proposes a number of methodologies to estimate various machines’ computational power on graph analytics. It also extends several state-of-the-art partitioning algorithms for heterogeneity-aware data distribution. Utilizing the estimation of machines’ graph processing capability to guide extended partitioning algorithms can reduce load imbalance when processing a large-scale graph in heterogeneous clusters. This results in significant performance improvement. Another contribution of the dissertation is the Hula, which optimizes the workload balance of distributed graph analytics on the fly. Hula offers a hybrid graph partitioning algorithm to split a large-scale graph in a locality-friendly manner and generate metadata for lightweight dynamic workload balancing. To track machines’ work intensity, Hula inserts hardware timers to count the time spent on the important operations (e.g., computational operations and atomic operations). This information can guide Hula’s workload scheduler to arrange work migration. With the support of metadata generated by the hybrid partitioner, Hula’s migration scheme only requires a minimal amount of data to transfer work between machines in the cluster. Hula’s workload balancing design achieves a lightweight imbalance reduction on the fly. Finally, this dissertation focuses on improving the computational efficiency of distributed graph processing. To do so, it reveals the root cause and the amount of redundant computations in distributed graph processing. SLFE is proposed as a system solution to reduce these redundant operations. SLFE develops a lightweight pre-processing technique to obtain the maximum propagation order of each vertex in a given graph. This information is defined as Redundancy Reduction Guidance (RRG) and is utilized by SLFE’s Redundancy Reduction (RR)-aware computing model to prune redundant operations on the fly. Moreover, SLFE provides RRaware APIs to maintain high promgrammablity. These techniques allow the redundancy optimizations of distributed graph processing to become transparent to users
Author: Dr P. SrinivasaRao Publisher: Lulu.com ISBN: 1387388762 Category : Education Languages : en Pages : 97
Book Description
A distributed system consists of many heterogeneous processors with different processing power and all processors are interconnected with a communication channel. In such a system, if some processors are less loaded or idle and others are heavily loaded, the system performance will be reduced drastically. System performance can be improved by using proper load balancing [1, 4]. The aim of load balancing is to improve the performance measures and reduce the overall completion time and cost
Author: Hisao Kameda Publisher: Springer Science & Business Media ISBN: 1447109694 Category : Technology & Engineering Languages : en Pages : 262
Book Description
An important consideration in improving the performance of a distributed computer system is the balancing of the load between the host computers. Load balancing may be either static or dynamic; static balancing strategies are generally based on information about the system's average behavior rather than its actual current state, while dynamic strategies react to the current state when making transfer decisions. Although it is often conjectured that dynamic load balancing outperforms static, careful investigation shows that this view is not always valid. Recent research on the problem of optimal static load balancing is clearly and intuitively presented, with coverage of distributed computer system models, problem formulation in load balancing, and effective algorithms for implementing optimization. Providing a thorough understanding of both static and dynamic strategies, this book will be of interest to all researchers and practitioners working to optimize performance in distributed computer systems.
Author: Junhua Fang Publisher: World Scientific ISBN: 9811216169 Category : Computers Languages : en Pages : 259
Book Description
This illustrative compendium analyzes the load balancing problem in distributed stream processing systems and explores a set of high-performance real-time processing scheme based on key-based balancing strategy, join-matrix model and fault tolerance mechanisms.The volume succinctly provides the theoretical support for the proposed techniques. Through a rich set of experiments and comparisons with the other state-of-the-art techniques using both standard benchmarks and real data sets, the book comprehensively verifies the correctness and effectiveness of the proposed methods.This unique title is an excellent reference text for researchers in the fields of distributed stream processing, parallel system, cloud computing, etc.
Author: Srikanta Patnaik Publisher: Springer Nature ISBN: 981991230X Category : Technology & Engineering Languages : en Pages : 297
Book Description
This book presents high-quality research in the field of 3D imaging technology. The fourth edition of International Conference on 3D Imaging Technology (3DDIT-MSP&DL) continues the good traditions already established by the first three editions of the conference to provide a wide scientific forum for researchers, academia, and practitioners to exchange newest ideas and recent achievements in all aspects of image processing and analysis, together with their contemporary applications. The conference proceedings are published in two volumes. The main topics of the papers comprise famous trends as: 3D image representation, 3D image technology, 3D images and graphics, and computing and 3D information technology. In these proceedings, special attention is paid at the 3D tensor image representation, the 3D content generation technologies, big data analysis, and also deep learning, artificial intelligence, the 3D image analysis and video understanding, the 3D virtual and augmented reality, and many related areas. The first volume contains papers in 3D image processing, transforms, and technologies. The second volume is about computing and information technologies, computer images and graphics and related applications. The two volumes of the book cover a wide area of the aspects of the contemporary multidimensional imaging and the related future trends from data acquisition to real-world applications based on various techniques and theoretical approaches.
Author: Garcia-Robledo, Alberto Publisher: IGI Global ISBN: 1522538003 Category : Computers Languages : en Pages : 232
Book Description
Recent years have witnessed the rise of analysis of real-world massive and complex phenomena in graphs; to efficiently solve these large-scale graph problems, it is necessary to exploit high performance computing (HPC), which accelerates the innovation process for discovery and invention of new products and procedures in network science. Creativity in Load-Balance Schemes for Multi/Many-Core Heterogeneous Graph Computing: Emerging Research and Opportunities is a critical scholarly resource that examines trends, challenges, and collaborative processes in emerging fields within complex network analysis. Featuring coverage on a broad range of topics such as high-performance computing, big data, network science, and accelerated network traversal, this book is geared towards data analysts, researchers, students in information communication technology (ICT), program developers, and academics.
Author: Dalibor Klusáček Publisher: Springer Nature ISBN: 3031439430 Category : Computers Languages : en Pages : 200
Book Description
This book constitutes the thoroughly refereed post-conference proceedings of the 26th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2023, held in St. Petersburg, FL, USA, during May 19, 2023. The 8 full papers and one keynote paper included in this book were carefully reviewed and selected from 14 submissions. The volume contains two sections: keynote and technical papers.
Author: Rajesh Vanchipura Publisher: CRC Press ISBN: 1351124129 Category : Mathematics Languages : en Pages : 1064
Book Description
The International Conference on Emerging Trends in Engineering, Science and Technology (ICETEST) was held at the Government Engineering College, Thrissur, Kerala, India, from 18th to 20th January 2018, with the theme, “Society, Energy and Environment”, covering related topics in the areas of Civil Engineering, Mechanical Engineering, Electrical Engineering, Chemical Engineering, Electronics & Communication Engineering, Computer Science and Architecture. Conflict between energy and environment has been of global significance in recent years. Academic research needs to support the industry and society through socially and environmentally sustainable outcomes. ICETEST 2018 was organized with this specific objective. The conference provided a platform for researchers from different domains, to discuss and disseminate their findings. Outstanding speakers, faculties, and scholars from different parts of the world presented their research outcomes in modern technologies using sustainable technologies.
Author: Shuang Song (Ph. D.)
Author: Minyang Han
Author: Dr P. SrinivasaRao
Author: Hisao Kameda
Author: Junhua Fang
Author: Srikanta Patnaik
Author: Garcia-Robledo, Alberto
Author: Joseph Jacob Green
Author: Dalibor Klusáček
Author: Rajesh Vanchipura