Natural Language Processing with Transformers, Revised Edition PDF Download

Author: Lewis Tunstall
Publisher: "O'Reilly Media, Inc."
ISBN: 1098136764
Category : Computers
Languages : en
Pages : 409

View

Book Description
Since their introduction in 2017, transformers have quickly become the dominant architecture for achieving state-of-the-art results on a variety of natural language processing tasks. If you're a data scientist or coder, this practical book -now revised in full color- shows you how to train and scale these large models using Hugging Face Transformers, a Python-based deep learning library. Transformers have been used to write realistic news stories, improve Google Search queries, and even create chatbots that tell corny jokes. In this guide, authors Lewis Tunstall, Leandro von Werra, and Thomas Wolf, among the creators of Hugging Face Transformers, use a hands-on approach to teach you how transformers work and how to integrate them in your applications. You'll quickly learn a variety of tasks they can help you solve. Build, debug, and optimize transformer models for core NLP tasks, such as text classification, named entity recognition, and question answering Learn how transformers can be used for cross-lingual transfer learning Apply transformers in real-world scenarios where labeled data is scarce Make transformer models efficient for deployment using techniques such as distillation, pruning, and quantization Train transformers from scratch and learn how to scale to multiple GPUs and distributed environments

Author: Denis Rothman
Publisher: Packt Publishing Ltd
ISBN: 1805123742
Category : Computers
Languages : en
Pages : 731

View

Book Description
The definitive guide to LLMs, from architectures, pretraining, and fine-tuning to Retrieval Augmented Generation (RAG), multimodal Generative AI, risks, and implementations with ChatGPT Plus with GPT-4, Hugging Face, and Vertex AI Key Features Compare and contrast 20+ models (including GPT-4, BERT, and Llama 2) and multiple platforms and libraries to find the right solution for your project Apply RAG with LLMs using customized texts and embeddings Mitigate LLM risks, such as hallucinations, using moderation models and knowledge bases Purchase of the print or Kindle book includes a free eBook in PDF format Book DescriptionTransformers for Natural Language Processing and Computer Vision, Third Edition, explores Large Language Model (LLM) architectures, applications, and various platforms (Hugging Face, OpenAI, and Google Vertex AI) used for Natural Language Processing (NLP) and Computer Vision (CV). The book guides you through different transformer architectures to the latest Foundation Models and Generative AI. You’ll pretrain and fine-tune LLMs and work through different use cases, from summarization to implementing question-answering systems with embedding-based search techniques. You will also learn the risks of LLMs, from hallucinations and memorization to privacy, and how to mitigate such risks using moderation models with rule and knowledge bases. You’ll implement Retrieval Augmented Generation (RAG) with LLMs to improve the accuracy of your models and gain greater control over LLM outputs. Dive into generative vision transformers and multimodal model architectures and build applications, such as image and video-to-text classifiers. Go further by combining different models and platforms and learning about AI agent replication. This book provides you with an understanding of transformer architectures, pretraining, fine-tuning, LLM use cases, and best practices.What you will learn Breakdown and understand the architectures of the Original Transformer, BERT, GPT models, T5, PaLM, ViT, CLIP, and DALL-E Fine-tune BERT, GPT, and PaLM 2 models Learn about different tokenizers and the best practices for preprocessing language data Pretrain a RoBERTa model from scratch Implement retrieval augmented generation and rules bases to mitigate hallucinations Visualize transformer model activity for deeper insights using BertViz, LIME, and SHAP Go in-depth into vision transformers with CLIP, DALL-E 2, DALL-E 3, and GPT-4V Who this book is for This book is ideal for NLP and CV engineers, software developers, data scientists, machine learning engineers, and technical leaders looking to advance their LLMs and generative AI skills or explore the latest trends in the field. Knowledge of Python and machine learning concepts is required to fully understand the use cases and code examples. However, with examples using LLM user interfaces, prompt engineering, and no-code model building, this book is great for anyone curious about the AI revolution.

Author: Savaş Yıldırım
Publisher: Packt Publishing Ltd
ISBN: 1837631506
Category : Computers
Languages : en
Pages : 462

View

Book Description
Explore transformer-based language models from BERT to GPT, delving into NLP and computer vision tasks, while tackling challenges effectively Key Features Understand the complexity of deep learning architecture and transformers architecture Create solutions to industrial natural language processing (NLP) and computer vision (CV) problems Explore challenges in the preparation process, such as problem and language-specific dataset transformation Purchase of the print or Kindle book includes a free PDF eBook Book DescriptionTransformer-based language models such as BERT, T5, GPT, DALL-E, and ChatGPT have dominated NLP studies and become a new paradigm. Thanks to their accurate and fast fine-tuning capabilities, transformer-based language models have been able to outperform traditional machine learning-based approaches for many challenging natural language understanding (NLU) problems. Aside from NLP, a fast-growing area in multimodal learning and generative AI has recently been established, showing promising results. Mastering Transformers will help you understand and implement multimodal solutions, including text-to-image. Computer vision solutions that are based on transformers are also explained in the book. You’ll get started by understanding various transformer models before learning how to train different autoregressive language models such as GPT and XLNet. The book will also get you up to speed with boosting model performance, as well as tracking model training using the TensorBoard toolkit. In the later chapters, you’ll focus on using vision transformers to solve computer vision problems. Finally, you’ll discover how to harness the power of transformers to model time series data and for predicting. By the end of this transformers book, you’ll have an understanding of transformer models and how to use them to solve challenges in NLP and CV.What you will learn Focus on solving simple-to-complex NLP problems with Python Discover how to solve classification/regression problems with traditional NLP approaches Train a language model and explore how to fine-tune models to the downstream tasks Understand how to use transformers for generative AI and computer vision tasks Build transformer-based NLP apps with the Python transformers library Focus on language generation such as machine translation and conversational AI in any language Speed up transformer model inference to reduce latency Who this book is for This book is for deep learning researchers, hands-on practitioners, and ML/NLP researchers. Educators, as well as students who have a good command of programming subjects, knowledge in the field of machine learning and artificial intelligence, and who want to develop apps in the field of NLP as well as multimodal tasks will also benefit from this book’s hands-on approach. Knowledge of Python (or any programming language) and machine learning literature, as well as a basic understanding of computer science, are required.

Author: Sebastian Raschka
Publisher: Packt Publishing Ltd
ISBN: 1801816387
Category : Computers
Languages : en
Pages : 775

View

Book Description
This book of the bestselling and widely acclaimed Python Machine Learning series is a comprehensive guide to machine and deep learning using PyTorch s simple to code framework. Purchase of the print or Kindle book includes a free eBook in PDF format. Key Features Learn applied machine learning with a solid foundation in theory Clear, intuitive explanations take you deep into the theory and practice of Python machine learning Fully updated and expanded to cover PyTorch, transformers, XGBoost, graph neural networks, and best practices Book DescriptionMachine Learning with PyTorch and Scikit-Learn is a comprehensive guide to machine learning and deep learning with PyTorch. It acts as both a step-by-step tutorial and a reference you'll keep coming back to as you build your machine learning systems. Packed with clear explanations, visualizations, and examples, the book covers all the essential machine learning techniques in depth. While some books teach you only to follow instructions, with this machine learning book, we teach the principles allowing you to build models and applications for yourself. Why PyTorch? PyTorch is the Pythonic way to learn machine learning, making it easier to learn and simpler to code with. This book explains the essential parts of PyTorch and how to create models using popular libraries, such as PyTorch Lightning and PyTorch Geometric. You will also learn about generative adversarial networks (GANs) for generating new data and training intelligent agents with reinforcement learning. Finally, this new edition is expanded to cover the latest trends in deep learning, including graph neural networks and large-scale transformers used for natural language processing (NLP). This PyTorch book is your companion to machine learning with Python, whether you're a Python developer new to machine learning or want to deepen your knowledge of the latest developments.What you will learn Explore frameworks, models, and techniques for machines to learn from data Use scikit-learn for machine learning and PyTorch for deep learning Train machine learning classifiers on images, text, and more Build and train neural networks, transformers, and boosting algorithms Discover best practices for evaluating and tuning models Predict continuous target outcomes using regression analysis Dig deeper into textual and social media data using sentiment analysis Who this book is for If you have a good grasp of Python basics and want to start learning about machine learning and deep learning, then this is the book for you. This is an essential resource written for developers and data scientists who want to create practical machine learning and deep learning applications using scikit-learn and PyTorch. Before you get started with this book, you’ll need a good understanding of calculus, as well as linear algebra.

Author: Valentina Alto
Publisher: Packt Publishing Ltd
ISBN: 1835462634
Category : Computers
Languages : en
Pages : 343

View

Book Description
Get hands-on with GPT 3.5, GPT 4, LangChain, Llama 2, Falcon LLM and more, to build LLM-powered sophisticated AI applications Key Features Embed LLMs into real-world applications Use LangChain to orchestrate LLMs and their components within applications Grasp basic and advanced techniques of prompt engineering Book DescriptionBuilding LLM Powered Applications delves into the fundamental concepts, cutting-edge technologies, and practical applications that LLMs offer, ultimately paving the way for the emergence of large foundation models (LFMs) that extend the boundaries of AI capabilities. The book begins with an in-depth introduction to LLMs. We then explore various mainstream architectural frameworks, including both proprietary models (GPT 3.5/4) and open-source models (Falcon LLM), and analyze their unique strengths and differences. Moving ahead, with a focus on the Python-based, lightweight framework called LangChain, we guide you through the process of creating intelligent agents capable of retrieving information from unstructured data and engaging with structured data using LLMs and powerful toolkits. Furthermore, the book ventures into the realm of LFMs, which transcend language modeling to encompass various AI tasks and modalities, such as vision and audio. Whether you are a seasoned AI expert or a newcomer to the field, this book is your roadmap to unlock the full potential of LLMs and forge a new era of intelligent machines.What you will learn Explore the core components of LLM architecture, including encoder-decoder blocks and embeddings Understand the unique features of LLMs like GPT-3.5/4, Llama 2, and Falcon LLM Use AI orchestrators like LangChain, with Streamlit for the frontend Get familiar with LLM components such as memory, prompts, and tools Learn how to use non-parametric knowledge and vector databases Understand the implications of LFMs for AI research and industry applications Customize your LLMs with fine tuning Learn about the ethical implications of LLM-powered applications Who this book is for Software engineers and data scientists who want hands-on guidance for applying LLMs to build applications. The book will also appeal to technical leaders, students, and researchers interested in applied LLM topics. We don’t assume previous experience with LLM specifically. But readers should have core ML/software engineering fundamentals to understand and apply the content.

Author: Denis Rothman
Publisher: Packt Publishing Ltd
ISBN: 1800568630
Category : Computers
Languages : en
Pages : 385

View

Book Description
Publisher's Note: A new edition of this book is out now that includes working with GPT-3 and comparing the results with other models. It includes even more use cases, such as casual language analysis and computer vision tasks, as well as an introduction to OpenAI's Codex. Key FeaturesBuild and implement state-of-the-art language models, such as the original Transformer, BERT, T5, and GPT-2, using concepts that outperform classical deep learning modelsGo through hands-on applications in Python using Google Colaboratory Notebooks with nothing to install on a local machineTest transformer models on advanced use casesBook Description The transformer architecture has proved to be revolutionary in outperforming the classical RNN and CNN models in use today. With an apply-as-you-learn approach, Transformers for Natural Language Processing investigates in vast detail the deep learning for machine translations, speech-to-text, text-to-speech, language modeling, question answering, and many more NLP domains with transformers. The book takes you through NLP with Python and examines various eminent models and datasets within the transformer architecture created by pioneers such as Google, Facebook, Microsoft, OpenAI, and Hugging Face. The book trains you in three stages. The first stage introduces you to transformer architectures, starting with the original transformer, before moving on to RoBERTa, BERT, and DistilBERT models. You will discover training methods for smaller transformers that can outperform GPT-3 in some cases. In the second stage, you will apply transformers for Natural Language Understanding (NLU) and Natural Language Generation (NLG). Finally, the third stage will help you grasp advanced language understanding techniques such as optimizing social network datasets and fake news identification. By the end of this NLP book, you will understand transformers from a cognitive science perspective and be proficient in applying pretrained transformer models by tech giants to various datasets. What you will learnUse the latest pretrained transformer modelsGrasp the workings of the original Transformer, GPT-2, BERT, T5, and other transformer modelsCreate language understanding Python programs using concepts that outperform classical deep learning modelsUse a variety of NLP platforms, including Hugging Face, Trax, and AllenNLPApply Python, TensorFlow, and Keras programs to sentiment analysis, text summarization, speech recognition, machine translations, and moreMeasure the productivity of key transformers to define their scope, potential, and limits in productionWho this book is for Since the book does not teach basic programming, you must be familiar with neural networks, Python, PyTorch, and TensorFlow in order to learn their implementation with Transformers. Readers who can benefit the most from this book include experienced deep learning & NLP practitioners and data analysts & data scientists who want to process the increasing amounts of language-driven data.

Author: Savaş Yıldırım
Publisher: Packt Publishing Ltd
ISBN: 1801078890
Category : Computers
Languages : en
Pages : 374

View

Book Description
Take a problem-solving approach to learning all about transformers and get up and running in no time by implementing methodologies that will build the future of NLP Key Features Explore quick prototyping with up-to-date Python libraries to create effective solutions to industrial problems Solve advanced NLP problems such as named-entity recognition, information extraction, language generation, and conversational AI Monitor your model's performance with the help of BertViz, exBERT, and TensorBoard Book DescriptionTransformer-based language models have dominated natural language processing (NLP) studies and have now become a new paradigm. With this book, you'll learn how to build various transformer-based NLP applications using the Python Transformers library. The book gives you an introduction to Transformers by showing you how to write your first hello-world program. You'll then learn how a tokenizer works and how to train your own tokenizer. As you advance, you'll explore the architecture of autoencoding models, such as BERT, and autoregressive models, such as GPT. You'll see how to train and fine-tune models for a variety of natural language understanding (NLU) and natural language generation (NLG) problems, including text classification, token classification, and text representation. This book also helps you to learn efficient models for challenging problems, such as long-context NLP tasks with limited computational capacity. You'll also work with multilingual and cross-lingual problems, optimize models by monitoring their performance, and discover how to deconstruct these models for interpretability and explainability. Finally, you'll be able to deploy your transformer models in a production environment. By the end of this NLP book, you'll have learned how to use Transformers to solve advanced NLP problems using advanced models.What you will learn Explore state-of-the-art NLP solutions with the Transformers library Train a language model in any language with any transformer architecture Fine-tune a pre-trained language model to perform several downstream tasks Select the right framework for the training, evaluation, and production of an end-to-end solution Get hands-on experience in using TensorBoard and Weights & Biases Visualize the internal representation of transformer models for interpretability Who this book is for This book is for deep learning researchers, hands-on NLP practitioners, as well as ML/NLP educators and students who want to start their journey with Transformers. Beginner-level machine learning knowledge and a good command of Python will help you get the best out of this book.

Author: Robert Johnson
Publisher: HiTeX Press
ISBN:
Category : Computers
Languages : en
Pages : 166

View

Book Description
"AI Transformers Unleashed: From BERT to Large Language Models and Generative AI" is an insightful exploration into one of the most transformative advancements in artificial intelligence. This book delves into the intricacies of AI transformer models, providing a comprehensive understanding of their architecture, functionality, and the profound impact they've had on natural language processing and beyond. Through clear explanations and detailed analysis, readers are guided from foundational concepts to the latest innovations, covering key models such as BERT and GPT, and examining their application across various domains. With a keen focus on both technical challenges and ethical considerations, this book also addresses the complexities surrounding bias, privacy, and transparency in AI technologies. It offers a balanced discourse on the potential for misuse and the imperative for responsible deployment. Designed to educate and inform, "AI Transformers Unleashed" serves as an essential resource for students, industry professionals, and anyone eager to grasp the current and future landscape of AI, ensuring readers are well-equipped to navigate the evolving field with confidence and insight.

Author: Hunaidkhan Pathan
Publisher: BPB Publications
ISBN: 9365891043
Category : Computers
Languages : en
Pages : 306

View

Book Description
DESCRIPTION The book is all about the basics of NLP, generative AI, and their specific component LLM. In this book, we have provided conceptual knowledge about different terminologies and concepts of NLP and NLG with practical hands-on. This comprehensive book offers a deep dive into the world of NLP and LLMs. Starting with the fundamentals of Python programming and code editors, the book gradually introduces NLP concepts, including text preprocessing, word embeddings, and transformer architectures. You will explore the architecture and capabilities of popular models like GPT-3 and BERT. The book also covers practical aspects of LLM usage for RAG applications using frameworks like LangChain and Hugging Face and deploying them in real world applications. With a focus on both theoretical knowledge and hands-on experience, this book is ideal for anyone looking to master the art of NLP and LLMs. The book also contains AWS Cloud deployment, which will help readers step into the world of cloud computing. As the book contains both theoretical and practical approaches, it will help the readers to gain confidence in the deployment of LLMs for any use cases, as well as get acquainted with the required generative AI knowledge to crack the interviews. KEY FEATURES ● Covers Python basics, NLP concepts, and terminologies, including LLM and RAG concepts. ● Provides exposure to LangChain, Hugging Face ecosystem, and chatbot creation using custom data. ● Guides on integrating chatbots with real-time applications and deploying them on AWS Cloud. WHAT YOU WILL LEARN ● Basics of Python, which contains Python concepts, installation, and code editors. ● Foundation of NLP and generative AI concepts and different terminologies being used in NLP and generative AI domain. ● LLMs and their importance in the cutting edge of AI. ● Creating chatbots using custom data using open source LLMs without spending a single penny. ● Integration of chatbots with real-world applications like Telegram. WHO THIS BOOK IS FOR This book is ideal for beginners and freshers entering the AI or ML field, as well as those at an intermediate level looking to deepen their understanding of generative AI, LLMs, and cloud deployment. TABLE OF CONTENTS 1. Introduction to Python and Code Editors 2. Installation of Python, Required Packages, and Code Editors 3. Ways to Run Python Scripts 4. Introduction to NLP and its Concepts 5. Introduction to Large Language Models 6. Introduction of LangChain, Usage and Importance 7. Introduction of Hugging Face, its Usage and Importance 8. Creating Chatbots Using Custom Data with LangChain and Hugging Face Hub 9. Hyperparameter Tuning and Fine Tuning Pre-Trained Models 10. Integrating LLMs into Real-World Applications–Case Studies 11. Deploying LLMs in Cloud Environments for Scalability 12. Future Directions: Advances in LLMs and Beyond Appendix A: Useful Tips for Efficient LLM Experimentation Appendix B: Resources and References

Author: Sebastian Raschka
Publisher: Simon and Schuster
ISBN: 1638355738
Category : Computers
Languages : en
Pages : 366

View

Book Description
Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up! In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You’ll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks. Build a Large Language Model (from Scratch) teaches you how to: • Plan and code all the parts of an LLM • Prepare a dataset suitable for LLM training • Fine-tune LLMs for text classification and with your own data • Use human feedback to ensure your LLM follows instructions • Load pretrained weights into an LLM Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you’ll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant. About the technology Physicist Richard P. Feynman reportedly said, “I don’t understand anything I can’t build.” Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning. About the book Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself! What's inside • Plan and code an LLM comparable to GPT-2 • Load pretrained weights • Construct a complete training pipeline • Fine-tune your LLM for text classification • Develop LLMs that follow human instructions About the reader Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs. About the author Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software. The technical editor on this book was David Caswell. Table of Contents 1 Understanding large language models 2 Working with text data 3 Coding attention mechanisms 4 Implementing a GPT model from scratch to generate text 5 Pretraining on unlabeled data 6 Fine-tuning for classification 7 Fine-tuning to follow instructions A Introduction to PyTorch B References and further reading C Exercise solutions D Adding bells and whistles to the training loop E Parameter-efficient fine-tuning with LoRA