Author: Reuben Harry Cohn-Gordon
Publisher:
ISBN:
Category :
Languages : en
Pages :
Book Description
Grice (1975) puts forward a view of linguistic meaning in which conversational agents enrich the semantic interpretation of linguistic expressions by recourse to pragmatic reasoning about their interlocutors and world knowledge. As a simple example, on hearing my friend tell me that she read some of War and Peace, I reason that, had she read all of it, she would have said as much, and accordingly that she read only part. It turns out that this perspective is well suited to a probabilistic formalization. In these terms, linguistic meaning is fully characterized by a joint probability distribution P(W; U) between states of the world W and linguistic expressions U. The Gricean perspective described above corresponds to a factoring of this enormously complex distribution into a semantics [[u]](w) : U -> (W -> {0, 1}, world knowledge P(W) and a pair of agents which reason about each other on the assumption that both are cooperative and have access to a commonly known semantics. This third component, of back and forth reasoning between agents, originates in work in game-theory (Franke, 2009; Lewis, 1969) and has been formalized in probabilistic terms by a class of models often collectively referred to as the Rational Speech Acts (RSA) framework (Frank and Goodman, 2012). By allowing for the construction of models which explain in precise terms how Gricean pressures like informativity and relevance interact with a semantics, this framework allows us to take an intuitive theory and explore its predictions beyond the limits of intuition. But it should be more than a theoretical tool. To the extent that its characterization of meaning is correct, it should allow for the construction of computational systems capable of reproducing the dynamics of opendomain natural language. For instance, on the assumption that humans produce language pragmatically, one would expect systems which generate natural language to most faithfully reproduce human behavior when aiming to be not only truthful, but also informative to a hypothetical interlocutor. Likewise, systems which interpret language in a human-like way should perform best when they model language as being generated by an informative speaker. Despite this, standard approaches to many natural language processing (NLP) tasks, like image captioning (Farhadi et al., 2010; Vinyals et al., 2015), translation (Brown et al., 1990; Bahdanau et al., 2014) and metaphor interpretation (Shutova et al., 2013), only incorporate pragmatic reasoning implicitly (in the sense that a supervised model trained on human data may learn to replicate pragmatic behavior). The approach of this dissertation is to take models which capture dynamics of pragmatic language use and apply them to open-domain settings. In this respect, my work builds on research in this vein for referential expression generation (Monroe and Potts, 2015; Andreas and Klein, 2016a), image captioning (Vedantam et al., 2017) and instruction following (Fried et al., 2017), as well as work using neural networks as generative models in Bayesian cognitive architectures (Wu et al., 2015; Liu et al., 2018). The content of the dissertation divides into two parts. The first (chapter 2) focuses on the interpretation of language (particularly non-literal language) using a model of non-literal language previously applied to hyperbole and metaphor interpretation in a setting with a hand-specified and idealized semantics. Here, the goal is to instantiate the same model, but with a semantics derived from a vector space model of word meaning. In this setting, the model remains unchanged, but states are points in an abstract word embedding space - a central computational linguistic representation of meaning (Mikolov et al., 2013; Pennington et al., 2014). The core idea here is that points in the space can be viewed as a continuous analogue of possible worlds, and that linear projections of a vector space are a natural way to represent the aspect of the world that is relevant in a conversation. The second part of the dissertation (chapters 3 and 4) focuses on the production of language, in settings where the length of utterances (and consequently the set of all possible utterances) is unbounded. The core idea here is that pragmatic reasoning can take place incrementally, that is, midway through the saying or hearing of an utterance. This incremental approach is applied to neural language generation tasks, producing informative image captions and translations. The result of these investigations is far from a complete picture, but nevertheless a substantial step towards Bayesian models of semantics and pragmatics which can handle the full richness of natural language, and by doing so provide both explanatory models of meaning and computational systems for producing and interpreting language.
Probabilistic Models of Pragmatics for Natural Language
Author: Reuben Harry Cohn-Gordon
Publisher:
ISBN:
Category :
Languages : en
Pages :
Book Description
Grice (1975) puts forward a view of linguistic meaning in which conversational agents enrich the semantic interpretation of linguistic expressions by recourse to pragmatic reasoning about their interlocutors and world knowledge. As a simple example, on hearing my friend tell me that she read some of War and Peace, I reason that, had she read all of it, she would have said as much, and accordingly that she read only part. It turns out that this perspective is well suited to a probabilistic formalization. In these terms, linguistic meaning is fully characterized by a joint probability distribution P(W; U) between states of the world W and linguistic expressions U. The Gricean perspective described above corresponds to a factoring of this enormously complex distribution into a semantics [[u]](w) : U -> (W -> {0, 1}, world knowledge P(W) and a pair of agents which reason about each other on the assumption that both are cooperative and have access to a commonly known semantics. This third component, of back and forth reasoning between agents, originates in work in game-theory (Franke, 2009; Lewis, 1969) and has been formalized in probabilistic terms by a class of models often collectively referred to as the Rational Speech Acts (RSA) framework (Frank and Goodman, 2012). By allowing for the construction of models which explain in precise terms how Gricean pressures like informativity and relevance interact with a semantics, this framework allows us to take an intuitive theory and explore its predictions beyond the limits of intuition. But it should be more than a theoretical tool. To the extent that its characterization of meaning is correct, it should allow for the construction of computational systems capable of reproducing the dynamics of opendomain natural language. For instance, on the assumption that humans produce language pragmatically, one would expect systems which generate natural language to most faithfully reproduce human behavior when aiming to be not only truthful, but also informative to a hypothetical interlocutor. Likewise, systems which interpret language in a human-like way should perform best when they model language as being generated by an informative speaker. Despite this, standard approaches to many natural language processing (NLP) tasks, like image captioning (Farhadi et al., 2010; Vinyals et al., 2015), translation (Brown et al., 1990; Bahdanau et al., 2014) and metaphor interpretation (Shutova et al., 2013), only incorporate pragmatic reasoning implicitly (in the sense that a supervised model trained on human data may learn to replicate pragmatic behavior). The approach of this dissertation is to take models which capture dynamics of pragmatic language use and apply them to open-domain settings. In this respect, my work builds on research in this vein for referential expression generation (Monroe and Potts, 2015; Andreas and Klein, 2016a), image captioning (Vedantam et al., 2017) and instruction following (Fried et al., 2017), as well as work using neural networks as generative models in Bayesian cognitive architectures (Wu et al., 2015; Liu et al., 2018). The content of the dissertation divides into two parts. The first (chapter 2) focuses on the interpretation of language (particularly non-literal language) using a model of non-literal language previously applied to hyperbole and metaphor interpretation in a setting with a hand-specified and idealized semantics. Here, the goal is to instantiate the same model, but with a semantics derived from a vector space model of word meaning. In this setting, the model remains unchanged, but states are points in an abstract word embedding space - a central computational linguistic representation of meaning (Mikolov et al., 2013; Pennington et al., 2014). The core idea here is that points in the space can be viewed as a continuous analogue of possible worlds, and that linear projections of a vector space are a natural way to represent the aspect of the world that is relevant in a conversation. The second part of the dissertation (chapters 3 and 4) focuses on the production of language, in settings where the length of utterances (and consequently the set of all possible utterances) is unbounded. The core idea here is that pragmatic reasoning can take place incrementally, that is, midway through the saying or hearing of an utterance. This incremental approach is applied to neural language generation tasks, producing informative image captions and translations. The result of these investigations is far from a complete picture, but nevertheless a substantial step towards Bayesian models of semantics and pragmatics which can handle the full richness of natural language, and by doing so provide both explanatory models of meaning and computational systems for producing and interpreting language.
Publisher:
ISBN:
Category :
Languages : en
Pages :
Book Description
Grice (1975) puts forward a view of linguistic meaning in which conversational agents enrich the semantic interpretation of linguistic expressions by recourse to pragmatic reasoning about their interlocutors and world knowledge. As a simple example, on hearing my friend tell me that she read some of War and Peace, I reason that, had she read all of it, she would have said as much, and accordingly that she read only part. It turns out that this perspective is well suited to a probabilistic formalization. In these terms, linguistic meaning is fully characterized by a joint probability distribution P(W; U) between states of the world W and linguistic expressions U. The Gricean perspective described above corresponds to a factoring of this enormously complex distribution into a semantics [[u]](w) : U -> (W -> {0, 1}, world knowledge P(W) and a pair of agents which reason about each other on the assumption that both are cooperative and have access to a commonly known semantics. This third component, of back and forth reasoning between agents, originates in work in game-theory (Franke, 2009; Lewis, 1969) and has been formalized in probabilistic terms by a class of models often collectively referred to as the Rational Speech Acts (RSA) framework (Frank and Goodman, 2012). By allowing for the construction of models which explain in precise terms how Gricean pressures like informativity and relevance interact with a semantics, this framework allows us to take an intuitive theory and explore its predictions beyond the limits of intuition. But it should be more than a theoretical tool. To the extent that its characterization of meaning is correct, it should allow for the construction of computational systems capable of reproducing the dynamics of opendomain natural language. For instance, on the assumption that humans produce language pragmatically, one would expect systems which generate natural language to most faithfully reproduce human behavior when aiming to be not only truthful, but also informative to a hypothetical interlocutor. Likewise, systems which interpret language in a human-like way should perform best when they model language as being generated by an informative speaker. Despite this, standard approaches to many natural language processing (NLP) tasks, like image captioning (Farhadi et al., 2010; Vinyals et al., 2015), translation (Brown et al., 1990; Bahdanau et al., 2014) and metaphor interpretation (Shutova et al., 2013), only incorporate pragmatic reasoning implicitly (in the sense that a supervised model trained on human data may learn to replicate pragmatic behavior). The approach of this dissertation is to take models which capture dynamics of pragmatic language use and apply them to open-domain settings. In this respect, my work builds on research in this vein for referential expression generation (Monroe and Potts, 2015; Andreas and Klein, 2016a), image captioning (Vedantam et al., 2017) and instruction following (Fried et al., 2017), as well as work using neural networks as generative models in Bayesian cognitive architectures (Wu et al., 2015; Liu et al., 2018). The content of the dissertation divides into two parts. The first (chapter 2) focuses on the interpretation of language (particularly non-literal language) using a model of non-literal language previously applied to hyperbole and metaphor interpretation in a setting with a hand-specified and idealized semantics. Here, the goal is to instantiate the same model, but with a semantics derived from a vector space model of word meaning. In this setting, the model remains unchanged, but states are points in an abstract word embedding space - a central computational linguistic representation of meaning (Mikolov et al., 2013; Pennington et al., 2014). The core idea here is that points in the space can be viewed as a continuous analogue of possible worlds, and that linear projections of a vector space are a natural way to represent the aspect of the world that is relevant in a conversation. The second part of the dissertation (chapters 3 and 4) focuses on the production of language, in settings where the length of utterances (and consequently the set of all possible utterances) is unbounded. The core idea here is that pragmatic reasoning can take place incrementally, that is, midway through the saying or hearing of an utterance. This incremental approach is applied to neural language generation tasks, producing informative image captions and translations. The result of these investigations is far from a complete picture, but nevertheless a substantial step towards Bayesian models of semantics and pragmatics which can handle the full richness of natural language, and by doing so provide both explanatory models of meaning and computational systems for producing and interpreting language.
Bayesian Natural Language Semantics and Pragmatics
Author: Henk Zeevat
Publisher: Springer
ISBN: 3319170643
Category : Language Arts & Disciplines
Languages : en
Pages : 256
Book Description
The contributions in this volume focus on the Bayesian interpretation of natural languages, which is widely used in areas of artificial intelligence, cognitive science, and computational linguistics. This is the first volume to take up topics in Bayesian Natural Language Interpretation and make proposals based on information theory, probability theory, and related fields. The methodologies offered here extend to the target semantic and pragmatic analyses of computational natural language interpretation. Bayesian approaches to natural language semantics and pragmatics are based on methods from signal processing and the causal Bayesian models pioneered by especially Pearl. In signal processing, the Bayesian method finds the most probable interpretation by finding the one that maximizes the product of the prior probability and the likelihood of the interpretation. It thus stresses the importance of a production model for interpretation as in Grice’s contributions to pragmatics or in interpretation by abduction.
Publisher: Springer
ISBN: 3319170643
Category : Language Arts & Disciplines
Languages : en
Pages : 256
Book Description
The contributions in this volume focus on the Bayesian interpretation of natural languages, which is widely used in areas of artificial intelligence, cognitive science, and computational linguistics. This is the first volume to take up topics in Bayesian Natural Language Interpretation and make proposals based on information theory, probability theory, and related fields. The methodologies offered here extend to the target semantic and pragmatic analyses of computational natural language interpretation. Bayesian approaches to natural language semantics and pragmatics are based on methods from signal processing and the causal Bayesian models pioneered by especially Pearl. In signal processing, the Bayesian method finds the most probable interpretation by finding the one that maximizes the product of the prior probability and the likelihood of the interpretation. It thus stresses the importance of a production model for interpretation as in Grice’s contributions to pragmatics or in interpretation by abduction.
The Handbook of Contemporary Semantic Theory
Author: Shalom Lappin
Publisher: John Wiley & Sons
ISBN: 1119046823
Category : Language Arts & Disciplines
Languages : en
Pages : 771
Book Description
The second edition of The Handbook of Contemporary Semantic Theory presents a comprehensive introduction to cutting-edge research in contemporary theoretical and computational semantics. Features completely new content from the first edition of The Handbook of Contemporary Semantic Theory Features contributions by leading semanticists, who introduce core areas of contemporary semantic research, while discussing current research Suitable for graduate students for courses in semantic theory and for advanced researchers as an introduction to current theoretical work
Publisher: John Wiley & Sons
ISBN: 1119046823
Category : Language Arts & Disciplines
Languages : en
Pages : 771
Book Description
The second edition of The Handbook of Contemporary Semantic Theory presents a comprehensive introduction to cutting-edge research in contemporary theoretical and computational semantics. Features completely new content from the first edition of The Handbook of Contemporary Semantic Theory Features contributions by leading semanticists, who introduce core areas of contemporary semantic research, while discussing current research Suitable for graduate students for courses in semantic theory and for advanced researchers as an introduction to current theoretical work
Pragmatics and Natural Language Understanding
Author: Georgia M. Green
Publisher: Routledge
ISBN: 1136492828
Category : Language Arts & Disciplines
Languages : en
Pages : 203
Book Description
This book differs from other introductions to pragmatics in approaching the problems of interpreting language use in terms of interpersonal modelling of beliefs and intentions. It is intended to make issues involved in language understanding, such as speech, text, and discourse, accessible to the widest group possible -- not just specialists in linguistics or communication theorists -- but all scholars and researchers whose enterprises depend on having a useful model of how communicative agents understand utterances and expect their own utterances to be understood. Based on feedback from readers over the past seven years, explanations in every chapter have been improved and updated in this thoroughly revised version of the original text published in 1989. The most extensive revisions concern the relevance of technical notions of mutual and normal belief, and the futility of using the notion 'null context' to describe meaning. In addition, the discussion of implicature now includes an extended explication of "Grice's Cooperative Principle" which attempts to put it in the context of his theory of meaning and rationality, and to preclude misinterpretations which it has suffered over the past 20 years. The revised chapter exploits the notion of normal belief to improve the account of conversational implicature.
Publisher: Routledge
ISBN: 1136492828
Category : Language Arts & Disciplines
Languages : en
Pages : 203
Book Description
This book differs from other introductions to pragmatics in approaching the problems of interpreting language use in terms of interpersonal modelling of beliefs and intentions. It is intended to make issues involved in language understanding, such as speech, text, and discourse, accessible to the widest group possible -- not just specialists in linguistics or communication theorists -- but all scholars and researchers whose enterprises depend on having a useful model of how communicative agents understand utterances and expect their own utterances to be understood. Based on feedback from readers over the past seven years, explanations in every chapter have been improved and updated in this thoroughly revised version of the original text published in 1989. The most extensive revisions concern the relevance of technical notions of mutual and normal belief, and the futility of using the notion 'null context' to describe meaning. In addition, the discussion of implicature now includes an extended explication of "Grice's Cooperative Principle" which attempts to put it in the context of his theory of meaning and rationality, and to preclude misinterpretations which it has suffered over the past 20 years. The revised chapter exploits the notion of normal belief to improve the account of conversational implicature.
Probabilistic Models of Natural Language Semantics
Building Probabilistic Models for Natural Language
Author: Stanley F. Chen
Publisher:
ISBN:
Category : Natural language processing (Computer science)
Languages : en
Pages : 152
Book Description
Publisher:
ISBN:
Category : Natural language processing (Computer science)
Languages : en
Pages : 152
Book Description
Speech & Language Processing
Author: Dan Jurafsky
Publisher: Pearson Education India
ISBN: 9788131716724
Category :
Languages : en
Pages : 912
Book Description
Publisher: Pearson Education India
ISBN: 9788131716724
Category :
Languages : en
Pages : 912
Book Description
A Foundation for General-purpose Natural Language Generation
Author: Irene Langkilde-Geary
Publisher:
ISBN:
Category :
Languages : en
Pages : 318
Book Description
Publisher:
ISBN:
Category :
Languages : en
Pages : 318
Book Description
The Alternative Mathematical Model of Linguistic Semantics and Pragmatics
Author: Vilém Novák
Publisher: Springer Science & Business Media
ISBN: 1489923179
Category : Language Arts & Disciplines
Languages : en
Pages : 216
Book Description
In opposition to the classical set theory of natural language, Novák's highly original monograph offers a theory based on alternative and fuzzy sets. This new approach is firmly grounded in semantics and pragmatics, and accounts for the vagueness inherent in natural language-filling a large gap in our current knowledge. The theory will foster fruitful debate among researchers in linguistics and artificial intellegence.
Publisher: Springer Science & Business Media
ISBN: 1489923179
Category : Language Arts & Disciplines
Languages : en
Pages : 216
Book Description
In opposition to the classical set theory of natural language, Novák's highly original monograph offers a theory based on alternative and fuzzy sets. This new approach is firmly grounded in semantics and pragmatics, and accounts for the vagueness inherent in natural language-filling a large gap in our current knowledge. The theory will foster fruitful debate among researchers in linguistics and artificial intellegence.
Foundations of Statistical Natural Language Processing
Author: Christopher Manning
Publisher: MIT Press
ISBN: 0262303795
Category : Language Arts & Disciplines
Languages : en
Pages : 719
Book Description
Statistical approaches to processing natural language text have become dominant in recent years. This foundational text is the first comprehensive introduction to statistical natural language processing (NLP) to appear. The book contains all the theory and algorithms needed for building NLP tools. It provides broad but rigorous coverage of mathematical and linguistic foundations, as well as detailed discussion of statistical methods, allowing students and researchers to construct their own implementations. The book covers collocation finding, word sense disambiguation, probabilistic parsing, information retrieval, and other applications.
Publisher: MIT Press
ISBN: 0262303795
Category : Language Arts & Disciplines
Languages : en
Pages : 719
Book Description
Statistical approaches to processing natural language text have become dominant in recent years. This foundational text is the first comprehensive introduction to statistical natural language processing (NLP) to appear. The book contains all the theory and algorithms needed for building NLP tools. It provides broad but rigorous coverage of mathematical and linguistic foundations, as well as detailed discussion of statistical methods, allowing students and researchers to construct their own implementations. The book covers collocation finding, word sense disambiguation, probabilistic parsing, information retrieval, and other applications.