Automated Scoring of Complex Tasks in Computer-based Testing PDF Download

Author: David M. Williamson
Publisher: Psychology Press
ISBN: 0805846344
Category : Education
Languages : en
Pages : 437

View

Book Description
This is the first volume to provide the latest methods and examples of "best practices" in the design, implementation, and evaluation of automated scoring for complex assessments. The contributing authors, all noted leaders in the field, introduce each m

Author: Robert W. Lissitz
Publisher: IAP
ISBN: 1617357278
Category : Education
Languages : en
Pages : 327

View

Book Description
The Race To The Top program strongly advocates the use of computer technology in assessments. It dramatically promotes computer-based testing, linear or adaptive, in K-12 state assessment programs. Moreover, assessment requirements driven by this federal initiative exponentially increase the complexity in assessment design and test development. This book provides readers with a review of the history and basics of computer-based tests. It also offers a macro perspective for designing such assessment systems in the K-12 setting as well as a micro perspective on new challenges such as innovative items, scoring of such items, cognitive diagnosis, and vertical scaling for growth modeling and value added approaches to assessment. The editors’ goal is to provide readers with necessary information to create a smarter computer-based testing system by following the advice and experience of experts from education as well as other industries. This book is based on a conference (http://marces.org/workshop.htm) held by the Maryland Assessment Research Center for Education Success. It presents multiple perspectives including test vendors and state departments of education, in designing and implementing a computer-based test in the K-12 setting. The design and implementation of such a system requires deliberate planning and thorough considerations. The advice and experiences presented in this book serve as a guide to practitioners and as a good source of information for quality control. The technical issues discussed in this book are relatively new and unique to K-12 large-scale computer-based testing programs, especially due to the recent federal policy. Several chapters provide possible solutions to psychometricians dealing with the technical challenges related to innovative items, cognitive diagnosis, and growth modeling in computer-based linear or adaptive tests in the K-12 setting.

Author: Klaus Zechner
Publisher: Routledge
ISBN: 1351676105
Category : Language Arts & Disciplines
Languages : en
Pages : 194

View

Book Description
Automated Speaking Assessment: Using Language Technologies to Score Spontaneous Speech provides a thorough overview of state-of-the-art automated speech scoring technology as it is currently used at Educational Testing Service (ETS). Its main focus is related to the automated scoring of spontaneous speech elicited by TOEFL iBT Speaking section items, but other applications of speech scoring, such as for more predictable spoken responses or responses provided in a dialogic setting, are also discussed. The book begins with an in-depth overview of the nascent field of automated speech scoring—its history, applications, and challenges—followed by a discussion of psychometric considerations for automated speech scoring. The second and third parts discuss the integral main components of an automated speech scoring system as well as the different types of automatically generated measures extracted by the system features related to evaluate the speaking construct of communicative competence as measured defined by the TOEFL iBT Speaking assessment. Finally, the last part of the book touches on more recent developments, such as providing more detailed feedback on test takers’ spoken responses using speech features and scoring of dialogic speech. It concludes with a discussion, summary, and outlook on future developments in this area. Written with minimal technical details for the benefit of non-experts, this book is an ideal resource for graduate students in courses on Language Testing and Assessment as well as teachers and researchers in applied linguistics.

Author: Duanli Yan
Publisher: CRC Press
ISBN: 1351264788
Category : Computers
Languages : en
Pages : 385

View

Book Description
"Automated scoring engines [...] require a careful balancing of the contributions of technology, NLP, psychometrics, artificial intelligence, and the learning sciences. The present handbook is evidence that the theories, methodologies, and underlying technology that surround automated scoring have reached maturity, and that there is a growing acceptance of these technologies among experts and the public." From the Foreword by Alina von Davier, ACTNext Senior Vice President Handbook of Automated Scoring: Theory into Practice provides a scientifically grounded overview of the key research efforts required to move automated scoring systems into operational practice. It examines the field of automated scoring from the viewpoint of related scientific fields serving as its foundation, the latest developments of computational methodologies utilized in automated scoring, and several large-scale real-world applications of automated scoring for complex learning and assessment systems. The book is organized into three parts that cover (1) theoretical foundations, (2) operational methodologies, and (3) practical illustrations, each with a commentary. In addition, the handbook includes an introduction and synthesis chapter as well as a cross-chapter glossary.

Author: Dave Bartram
Publisher: John Wiley & Sons
ISBN: 0470861932
Category : Psychology
Languages : en
Pages : 272

View

Book Description
No topic is more central to innovation and current practice in testing and assessment today than computers and the Internet. This timely publication highlights four main themes that define current issues, technical advances and applications of computer-based testing: Advances in computer-based testing -- new test designs, item selection algorithms, exposure control issues and methods, and new tests that capitalize on the power of computer technology. Operational issues -- systems design, test security, and legal and ethical matters. New and improved uses -- for tests in employment and credentialing. The future of computer-based testing -- identifying potential issues, developments, major advances and problems to overcome. Written by internationally recognized contributors, each chapter focuses on issues of control, quality, security and technology. These issues provide the basic structure for the International Test Commission's new Guidelines on Computer-Based Testing and Testing on the Internet. The contributions to this book have played a key role in the development of these guidelines. Computer-Based Testing and the Internet is a comprehensive guide for all professionals, academics and practitioners working in the fields of education, credentialing, personnel testing and organizational assessment. It will also be of value to students developing expertise in these areas.

Author: Craig S. Wells
Publisher: Guilford Publications
ISBN: 1462525636
Category : Social Science
Languages : en
Pages : 494

View

Book Description
This book introduces and explores major topics in contemporary educational measurement: criterion-referenced testing, item response theory (IRT), computer-based testing, cross-lingual and cross-cultural assessment, and accountability testing. Psychometric experts describe forward-looking measurement practices and provide a contextualized understanding of how and why they were developed, how they can be used, and where they may go in the future. In addition to presenting key concepts and formulas, the volume covers established and emerging applications and discusses recurrent challenges that require additional research. A helpful glossary of abbreviations is included. The book is grounded in the work of Ronald K. Hambleton.

Author: Patrick Griffin
Publisher: Springer Science & Business Media
ISBN: 9400723245
Category : Education
Languages : en
Pages : 359

View

Book Description
Rapid—and seemingly accelerating—changes in the economies of developed nations are having a proportional effect on the skill sets required of workers in many new jobs. Work environments are often technology-heavy, while problems are frequently ill-defined and tackled by multidisciplinary teams. This book contains insights based on research conducted as part of a major international project supported by Cisco, Intel and Microsoft. It faces these new working environments head-on, delineating new ways of thinking about ‘21st-century’ skills and including operational definitions of those skills. The authors focus too on fresh approaches to educational assessment, and present methodological and technological solutions to the barriers that hinder ICT-based assessments of these skills, whether in large-scale surveys or classrooms. Equally committed to defining its terms and providing practical solutions, and including international perspectives and comparative evaluations of assessment methodology and policy, this volume tackles an issue at the top of most educationalists’ agendas.

Author: Victoria Yaneva
Publisher: Taylor & Francis
ISBN: 1000904199
Category : Education
Languages : en
Pages : 339

View

Book Description
Advancing Natural Language Processing in Educational Assessment examines the use of natural language technology in educational testing, measurement, and assessment. Recent developments in natural language processing (NLP) have enabled large-scale educational applications, though scholars and professionals may lack a shared understanding of the strengths and limitations of NLP in assessment as well as the challenges that testing organizations face in implementation. This first-of-its-kind book provides evidence-based practices for the use of NLP-based approaches to automated text and speech scoring, language proficiency assessment, technology-assisted item generation, gamification, learner feedback, and beyond. Spanning historical context, validity and fairness issues, emerging technologies, and implications for feedback and personalization, these chapters represent the most robust treatment yet about NLP for education measurement researchers, psychometricians, testing professionals, and policymakers. The Open Access version of this book, available at www.taylorfrancis.com, has been made available under a Creative Commons Attribution-NonCommercial-No Derivatives 4.0 license.

Author: Michael Fetzer
Publisher: Springer Science & Business Media
ISBN: 146147681X
Category : Psychology
Languages : en
Pages : 281

View

Book Description
​This book provides a comprehensive and state-of-the-art overview of simulation development, technologies, and implementation, including real-world examples and results followed by a preview of what’s on the horizon that will further revolutionize the industry. More than a handful of books have been written on the use of simulations for training purposes, but this book focuses solely on simulations in employee selection contexts (e.g., hiring, promotion), making it a truly unique and valuable resource for both practitioners and academics. The science and practice of employee selection has advanced at a steady pace over the past two or three decades. However, recent advancements in both technology and assessment methods have been the catalyst for an evolutionary leap in the use of simulations in this area.

Author: Craig N. Mills
Publisher: Routledge
ISBN: 1135651655
Category : Education
Languages : en
Pages : 339

View

Book Description
Although computer-based tests (CBT) have been administered for many years, improvements in the speed and power of computers coupled with reductions in their cost have made large-scale computer delivery of tests increasingly feasible. CBT is now a common form of test delivery for licensure, certification, and admissions tests. Many large-scale, high-stakes testing programs have introduced CBT either as an option or as the sole means of test delivery. Although this movement to CBT has, to a great extent, been successful, it has not been without problems. Advances in psychometrics are required to ensure that those who rely on test results can have at least the same confidence in CBTs as they have in traditional forms of assessment. This volume stems from an ETS-sponsored colloquium in which more than 200 measurement professionals from eight countries and 29 states convened to assess the current and future status of CBT. The formal agenda for the colloquium was divided into three major segments: Test Models, Test Administration, and Test Analysis and Scoring. Each segment consisted of several presentations followed by comments from noted psychometricians and a break-out session in which presenters and discussants identified important issues and established priorities for a CBT research agenda. This volume contains the papers presented at the colloquium, the discussant remarks based on those papers, and the research agenda that was generated from the break-out sessions. Computer-Based Testing: Building the Foundation for Future Assessments is must reading for professionals, scholars, and advanced students working in the testing field, as well as people in the information technology field who have an interest in testing.