Theoretical and Practical Advances in Computer-Based Educational Measurement.

Bibliographic Details
Main Author: Veldkamp, Bernard P.
Other Authors: Sluijter, Cor.
Format: eBook
Language:English
Published: Cham : Springer International Publishing AG, 2019.
Edition:1st ed.
Series:Methodology of Educational Measurement and Assessment Series
Subjects:
Online Access:Click to View
Table of Contents:
  • Intro
  • Preface
  • Introduction
  • Contents
  • Improving Test Quality
  • 1 The Validity of Technology Enhanced Assessments-Threats and Opportunities
  • 1.1 Introduction
  • 1.2 Innovations in Technology-Enhanced Assessments
  • 1.2.1 Innovations in Items and Tasks
  • 1.2.2 Innovations in Test Construction, Assembly and Delivery
  • 1.2.3 Innovations Regarding Personal Needs and Preferences
  • 1.3 Validity and Validation
  • 1.4 Validity of Innovative Technology-Enhanced Assessments
  • 1.4.1 Inferences Within the IUA
  • 1.4.2 Validity Argument of Technology-Enhanced Assessments
  • 1.5 Concluding Remarks
  • References
  • 2 A Framework for Improving the Accessibility of Assessment Tasks
  • 2.1 Accessibility of Assessments
  • 2.2 Principles that Underlie Accessible Assessment Design
  • 2.2.1 Principles from Universal Design
  • 2.2.2 Principles from Cognitive Load Theory
  • 2.3 Evaluating and Improving Accessibility of Assessment Tasks from a Test Takers' Perspective
  • 2.3.1 Supporting Orientation by a Clear Assignment
  • 2.3.2 Supporting Information Processing and Devising Solutions
  • 2.3.3 Facilitating Responding
  • 2.3.4 Facilitating Monitoring and Adjusting
  • 2.4 An Application of the Test Accessibility Framework: The Dutch Driving Theory Exam
  • 2.4.1 Innovations in the Dutch Traffic Theory Exam for Car Drivers
  • 2.4.2 Applied Modifications in the Response Mode of Theory Items
  • 2.4.3 Psychometric Indications of Accessibility Improvement?
  • 2.4.4 Item Selection
  • 2.4.5 Data Collection
  • 2.4.6 Data Analyses
  • 2.4.7 Results
  • 2.5 Discussion
  • References
  • 3 The Design and Validation of the Renewed Systems-Oriented Talent Management Model
  • 3.1 Introduction
  • 3.1.1 Problem Situation and Purpose of the Study
  • 3.2 Theoretical Framework
  • 3.2.1 The Management Building Blocks Framework
  • 3.2.2 Systems Theory.
  • 3.2.3 Evidence-Based Systems-Oriented Talent Management
  • 3.3 Renewed STM Diagrams
  • 3.3.1 Renewed STM Diagram 1: Aligning Organisational Structure and Human Talent
  • 3.3.2 Renewed STM Diagram 2: Aligning Organisational Culture and Human Talent
  • 3.3.3 Renewed STM Diagram 3: Aligning Business Strategy and Human Talent
  • 3.4 Implications of the STM for Educational Measurement
  • 3.5 Conclusion, Limitations, and Recommendations
  • 3.5.1 Conclusion
  • 3.5.2 Application to Educational Measurement
  • 3.5.3 Limitations
  • 3.5.4 Recommendations and Implications
  • References
  • 4 Assessing Computer-Based Assessments
  • 4.1 Introduction
  • 4.2 The RCEC Review System for the Evaluation of Computer-Based Tests
  • 4.2.1 Purpose and Use of the Educational Test or Exam
  • 4.2.2 Quality of Test Material
  • 4.2.3 Representativeness
  • 4.2.4 Reliability
  • 4.2.5 Standard Setting and Standard Maintenance
  • 4.2.6 Test Administration and Security
  • 4.3 Reviewing a Computer Based Test
  • 4.3.1 Purpose and Use of the Test
  • 4.3.2 Quality of Test Material
  • 4.3.3 Representativeness
  • 4.3.4 Reliability (Measurement Precision)
  • 4.3.5 Standard Setting and Standard Maintenance
  • 4.3.6 Test Administration and Security
  • 4.3.7 Review Conclusion
  • 4.4 Discussion
  • References
  • Psychometrics
  • 5 Network Psychometrics in Educational Practice
  • 5.1 Introduction
  • 5.2 The Curie-Weiss Model
  • 5.2.1 Some Statistical Properties of the Curie-Weiss Model
  • 5.2.2 The Curie-Weiss to Rasch Connection
  • 5.3 Maximum Likelihood Estimation of the Curie-Weiss Model
  • 5.3.1 Maximum Likelihood in the Complete Data Case
  • 5.3.2 Maximum Likelihood Estimation in the Incomplete Data Case
  • 5.3.3 The M-Step
  • 5.4 Numerical Illustrations
  • 5.4.1 Simulated Example
  • 5.4.2 The Cito Eindtoets 2012
  • 5.5 Discussion
  • References.
  • 6 On the Number of Items in Testing Mastery of Learning Objectives
  • 6.1 Introduction
  • 6.2 Method
  • 6.2.1 Simulation Study with Homogeneous Item Characteristics
  • 6.2.2 Empirical Example
  • 6.2.3 Simulation Study Based on Empirical Data and Heterogeneous Item Characteristics
  • 6.2.4 Estimating and Validating a Predictive Model for Bayes Factors
  • 6.3 Results
  • 6.3.1 Simulation Study with Homogeneous Item Characteristics
  • 6.3.2 Empirical Example
  • 6.3.3 Simulation Based on the Empirical Data and with Heterogeneous Item Characteristics
  • 6.3.4 Prediction Model
  • 6.4 Discussion and Conclusions
  • References
  • 7 Exponential Family Models for Continuous Responses
  • 7.1 Introduction
  • 7.2 A Rasch Model for Continuous Responses
  • 7.2.1 The Model
  • 7.2.2 Parameter Estimation
  • 7.3 An Extension of the Müller Model
  • 7.3.1 The Model
  • 7.3.2 Parameter Estimation
  • 7.4 Comparison of Information Functions Across Models
  • 7.4.1 The Unit of the Latent Variable
  • 7.4.2 An Example
  • 7.5 Discussion
  • Appendix
  • References
  • 8 Tracking Ability: Defining Trackers for Measuring Educational Progress
  • 8.1 Introduction
  • 8.2 Methods
  • 8.2.1 Formalizing a Tracker
  • 8.2.2 Example of a Tracker
  • 8.2.3 Convergence in Kullback-Leibler Divergence
  • 8.2.4 Simulating Surveys
  • 8.3 Discussion
  • References
  • 9 Finding Equivalent Standards in Small Samples
  • 9.1 Introduction
  • 9.2 Method
  • 9.3 Results
  • 9.4 Conclusion and Discussion
  • References
  • Large Scale Assessments
  • 10 Clustering Behavioral Patterns Using Process Data in PIAAC Problem-Solving Items
  • 10.1 Introduction
  • 10.1.1 Problem-Solving Items in PIAAC
  • 10.1.2 Employability and PSTRE Skills
  • 10.2 Method
  • 10.2.1 Sample
  • 10.2.2 Instrumentation
  • 10.2.3 Features Extracted from Process Data
  • 10.2.4 Clustering Sequence Data
  • 10.2.5 K-Means Clustering
  • 10.3 Results.
  • 10.3.1 Cluster Determination
  • 10.3.2 Cluster Membership and Proficiency Level
  • 10.3.3 Cluster Membership and Employment-Based Background Variables
  • 10.4 Discussion
  • References
  • 11 Reliability Issues in High-Stakes Educational Tests
  • 11.1 Outline of the Problem
  • 11.2 Preliminaries
  • 11.3 MAP Proficiency Estimates Based on Number-Correct Scores
  • 11.4 Equating Error
  • 11.5 Simulation Study of Equating Errors
  • 11.6 Conclusion
  • References
  • 12 Differential Item Functioning in PISA Due to Mode Effects
  • 12.1 Introduction
  • 12.2 Changes in PISA 2015
  • 12.3 Data
  • 12.4 Differential Item Functioning
  • 12.5 Results
  • 12.5.1 DIF Between Modes
  • 12.5.2 Trend Effects in the Netherlands
  • 12.6 Conclusions and Discussion
  • References
  • 13 Investigating Rater Effects in International Large-Scale Assessments
  • 13.1 Introduction
  • 13.2 Scoring Human-Coded Items in PISA 2015
  • 13.2.1 Categorization of Items by Item Formats
  • 13.2.2 Coding Design and Procedures
  • 13.3 Construct Equivalence of Different Scoring Types in PISA
  • 13.3.1 Methods
  • 13.3.2 Findings
  • 13.4 Rater Effects that Are Comparable Across Countries
  • 13.4.1 Methods
  • 13.4.2 Findings
  • 13.5 Conclusion
  • References
  • Computerized Adaptive Testing in Educational Measurement
  • 14 Multidimensional Computerized Adaptive Testing for Classifying Examinees
  • 14.1 Introduction
  • 14.2 Multidimensional Item Response Theory
  • 14.3 Classification Methods
  • 14.3.1 The SPRT for Between-Item Multidimensionality
  • 14.3.2 The Confidence Interval Method for Between-Item Multidimensionality
  • 14.3.3 The SPRT for Within-Item Multidimensionality
  • 14.3.4 The Confidence Interval Method for Within-Item Multidimensionality
  • 14.4 Item Selection Methods
  • 14.4.1 Item Selection Methods for Between-Item Multidimensionality.
  • 14.4.2 Item Selection Methods for Within-Item Multidimensionality
  • 14.5 Examples
  • 14.5.1 Example 1: Between-Item Multidimensionality
  • 14.5.2 Example 2: Within-Item Multidimensionality
  • 14.6 Conclusions and Discussion
  • References
  • 15 Robust Computerized Adaptive Testing
  • 15.1 Introduction
  • 15.2 Robust Test Assembly
  • 15.3 Robust CAT Assembly
  • 15.3.1 Constructing a Robust Item Pool
  • 15.3.2 Numerical Example to Illustrate the Concept of Robust Item Pools
  • 15.3.3 Towards an Algorithm for Robust CAT
  • 15.4 Simulation Studies
  • 15.4.1 Study 1
  • 15.4.2 Study 2
  • 15.4.3 Study 3
  • 15.4.4 Study Setup
  • 15.5 Results
  • 15.6 Conclusion
  • References
  • 16 On-the-Fly Calibration in Computerized Adaptive Testing
  • 16.1 Introduction
  • 16.1.1 Replenishment Strategies and On-the-Fly Calibration
  • 16.1.2 On-the-Fly Calibration Methods
  • 16.1.3 The Use of Reference Items in Modelling Bias
  • 16.1.4 The Need for Underexposure Control
  • 16.1.5 A Combination of Calibration Methods
  • 16.2 Research Questions
  • 16.3 Simulation Studies
  • 16.3.1 Use of Reference Items in Elimination of Bias
  • 16.3.2 Comparison of the Methods
  • 16.4 Discussion
  • References
  • 17 Reinforcement Learning Applied to Adaptive Classification Testing
  • 17.1 Introduction
  • 17.2 Method
  • 17.3 Framework
  • 17.3.1 General Idea
  • 17.3.2 Sequential Classification
  • 17.3.3 Item Selection
  • 17.3.4 Algorithm
  • 17.4 Experiments
  • 17.5 Discussion
  • References
  • Technological Developments in Educational Measurement
  • 18 Feasibility and Value of Using a GoPro Camera and iPad to Study Teacher-Student Assessment Feedback Interactions
  • 18.1 Introduction
  • 18.1.1 The Value of Video Feedback
  • 18.2 Method
  • 18.2.1 Participants and Context
  • 18.2.2 Data Collection Instruments and Procedures
  • 18.2.3 Analysis
  • 18.3 Results
  • 18.3.1 Technical Results.
  • 18.3.2 Teacher and Student Experiences.