NDLI: Addressing Common Analytic Challenges to Randomized Experiments in MOOCs: Attrition and Zero-Inflation

Please wait, while we are loading the content...

Achieving 96% Mastery at National Scale through Inspired Learning and Generative Adaptivity

Effective Sampling for Large-scale Automated Writing Evaluation Systems

The Prediction of Student First Response Using Prerequisite Skills

PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance

Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC

Bayesian Ordinal Peer Grading

Staggered Versus All-At-Once Content Release in Massive Open Online Courses: Evaluating a Natural Experiment

Machine Learning for Learning at Scale

Towards Capturing Learners Sentiment and Context

Improving Student Modeling Through Partial Credit and Problem Difficulty

Blended Learning in Indian Colleges with Massively Empowered Classroom

Alumni & Tenured Participants in MOOCs: Analysis of Two Years of MOOC Discussion Channel Activity

Exploring the Effect of Confusion in Discussion Forums of Massive Open Online Courses

An Automated Grading/Feedback System for 3-View Engineering Drawings using RANSAC

Autonomously Generating Hints by Inferring Problem Solving Policies

Measuring and Maximizing the Effectiveness of Honor Codes in Online Courses

Addressing Common Analytic Challenges to Randomized Experiments in MOOCs: Attrition and Zero-Inflation

Attrition and Achievement Gaps in Online Learning

A Playful Game Changer: Fostering Student Retention in Online Education with Social Gamification

Uncovering Trajectories of Informal Learning in Large Online Communities of Creators

Mathematical Language Processing: Automatic Grading and Feedback for Open Response Mathematical Questions

Problems Before Solutions: Automated Problem Clarification at Scale

Technology-Enhanced Learning: Evidence-based Improvement

Structure and messaging techniques for online peer learning systems that increase stickiness

Towards Detecting Wheel-Spinning: Future Failure in Mastery Learning

moocRP: An Open-source Analytics Platform

Probabilistic Use Cases: Discovering Behavioral Patterns for Predicting Certification

BayesRank: A Bayesian Approach to Ranked Peer Grading

A Modern Student Experience inSystems Programming

Educational Evaluation in the PKU SPOC Course "Data Structures and Algorithms"

Learn With Friends: The Effects of Student Face-to-Face Collaborations on Massive Open Online Course Activities

Who You Are or What You Do: Comparing the Predictive Power of Demographics vs. Activity Patterns in Massive Open Online Courses (MOOCs)

Connecting the Dots: Predicting Student Grade Sequences from Bursty MOOC Interactions over Time

Connecting Stories and Pedagogy Increases Participant Engagement in Discussions

Source Effects in Online Education

AutoStyle: Toward Coding Style Feedback at Scale

Assessment of KNOWLA: Knowledge Assembly for Learning and Assessment

An E-Report Scoring Method based on Student Peer Evaluation using Groupware

Adding Third-Party Authentication to Open edX: A Case Study

Game Theory based Peer Grading Mechanisms for MOOCs

Peers in MOOCs: Lessons Based on the Education Production Function, Collective Action, and an Experiment

TELLab: An Experiential Learning Tool for Psychology

Identifying Content-Related Threads in MOOC Discussion Forums

Understanding Learners' General Perception Towards Learning with MOOC Classmates: An Exploratory Study

M-CAFE: Managing MOOC Student Feedback with Collaborative Filtering

Item Ordering Effects with Qualitative Explanations using Online Adaptive Tutoring Data

Learnersourcing of Complex Assessments

Supporting Face-to-Face Like Communication Modalities for Asynchronous Assignment Feedback in Math Education

Practical Measures of Learning Behaviors

Demographic Differences in a Growth Mindset Incentive Structure for Educational Games

Behavior Prediction in MOOCs using Higher Granularity Temporal Information

Analysis of a Large-Scale Formative Writing Assessment System with Automated Feedback

Designing MOOCs as Interactive Places for Collaborative Learning

Toward the Evaluation of Educational Videos using Bayesian Knowledge Tracing and Big Data

The Visible and Invisible in a MOOC Discussion Forum

Exploring Collaborative Storytelling as a Method for Creating Educational Games

Do Numeric Ratings Impact Peer Reviewers?

Learner-Sourcing in an Engineering Class at Scale

Clustering Student Programming Assignments to Multiply Instructor Leverage

Feature Factory: Crowd Sourced Feature Discovery

Are You Listening?: Social Roles and Perceived Value of Statements in Online Learning Communities

All It Takes Is One: Evidence for a Strategy for Seeding Large Scale Peer Learning Interactions

Evaluating the Relationship Between Course Structure, Learner Activity, and Perceived Value of Online Courses

Cultural Relevance in MOOCs: Asking About Socioeconomic Context

Does Online Q&A Activity Vary Based on Topic: A Comparison of Technical and Non-technical Stack Exchange Forums

Clustering-Based Active Learning for CPSGrader

Learning Experiments Using AB Testing at Scale

Using and Designing Platforms for In Vivo Educational Experiments

Supporting Instructors in Collaborating with Researchers using MOOClets

Viz-R: Using Recency to Improve Student and Domain Models

Learning the Features Used To Decide How to Teach

Addressing Common Analytic Challenges to Randomized Experiments in MOOCs: Attrition and Zero-Inflation

Content Provider	ACM Digital Library
Author	Lamb, Anne Smilack, Jascha Reich, Justin Ho, Andrew
Abstract	Massive open online course (MOOC) platforms increasingly allow easily implemented randomized experiments. The heterogeneity of MOOC students, however, leads to two methodological obstacles in analyzing interventions to increase engagement. (1) Many MOOC participation metrics have distributions with substantial positive skew from highly active users as well as zero-inflation from high attrition. (2) High attrition means that in some experimental designs, most users assigned to the treatment never receive it; analyses that do not consider attrition result in "intent-to-treat" (ITT) estimates that underestimate the true effects of interventions. We address these challenges in analyzing an intervention to improve forum participation in the 2014 JusticeX course offered on the edX MOOC platform. We compare the results of four ITT models (OLS, logistic, quantile, and zero-inflated negative binomial regressions) and three "treatment-on-treated" (TOT) models (Wald estimator, 2SLS with a second stage logistic model, and instrumental variables quantile regression). A combination of logistic, quantile, and zero-inflated negative binomial regressions provide the most comprehensive description of the ITT effects. TOT methods then adjust the ITT underestimates. Substantively, we demonstrate that self-assessment questions about forum participation encourage more students to engage in forums and increases the participation of already active students.
Starting Page	21
Ending Page	30
Page Count	10
File Format	PDF
ISBN	9781450334112
DOI	10.1145/2724660.2724669
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2015-03-14
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Engagement Treatment-on-treated A/b testing Zero-inflation Moocs Attrition Randomized controlled trials
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in