AI agent skill · by @NeuroAIHub

Reading Time Analysis

Guides analysis of eye-tracking reading measures including first fixation, gaze duration, regression path, and total reading time

Archived 2026-10-23 · Source updated 2026-10-02 · License: AGPL-3.0. Check the original repository for the latest version.

SKILL.md instructions

# Reading Time Analysis

## Purpose

This skill encodes expert methodological knowledge for analyzing eye-tracking data from reading experiments. A competent programmer without psycholinguistics training would likely compute a single "reading time" per word, missing the critical insight that different eye-tracking measures tap different stages of language processing. Choosing the wrong measure for your research question -- or failing to account for spillover effects, skipping patterns, and the distinction between first-pass and second-pass reading -- leads to misattribution of cognitive processes.

## When to Use

Use this skill when:

- Analyzing eye-movement data from reading experiments (sentence or passage reading)
- Selecting which eye-tracking measures to report for a given linguistic manipulation
- Defining regions of interest and handling spillover effects
- Setting up statistical models for eye-tracking reading data
- Cleaning and filtering fixation data for reading analyses

Do **not** use this skill when:

- Analyzing self-paced reading data (see `self-paced-reading-designer` for that paradigm)
- Analyzing eye movements in visual search or scene viewing (different fixation patterns)
- Working with eye-tracking data from non-reading tasks (e.g., visual world paradigm)

## Research Planning Protocol

Before executing the domain-specific steps below, you MUST:

1. **State the research question** -- What specific question is this analysis/paradigm addressing?
2. **Justify the method choice** -- Why is this approach appropriate? What alternatives were considered?
3. **Declare expected outcomes** -- What results would support vs. refute the hypothesis?
4. **Note assumptions and limitations** -- What does this method assume? Where could it mislead?
5. **Present the plan to the user and WAIT for confirmation** before proceeding.

For detailed methodology guidance, see the `research-literacy` skill.


## ⚠️ Verification Notice

This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please [open an issue](https://github.com/HaoxuanLiTHUAI/awesome_cognitive_and_neuroscience_skills/issues).

## Eye-Tracking Reading Measures Hierarchy

### Measure Definitions and Cognitive Interpretations

The following measures are ordered from earliest to latest processing stages. This hierarchy reflects the temporal unfolding of language comprehension during reading (Rayner, 1998, 2009; Clifton et al., 2007).

#### First-Pass Measures (Before Leaving the Region)

| Measure | Definition | Cognitive Process | When to Use |
|---------|-----------|------------------|-------------|
| **First Fixation Duration (FFD)** | Duration of the first fixation on a word during first pass | Early lexical access; initial contact with the word (Rayner, 1998) | When testing early word recognition effects (frequency, predictability) |
| **Single Fixation Duration (SFD)** | Duration of the only fixation on a word, when exactly one first-pass fixation occurs | Cleaner measure of early lexical processing than FFD (Rayner, 2009) | When most words receive one fixation; avoids refixation confounds |
| **Gaze Duration (GD)** | Sum of all first-pass fixation durations on a word (before eyes leave the word in either direction) | Lexical processing / word identification (Rayner, 1998, 2009) | **Default first-pass measure** for most word-level analyses |

#### Late Measures (After Leaving the Region)

| Measure | Definition | Cognitive Process | When to Use |
|---------|-----------|------------------|-------------|
| **Go-Past Time (GPT)** / Regression Path Duration | Time from first fixation on the word until first fixation to the right of the word (includes any regressions out and back) | Integration difficulty; signals reanalysis of prior material (Clifton et al., 2007) | When testing syntactic garden-path effects, semantic anomalies, discourse integration |
| **Total Reading Time (TRT)** | Sum of all fixation durations on a word (first pass + regressions back) | Overall processing difficulty (Rayner, 1998) | When interested in total processing cost regardless of time course |
| **Regression Probability (Reg-out)** | Binary: did the reader make a regression from this region? | Reanalysis / comprehension difficulty (Clifton et al., 2007) | When interested in whether (not how long) reanalysis occurred |
| **Regression-in Probability** | Binary: did the reader regress back to this region from downstream? | Downstream difficulty triggers revisitation (Rayner & Pollatsek, 1989) | When testing whether a region is revisited after later processing fails |

### Decision Tree: Which Measure for Which Question?

```
What stage of processing is your manipulation expected to affect?
|
+-- EARLY LEXICAL (word frequency, orthographic regularity, predictability)
| |
| +-- Use GAZE DURATION as primary measure (Rayner, 1998, 2009)
| +-- Report FIRST FIXATION DURATION as supplementary
| +-- Report SINGLE FIXATION DURATION if high proportion of
| single-fixation cases (Rayner, 2009)
|
+-- LATE LEXICAL / POST-LEXICAL (semantic plausibility, thematic fit)
| |
| +-- Use GAZE DURATION for early effects
| +-- Use GO-PAST TIME for integration effects (Clifton et al., 2007)
| +-- Use TOTAL READING TIME for overall effects
|
+-- SYNTACTIC (garden-path, structural ambiguity, reanalysis)
| |
| +-- Use GO-PAST TIME as primary measure (Clifton et al., 2007)
| +-- Use REGRESSION PROBABILITY as complementary binary measure
| +-- Effects often appear in the SPILLOVER REGION (1-2 words
| post-critical; Rayner & Pollatsek, 1989)
|
+-- DISCOURSE / PRAGMATIC (reference resolution, inference, coherence)
| |
| +-- Use GO-PAST TIME and TOTAL READING TIME
| +-- Effects are typically late and may span multiple words
| +-- Consider REGRESSION-IN probability for earlier regions
|
+-- EXPLORATORY / UNKNOWN TIMING
 |
 +-- Report ALL major measures: FFD, GD, GPT, TRT, Reg-out
 +-- Let the pattern across measures inform process interpretation
```

### First-Pass vs. Second-Pass Distinction

| Category | Definition | Includes |
|----------|-----------|----------|
| **First pass** | All fixations from first entering a region until first leaving it (in either direction) | FFD, SFD, GD |
| **Second pass** | All fixations on a region after first leaving it | Re-reading time (TRT minus first-pass time) |

**Why this matters**: First-pass measures reflect initial processing; second-pass measures reflect recovery from processing difficulty encountered downstream. Conflating them obscures when processing difficulty arose.

## Region of Interest (ROI) Definition

### Word-Level ROIs

- The most common unit of analysis is the **single word** (Rayner, 1998)
- For multi-word critical regions, report analyses at both word level and region level

### Multi-Word ROIs

- Sometimes necessary for syntactic manipulations where the critical structure spans multiple words
- Define ROIs **a priori** based on linguistic structure, not post-hoc based on where effects appear
- Report the number of characters and words in each ROI

### Spillover Effects

Spillover is the delayed manifestation of a processing effect on fixations one or more words downstream of the critical word (Rayner & Pollatsek, 1989).

- **Typical spillover range**: **1-2 words** after the critical word (Rayner, 1998)
- **Always analyze the spillover region** (word n+1, sometimes n+2) in addition to the critical word
- Spillover is most common for first-pass measures (GD, FFD)
- **Pre-target region** (word n-1) should also be checked to verify no confounding baseline differences

### Parafoveal Preview Effects

- Words are partially processed before they are directly fixated -- the **parafoveal preview benefit** (Rayner, 1975; Rayner, 2009)
- Parafoveal preview extends to approximately **7-8 characters** to the right of fixation in English (McConkie & Rayner, 1975)
- This means effects of word n's properties can appear on the **last fixation of word n-1** (parafoveal-on-foveal effects; Drieghe et al., 2008)

## Data Cleaning

### Fixation Duration Cutoffs

| Criterion | Value | Rationale | Citation |
|-----------|-------|-----------|----------|
| **Short fixation merge** | < **80 ms** within **1 character** of another fixation: merge with nearest fixation | Too brief for meaningful processing; likely corrective saccade (Rayner & Pollatsek, 1989) |
| **Short fixation exclude** | < **80 ms** (not adjacent to another fixation): exclude | Not informative for reading (Rayner & Pollatsek, 1989) |
| **Long fixation exclude** | > **800 ms**: exclude | Likely track loss, inattention, or blink artifact (Rayner & Pollatsek, 1989) |
| **Alternative long cutoff** | > **1000 ms** or > **1200 ms** | Used in some labs; report which cutoff and justify |

**Note**: Some researchers use **50 ms** as the lower bound and **1000-1200 ms** as the upper bound. The critical requirement is to **report your exact cutoffs and the percentage of data excluded**.

### Trial-Level Exclusions

| Criterion | Action | Rationale |
|-----------|--------|-----------|
| **Track loss** | Exclude trial | Unreliable position data |
| **Blinks on critical region** | Exclude trial | Missing fixation data on the ROI |
| **First-pass skip of critical word** | Exclude from first-pass measures (FFD, SFD, GD); include in TRT | Word was not fixated during first pass |
| **Comprehension accuracy** | Exclude participants below **80%** on comprehension questions | Ensures reading for comprehension (Rayner et al., 2006) |

### Skipping Rate Considerations

- Short, high-frequency, and predictable words are skipped **10-30%** of the time (Rayner, 1998, 2009)
- Content words are skipped ~**15%** of the time; function words ~**35%** (Rayner, 2009)
- If skipping rates differ across conditions, this is informative -- report it
- For first-pass measures, words that are skipped contribute **no data**, not zero reading time
- **Do not substitute zero for skipped words** -- this conflates fast processing with no fixation

## Statistical Modeling

### Linear Mixed-Effects Models (LMMs)

Eye-tracking reading data should be analyzed with LMMs with **crossed random effects** for subjects and items (Baayen et al., 2008; Baayen, Davidson, & Bates, 2008):

```
# R formula (lme4 syntax):
gaze_duration ~ condition + (1 + condition | subject) + (1 + condition | item)
```

**Why crossed random effects**: Reading experiments use a **Latin square** design where every subject sees every item, but items rotate across conditions between subjects. Both subjects and items are random samples, and both contribute variance (Clark, 1973; Baayen et al., 2008).

### Random Effects Structure

| Approach | Specification | When to Use | Citation |
|----------|--------------|-------------|----------|
| **Maximal** | Random intercepts + all random slopes justified by design | Default starting point | Barr et al., 2013 |
| **Parsimonious** | Remove random correlations first, then random slopes that explain ~0 variance | When maximal model fails to converge | Bates et al., 2015; Matuschek et al., 2017 |

**Convergence protocol** (Barr et al., 2013; Bates et al., 2015):

1. Fit maximal model (all by-subject and by-item random slopes for within-unit factors)
2. If convergence fails: remove correlations between random effects (use `||` in lme4)
3. If still fails: remove the random slope with the smallest variance component
4. Report the final model structure and note any simplifications

### Distributional Considerations

Reading times are **right-skewed** and bounded below by zero. Options:

| Approach | When to Use | Citation |
|----------|-------------|----------|
| **Log-transform** | Simple; commonly used; adequate for many datasets | Standard in psycholinguistics |
| **Inverse transform (-1000/RT)** | Can outperform log for skewed RT data | Baayen & Milin, 2010 |
| **Generalized LMM (Gamma)** | Models the skewness directly; avoids back-transformation issues | Lo & Andrews, 2015 |
| **Raw RT with residual checks** | When effects are large and residuals are approximately normal | Baayen et al., 2008 |

**Recommendation**: Start with **raw reading times** in the LMM. Check residual plots. If residuals are non-normal, apply **log-transformation** or fit a **GLMM with Gamma family and identity link** (Lo & Andrews, 2015).

### Multiple Comparisons

When analyzing multiple reading measures on the same data:

- **Do not apply Bonferroni correction across measures** -- each measure tests a different theoretical question (Clifton et al., 2007)
- **Do correct** within each measure if testing multiple contrasts
- Report effect sizes and confidence intervals alongside p-values

## Typical Fixation Duration Benchmarks

These values serve as sanity checks for data quality (Rayner, 1998, 2009):

| Measure | Typical Range (Silent Reading) | Citation |
|---------|-------------------------------|----------|
| **Average fixation duration** | **200-250 ms** | Rayner, 1998, 2009 |
| **Average saccade length** | **7-9 characters** (~2 degrees) | Rayner, 1998, 2009 |
| **Regression rate** | **10-15%** of all saccades | Rayner, 1998 |
| **Word skipping rate** | **Content words ~15%; function words ~35%** | Rayner, 2009 |
| **Fixation duration range** | **50-500 ms** (bulk of distribution) | Rayner, 1998 |

If your data substantially deviates from these benchmarks, check calibration quality, task instructions, and participant compliance.

## Common Pitfalls

1. **Using only total reading time**: TRT conflates early and late processing. If you only report TRT, you cannot determine when the effect arose. Always report at least one first-pass measure (GD) and one late measure (GPT or TRT) (Clifton et al., 2007).

2. **Ignoring spillover effects**: Many effects appear 1-2 words downstream of the critical word, especially for syntactic manipulations. Always analyze the spillover region (Rayner, 1998; Rayner & Pollatsek, 1989).

3. **Substituting zero for skipped words**: Skipped words should be treated as missing data for first-pass measures, not as zero reading time. Substituting zero artificially deflates means and inflates variance.

4. **Using ANOVA instead of LMMs**: F1/F2 ANOVA is outdated for psycholinguistic data. LMMs with crossed random effects properly handle the variance structure (Baayen et al., 2008; Barr et al., 2013).

5. **Over-interpreting first fixation duration**: FFD is contaminated by refixation planning. When a substantial proportion of words receive multiple first-pass fixations, GD is more informative (Rayner, 2009).

6. **Defining ROIs post-hoc**: Selecting regions of interest after seeing the data inflates Type I error. Define ROIs a priori based on linguistic theory.

7. **Ignoring comprehension accuracy**: If participants are not reading for comprehension (accuracy < 80%), eye-movement patterns are not interpretable as reflecting normal reading processes (Rayner et al., 2006).

8. **Not reporting data loss**: Always report the percentage of trials excluded at each cleaning step and the percentage of words skipped in the critical region.

## Minimum Reporting Checklist

Based on Clifton et al. (2007) and current standards in psycholinguistics:

- [ ] Eye-tracker model and sampling rate (minimum **1000 Hz** recommended; **500 Hz** acceptable; Rayner, 2009)
- [ ] Viewing distance and display specifications (font size, characters per degree)
- [ ] Calibration procedure and accuracy threshold (typically < **0.5 degrees** average error)
- [ ] Fixation duration cutoffs (lower and upper bounds) with citations
- [ ] Data cleaning steps and percentage of data excluded at each step
- [ ] Skipping rates for the critical region by condition
- [ ] ROI definitions with linguistic justification
- [ ] All relevant reading measures (at minimum: GD, GPT, TRT for the critical region; GD for spillover)
- [ ] Statistical model specification (random effects structure, any transformations)
- [ ] Software for data analysis (with version)
- [ ] Comprehension question accuracy (mean, exclusion threshold)
- [ ] Number of participants and items after exclusions

## References

- Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. *Journal of Memory and Language*, 59, 390-412.
- Baayen, R. H., & Milin, P. (2010). Analyzing reaction times. *International Journal of Psychological Research*, 3, 12-28.
- Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. *Journal of Memory and Language*, 68, 255-278.
- Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). Parsimonious mixed models. arXiv:1506.04967.
- Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. *Journal of Verbal Learning and Verbal Behavior*, 12, 335-359.
- Clifton, C., Staub, A., & Rayner, K. (2007). Eye movements in reading words and sentences. In R. P. G. van Gompel, M. H. Fischer, W. S. Murray, & R. L. Hill (Eds.), *Eye movements: A window on mind and brain*. Amsterdam: Elsevier.
- Drieghe, D., Rayner, K., & Pollatsek, A. (2008). Mislocated fixations can account for parafoveal-on-foveal effects in eye movements during reading. *Quarterly Journal of Experimental Psychology*, 61, 1239-1249.
- Lo, S., & Andrews, S. (2015). To transform or not to transform: Using generalized linear mixed models to analyse reaction time data. *Frontiers in Psychology*, 6, 1171.
- Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., & Bates, D. (2017). Balancing Type I error and power in linear mixed models. *Journal of Memory and Language*, 94, 305-315.
- McConkie, G. W., & Rayner, K. (1975). The span of the effective stimulus during a fixation in reading. *Perception & Psychophysics*, 17, 578-586.
- Rayner, K. (1975). The perceptual span and peripheral cues in reading. *Cognitive Psychology*, 7, 65-81.
- Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. *Psychological Bulletin*, 124, 372-422.
- Rayner, K. (2009). The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search. *Quarterly Journal of Experimental Psychology*, 62, 1457-1506.
- Rayner, K., Chace, K. H., Slattery, T. J., & Ashby, J. (2006). Eye movements as reflections of comprehension processes in reading. *Scientific Studies of Reading*, 10, 241-255.
- Rayner, K., & Pollatsek, A. (1989). *The psychology of reading*. Englewood Cliffs, NJ: Prentice Hall.

See `references/measure-computation-guide.md` for step-by-step computation procedures and worked examples.