NLP Word Dictionaries Predict Selection Outcomes Across 67,652 People
Image: "Woman Conducting an Interview" by amtec_photos (CC BY-SA 2.0) via Wikimedia Commons, recoloured.
Can simple lists of words score narrative application material well enough to support selection decisions? Emily and Michael Campion (Journal of Applied Psychology, 2026) built 17 natural language processing dictionaries and tested them in 12 samples (N = 67,652), finding consistent prediction, gains beyond ability and personality tests, and small gender and racioethnic differences.
Transparent text scoring for selection research
Much of the text that organizations collect during selection, such as interview answers, application essays and descriptions of past jobs, is read once and then discarded as data. Emily D. Campion (University of Iowa) and Michael A. Campion (Purdue University) ask whether that material can be scored automatically with one of the most transparent tools in natural language processing: word dictionaries, which are curated lists of words and phrases that signal a construct. Their article in the Journal of Applied Psychology focuses on passive scoring, meaning the text was gathered for other purposes and is scored after the fact.
The authors developed 17 dictionaries covering competency-, personality- and motivation-based language. They then applied them to narrative application information in 12 samples drawn from six studies, with a total of 67,652 people. The samples span employment and school settings, experimental and operational contexts, concurrent and predictive designs, and a range of occupations, which lets the paper test whether the dictionaries travel across contexts rather than fitting a single data set.
According to Campion and Campion (Journal of Applied Psychology, 2026), the dictionaries predicted selection ratings, selection decisions and later performance consistently. Uncorrected grand averages across dictionaries and studies were in the low .20s for selection ratings and closer to .10 for performance criteria. The scores also added incremental prediction beyond mental ability and personality tests, captured constructs such as motivation and personality without relying on self-reports, and showed small gender and racioethnic subgroup differences.
Readers should keep the effect sizes in proportion. Correlations near .10 with performance are modest, and the authors present dictionaries as a complement that adds to existing measures rather than a replacement for them. The figures reported are uncorrected averages, and because the material is passively scored, its value depends on what candidates happened to write for another purpose. The authors' case for the method rests less on raw accuracy than on properties that opaque machine learning models often lack: dictionaries are explainable, straightforward to content validate and usable by researchers with typical statistical skills.
Why it matters: For researchers in management, organizational psychology and marketing, the study is a reminder that the most useful text analysis tool is not always the largest model. A validated, inspectable word list can turn existing archives of essays, open-ended survey answers or interview transcripts into measures of hard-to-capture constructs, with a clear audit trail for reviewers and ethics boards. The paper also speaks to the long-standing validity and adverse impact trade-off in selection, suggesting that transparent text scoring can add predictive information while keeping subgroup differences small, a claim other teams can now test in their own samples.
Campion, E. D., & Campion, M. A. (2026). Natural language processing word dictionaries for personnel selection: Achieving explainable, generalizable, and incremental prediction with low impact when passively scoring narrative application information. *Journal of Applied Psychology*. Advance online publication. https://doi.org/10.1037/apl0001438. Independent editorial summary of third-party research.
Comments