Psychometrician

This is a fully remote position, open to applicants in Wisconsin.

📋 Description

• Assist in the operational and research activities related to WIDA’s language assessments in the field of psychometrics.

• Oversee and assess psychometric projects and analyses carried out by external vendors.

• Create and evaluate research reports and psychometric outputs.

• Aid in the psychometric operations and ongoing enhancement of the MODEL assessment program, including initiatives for automated scoring.

• Perform and assess test security analyses for ACCESS for ELLs.

• Assist in the maintenance of the item bank and quality assurance processes.

• Ensure psychometric quality control for the ACCESS and Alternate ACCESS assessment programs.

• Contribute to projects that support the validity, reliability, and fairness of WIDA assessments.

• Analyze, investigate, and suggest solutions related to psychometric data and issues.

• Communicate findings and solutions to both internal and external stakeholders.

• Design and implement psychometric tools aimed at assessing learner abilities and improving learning outcomes.

• Share technical psychometric concepts with both technical and non-technical audiences.

• Observe and evaluate learner performance, making adjustments to procedures as required.

• Administer psychological tests.

• Report directly to the Psychometrics Manager.

⛳️ Requirements

• At least 1 year of experience in psychometric theory, research, and statistics.

• Knowledge of classical test theory, item response theory, scaling and equating, sampling, generalizability theory, linear modeling, or structural equation modeling.

• Proficiency in data manipulation and statistical/psychometric analyses using software such as Winsteps, SAS, R, or similar tools.

• Familiarity with multistage adaptive testing, the Rasch model, and graded response IRT models.

• Capability to write technical and research reports.

• Ability to work both independently and collaboratively as part of a team.

• Experience in collaborating with external vendors or partners.

• Preferred: programming skills, especially in Python or comparable programming languages.

• Preferred: experience with applying AI methods to assessment and measurement, including NLP, automated scoring, or machine learning.

• Preferred: experience or training in test security-related analysis.

• Preferred: familiarity with K–12 educational assessments, language proficiency assessments, or large-scale standardized testing programs.

• Ph.D. in Educational Measurement or a related discipline, focusing on psychometrics or quantitative methods.

• A cover letter and a current resume/CV are required for the application.

• Employment may require a criminal background check.

• UW-Madison is not eligible to hire F1 STEM OPT Extension participants.

🏝️ Benefits

• Generous vacation, holidays, and paid time off.

• Competitive insurance and savings accounts.

• Retirement benefits.

• Comprehensive benefits package.

• Affordable health, dental, vision, and life insurance options.

• Tax-advantaged savings accounts.

• Participation in the Wisconsin Retirement System (WRS) pension fund.

• Option for voluntary remote work.

• Dell PC laptop provided for use in the role.

• Support for business travel and efforts to minimize or eliminate out-of-pocket expenses wherever feasible.