Senior Director, Assessment

This is a fully remote position, open to applicants in United States.

📋 Description

• Direct the strategy, planning, execution, assessment, and ongoing enhancement of existing and future assessment products and services.

• Align assessment objectives with organizational aims and member requirements.

• Keep abreast of advancements in educational measurement, psychometrics, assessment technology, artificial intelligence, and health professions education.

• Collaborate with organizational leaders on assessment projects, business planning, and product innovation.

• Manage the entire assessment lifecycle, which includes blueprint creation, item banking, test construction, administration, scoring, scaling, equating, standard setting, reporting, quality assurance, and technical documentation.

• Ensure assessments maintain validity, reliability, fairness, accessibility, security, legal defensibility, and compliance with established standards and policies.

• Lead psychometric analyses and measurement research efforts.

• Guide the development and utilization of data tools, analytical solutions, and measurement technologies.

• Communicate technical findings, score interpretation guidance, and assessment information to both technical and non-technical audiences.

• Supervise, coach, and mentor Assessment staff; manage performance and promote cross-functional alignment.

• Support staff development, professional engagement, knowledge sharing, cross-training, and succession planning.

• Oversee exam development, production timelines, budgets, resource allocation, volunteer engagement, and vendor performance.

• Coordinate assessment staff, subject matter experts, vendors, and external collaborators.

• Implement process enhancements that improve quality, efficiency, scalability, sustainability, and stakeholder experience.

• Represent PAEA through presentations, publications, workshops, consultations, and professional services.

• Enhance members’ assessment literacy through educational programs, consultations, score interpretation resources, and dissemination of best practices.

• Safeguard the confidentiality, integrity, and security of assessment materials, data, and processes.

• Coordinate incident response, risk management, and compliance efforts.

⛳️ Requirements

• Doctoral degree (PhD, EdD, or equivalent) in Psychometrics, Educational Measurement, Quantitative Psychology, Educational Psychology, Statistics, Research Methodology, or a related field.

• At least 8 years of progressively responsible experience in assessment, psychometrics, educational measurement, testing, or related disciplines.

• Minimum of 3 years of supervisory or leadership experience overseeing professional staff and complex projects.

• Extensive knowledge of Classical Test Theory, Item Response Theory, Rasch measurement, standard setting, test development, validation, scoring, equating, and assessment reporting.

• Proven experience in leading assessment programs, initiatives, or portfolios.

• Experience in managing external vendors, consultants, and stakeholder relationships.

• Strong strategic planning, organizational leadership, and project management capabilities.

• Exceptional written and verbal communication skills, with the ability to convey technical concepts to non-technical audiences.

• Capacity to manage multiple priorities and spearhead complex cross-functional initiatives.

• Strong analytical, problem-solving, and decision-making abilities.

• Demonstrated commitment to assessment quality, fairness, accessibility, and ongoing improvement.

• Must be authorized to work in the United States.

• Experience in health professions education and assessment is preferred.

• Preference for experience in developing innovative assessment products, technologies, or services.

• A record of professional presentations, publications, or contributions to the assessment and measurement field is preferred.

• Experience in managing assessment security programs and policies is preferred.

• Familiarity with R, data visualization tools, assessment platforms, and modern analytical technologies is preferred.

• Active participation in professional measurement or testing organizations is preferred.

• Must be able to satisfactorily perform essential duties with reasonable accommodation.

🏝️ Benefits

• Health insurance

• Life insurance

• Disability insurance

• Paid time off

• Paid holidays

• 403(b) retirement plan with an 8% employer contribution

• Monthly wellness stipend

• Professional development funding

• Tuition assistance

• Occasional travel to all-staff meetings, PAEA events, professional development opportunities, and other necessary meetings