Company Description
***Please submit Resume and Cover Letter when applying***
The Center for Applied Linguistics is a nonprofit organization promoting access, equity and mutual understanding for linguistically and culturally diverse people around the world.The mission of the Center for Applied Linguistics (CAL) is to promote language learning and cultural understanding by serving as a trusted source for research, resources, and policy analysis. Through its work, CAL seeks solutions to issues involving language and culture as they relate to access and equity in education and society around the globe.
CAL is located in Washington, D.C. For this position, the employee may choose to work in their preferred combination of in person at the D.C. office in a hoteling capacity, or remotely (up to 100% remote). CAL does not compensate or reimburse candidates for relocation expenses. This position is not open to candidates residing in New York, New Jersey, Pennsylvania, Texas, California, or the U.S. Territories.
Job Description
The Associate Manager, Rater Training plays a crucial role in managing the creation and refinement of rater training and scoring materials for large-scale English language assessments. This position requires a dynamic individual with a strong background in educational assessment and project management, and a commitment to improving educational outcomes for K–12 English learners.The Associate Manager, Rater Training will work closely with content experts and project stakeholders to ensure that high-quality training materials (e.g., anchor responses, training responses, score explanations, scoring guidance) are created following best practices, ensuring the reliable, consistent scoring of writing and speaking tasks on high stakes assessments.This role involves balancing strategic oversight and planning with detail-oriented review and editing, as well as contributing to the continuous improvement of work processes to make a meaningful impact on the educational experiences of multilingual learners.
General
- Create and update procedures for developing rater training materials
- Create and update general rater training presentations
- Monitor and update team access to scoring resources
- Participate in special projects related to the training of test raters and scoring of constructed response tasks
- Collaborate with test development managers to document observations about task performance, including synthesizing rater feedback on tasks and communicating key takeaways to the larger team
Field Test Scoring
- Select anchor responses and create training response sets using response lists prepared by test development staff
- Review, revise, and approve written score justifications
- Review and revise task-specific scoring guidance, in collaboration with test development managers
- Ensure scoring rubrics and principles are applied consistently across all tasks and grade level clusters
- Ensure anchor response criteria and principles for score justifications are adhered to
- Identify potential scoring challenges and collaborate with test development managers on resolutions
- Monitor field test response availability and communicate updates with team members
Operational Scoring
- Select additional responses to fill gaps in rater training materials for newly operational tasks, including drafting written score justifications as needed
- Review responses selected by the scoring vendor to confirm score accuracy
- Monitor rater agreement rates and score distributions
- Review score changes proposed by the scoring vendor to confirm score accuracy
- Collaborate with test development managers to respond to client and scoring vendor questions
Qualifications
Education and Experience
The Associate Manager, Rater Training must have
- A Bachelor’s degree from an accredited college or university, in a relevant field (e.g., Applied Linguistics, Educational Assessment, TESOL), with at least four years of relevant experience; or a Master’s degree from an accredited college or university, in a relevant field, with at least two years of relevant experience
- Experience scoring written and/or spoken responses produced by K–12 students
The ideal candidate for this role has
- Experience scoring written and/or spoken responses produced by second language learners
- Experience working with K–12 multilingual learners
- Experience teaching English to speakers of other languages
Knowledge, Skills, and Abilities
The Associate Manager, Rater Training must be able to
- Work independently with minimal support/supervision
- Prioritize tasks appropriately, meet deadlines, and work well within a team
- Proactively identify potential challenges and propose solutions
- Closely attend to details without losing sight of bigger picture
- Effectively coordinate and oversee others’ work in a remote environment
The ideal candidate for this role has
- Strong oral and interpersonal communication skills
- Excellent organization, planning, and time management skills
- Excellent writing, editing, and proofreading skills
- Excellent grasp of second language development
- Knowledge of a language other than English
- Proficiency with project management tools (e.g., Microsoft Project, Smartsheet)
Travel Requirements
- This position requires occasional travel (1-2 times per year).
Additional Information
COMPENSATION:
Minimum Salary: 67,800.00; Benefits eligible
PHYSICAL DEMANDS
The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. While performing the duties of this job, the employee is regularly required to effectively communicate. The employee may be required to sit or stand in a stationary position for 50% of the time. This position constantly operates a computer and other office productivity equipment. CAL adheres to ADA compliance and guidelines as a common practice.
ADDITIONAL
INFORMATION
This job description summarizes the main duties of the job. It neither prescribes nor restricts the exact tasks that may be assigned to carry out these duties. This document should not be construed in any way to represent a contract of employment. Management reserves the right to review and revise this document at any time.
AFFIRMATIVE
ACTION/EQUAL
OPPORTUNITY
EMPLOYMENT
STATEMENT
The Center for Applied Linguistics (CAL) provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity, disability, or genetics. In addition to federal law requirements, CAL complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.CAL expressly prohibits any form of workplace harassment based on race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status.
Improper interference with the ability of CAL employees to perform their job duties may result in discipline up to and including discharge.
