Attachment III
EMPLOYMENT TEST VALIDATION PROCESS
The Employment Test
Validation Process is accomplished through the use of a Rating Committee. The
Rating Committee’s role is to determine if a pre-employment test being used as a job selection criteria is a valid measure of the
knowledge, skills, or abilities (KSAs) it is being
used to measure without creating an adverse impact against any protected group
in the candidate population.
Rating Committees are
composed of five to ten subject matter experts experienced and familiar with
the job for which a pre-employment test is to be validated. They will follow
the Test Validation Steps noted below and document their meetings and findings.
Test Validation Steps
1. Determine the KSAs – Determine the KSAs
to be measured by a written or performance test that are needed on the
employee’s first day of employment, necessary and important to job performance,
and linked to one or more important or frequently performed job duties.
2. Develop a Test Plan for Measuring the Selected KSAs – Includes establishing the general components of the
test and choosing the number and type of test items.
a. General Components of a Test Plan – Should address
the purpose of the test and the mastery level to be evaluated, manner of
scoring such as requiring a passing score on each area of the test or allowing
compensatory scoring which allows for a high score in one area of the test to
compensate for a low score in another area, the target population being tested,
manner of test measurement such as speed or power which allows for completion
by the majority of test takers, reading level and delivery mode.
b. Choosing the Number of Test Items – Should address
ensuring the appropriate KSAs are being measured,
that there is a proportional sampling of these KSAs,
and a sufficient number of items and/or a sufficient sampling the KSAs are included to generate high test reliability.
c. Choosing the Type of Test Items – Should address
whether the items to be included measure the difficult, complex, and evaluative
aspects of the knowledge, or just simple facts and definitions. KSAs must be measured in a relevant way using items that
are appropriately geared to the level of the KSA that is required on the job.
The format of written test questions should be addressed and may include
multiple choice, true or false, open-ended, or essay type. The format of the
job performance or work sample items should be as similar to performance on the
job as possible.
3. Developing Test Content – Develop the test content
and thoroughly review the test plan before beginning work.
4. Validate the Test – A second group of subject matter
experts should convene to review the contents of the test. They should consider
if test questions read well, if there is sufficient information for a test-taker
to be able to answer the item correctly, if the question is free from
unnecessary complexity, if the question is linked to an important job duty,
what percent of minimally qualified applicants would they expect to answer the
item correctly, and how serious of a consequence will occur if a person
performing the target job did not possess the knowledge to answer the question
or perform the related task correctly.
5. Score and Analyze the Test – Administer the test to
the test population. Human Resources will analyze the test results, providing
item-level and test-level analysis, and determine if the test can be declared
valid.
a. Item-level Analysis – Is conducted by correlating the
scores on the test item to the total scores for each applicant on the overall
test. Items that possess inappropriate differential functioning for a protected
group of applicants should be removed before the final scores are calculated.
b. Test-level Analysis – Involves calculating the mean
score or mathematical average and standard deviation associated with test
scores.
6. If the test has been validated, the final test score
for each applicant should be reflected in a field on the hiring matrix and
incorporated into the overall assessment of each applicant.