Reading Opscan 6 Reports

Before You Score | Operation and Scanning | Reading Reports

Individual Test Results	Relative Frequency Distribution
Individual Item Response	Absolute Frequency Distribution
Item Analysis	Test Score Distribution

Individual Test Results

The Individual Test Results report provides a listing of students along with the number correct, percentage correct, and percentile for each student. Also included are the average and median scores, highest and lowest scores, and standard deviation.

Individual Test Results

In the Total column, the number designated as RS is the Raw Score. In the example below, 20 students took an exam of 50 questions. As the example shows, Sam correctly answered 10 of the 50 questions. Below the Raw Score is the percentage correct. Sam's percentage correct is 20% (10 correct / 50 questions). By comparing his score to the other students, we learn that Sam is in the 28th percentile -- he scored better than or equal to 28% of the students who took the exam.

Individual Test results Summary

Located at the bottom of the report is other statistical information that may be useful.

Individual Item Response

The Individual Item Response report provides a listing of students along with the number correct and an itemized list of the answer each student gave for each question. An answer key is listed at the top. A legend is listed at the bottom to interpret the symbols that signify the student's answers.

Individual Item Response Legend

The legend is broken down into Key Symbols, Respondent Symbols, and statistical information. The Key Symbols refers to the information the instructor fills in on the answer key. These symbols may also be displayed in the answer key section of the report. For example, if an instructor were to decide to accept multiple responses for question 22, he would see an asterisk below the number 22 on the answer key.

Individual Item Responses

The Respondent Symbols section of the key refers to answers provided by the students. Using the key and the report, we can see that Sam correctly answered 10 of the 50 questions. Looking at the itemized responses shows which ones he answered correctly, denoted by a period. The questions he answered incorrectly display the answer that he selected. (On the answer sheets for the Opscan6, A through E are also labeled as 1 through 5.) Question 7 has an asterisk, which indicates that Sam mistakenly filled in two bubbles for that question.

Item Analysis

The Item Analysis is helpful in determining the validity of the exam. It's results can be used to determine whether items were too difficult or too easy, whether the items discriminated between students who knew the material and students who did not, and whether the incorrect responses distract from the correct response or have no value whatsoever.

Item Analysis

On the answer sheets for the Opscan6, A through E are also labeled as 1 through 5. So in question 1, 2 (meaning B) is starred as the correct answer.

The Total Count column shows how many students selected each answer.

The Total % column is calculated as (# Students Who Chose That Answer)/(# Students Who Took The Exam). In this case, there are 20 students who took the exam. By following across from the correct answer, we can see that 45% of the students answered the question 1 correctly, and 25% of the students answered question 2 correctly.

Generally, all test questions will have a Weight of 1, meaning that each question is worth the same amount of points as the other question.

The Upper and Lower Quartile columns are used in calculating the Discrimination Index. The Upper and Lower quartiles refer to the students who scored in the top 25% overall and bottom 25% overall. The Discrimination Index gives an indication of whether the question discrimates between students who truly knew the material and those who did not. You should look at the discrimination index value directly accross from the starred correct answer. A negative value means that students receiving a low score overall tended to select the option more than the high-scoring students. Conversely, a positive value means that the higher-scoring students tended to select the response more often. A 0.0 would indicate that there was no difference between the two groups. The discrimination index is calculated as [(# Responses in Upper Quartile)-(# Responses in Lower Quartile)]/(Number of Respondents in a Quartile).

The Difficulty Factor of a question is the proportion of respondents who selected the correct answer to that question. It measures how difficult the question was to answer, and is calculated as (# Correct Responses)/(# Students Who Took The Exam). This number is the same as above, when we looked at the Total % of students who selected the starred correct answer. In question number 1, 45% of the students selected the correct answer and the Difficulty Factor is .450. These two number are calculated in the same manner, only one is displayed as a percentage while the other is displayed in its decimal form. The difficulty level can range from 0.000 to 1.000. The optimal level is 0.500. The higher the difficulty factor, the easier the question.

An item's difficulty can affect the discrimination index. Items which are very easy or very difficult will not discriminate well between high- and low-scoring groups. Nearly everyone will have gotten the item either right or wrong. Items discriminate well when they have a difficulty level between 0.300 and 0.700. If the difficulty level is within this range and a 0.0 discrimination index results, the test question should be thrown out because it does not measure the students' understanding of the material. (Question 1 above is an example.)

Item Analysis Summary

Located at the bottom of each page of the report is a key. This key contains the number of students who took the test (20), the number of students located in each quartile (25% of 20 Students), and the number of test items (50). All of these number were used in calculating the data displayed in the report.

The Kuder-Richardson index is a measure of test reliability. The formula, in effect, splits the test in half and creates 2 tests that are equivalent in content and difficulty, and then compares them to calculate the test's statistical reliability. The values calculated are an indication of the relationship/consistency between test items. (A lower value means a weaker relationship.) Values range from 0.00 to 1.00, with the better test results being in the .80 to .85 range. The Kuder-Richardson 20 and 21 are just two different methods of calculating the test reliability. The KR20 will always be greater than or equal to the KR21. In our above example, the KR20 is 0.05 and the KR21 is -0.09. This is not a reliable test.

Relative Frequency Distribution

The Relative Frequency Distribution report provides a listing by score with the number of students that achieved each score, the percentile that those students are in, and a visual layout based on the percentages.

Relative Frequency Distribution Chart

In the above example, a test made up of 10 question was given to 10 students. The Score column lists the possible scores students could have gotten. The Frequency column shows the number of students that actually got that score.

The Percentile column places the students into percentiles by comparing their scores against the scores on the other students who took the exam. All scores above the high score (9) are considered to be in the 100th percentile, but no student will ever actually be in the 100th percentile. All scores below the low score (1) are considered to be in the 0th percentile, but no student will ever be in the 0th percentile either. The student who scored a 9 out of 10 and is in the 95th percentile can be said to have scored better than or equal to 95% of the students who took the exam.

The even numbers 2 through 100 (located across the top of the right hand side of the report) are the percentages of students (based on the Frequency column). Each star represents 2% of the respondents. Since there are 10 students, each student is worth 10% (100%/10 Students). That is why a frequency of 1 yields 5 stars, a frequency of 2 yields 10 stars, etc. In each test, the number of stars that represent each student will change, but 1 star will always equal 2% of the respondents.

The percentiles are somewhat complicated to calculate. Based on our calculation at each student being worth 10%, we would expect that the student who scored 9/10 would be in the 90th percentile (100%-10% per student). However, this program uses a midpoint system, so instead of the 90th percentile, we see the 95th [(100+90)/2]. This student is in the 95th percentile. By retaining the 90 as our new top number we can calculate the percentile for the students who scored 8/10. This 1 student is also worth 10%, so the interval is now from 90% to 80% (90%-10%), making the midpoint 85% [(90+80)/2]. This student is in the 85th percentile. By retaining 80 as our new top number we can calculate the percentile for the students who scored 7/10. These 3 students are each worth 10%, for a total of 30%, so the interval is now from 80% to 50% (80%-30%), making the midpoint 65% [(80+50)/2]. These 3 students are each considered to be in the 65th percentile.

Other statistical data are located at the bottom of the report. The Average (mean) Score is calculated as (Sum of the Students Scores)/(# Students). The Median Score is found by listing all the scores out from highest to lowest and selecting the score in the exact middle. In the case of an even number of students, the median is the average of the 2 middle scores. The Standard Deviation is a measure of how far a score is located from the Average (mean) Score. A higher standard deviation indicates that students did not score close to the Average Score (some much higher and some much lower). A smaller standard deviation indicates that most students scored near the Average Score.

If you try to verify any of the calculations on your report, keep in mind that the examSYSTEM II program rounds many of the numbers it displays. When rounding occurs, it is always upwards to nearest percent (or in some cases, the nearest 2 percent).

Absolute Frequency Distribution

The Absolute Frequency Distribution report provides a listing by score with the number of students that achieved each score, the percentile that those students are in, and a visual layout of the frequency.

Absolute Frequency Distribution Chart

The number 1 through 50 (located across the top of the right hand side of the report) are the numbers of students (based on the Frequency column). Each star represents 1 student.

The percentiles are somewhat complicated to calculate. There are 10 students, so each student is worth 10% (100%/10 Students). Based on this calculation, we would expect that the student who scored 9/10 would be in the 90th percentile (100%-10% per student). However, this program uses a midpoint system, so instead of the 90th percentile, we see the 95th [(100+90)/2]. This student is in the 95th percentile. By retaining the 90 as our new top number we can calculate the percentile for the students who scored 8/10. This 1 student is also worth 10%, so the interval is now from 90% to 80% (90%-10%), making the midpoint 85% [(90+80)/2]. This student is in the 85th percentile. By retaining 80 as our new top number we can calculate the percentile for the students who scored 7/10. These 3 students are each worth 10%, for a total of 30%, so the interval is now from 80% to 50% (80%-30%), making the midpoint 65% [(80+50)/2]. These 3 students are each considered to be in the 65th percentile.

Test Score Distribution

The Test Score Distribution is a listing of the percentile, z-score, t-score, and stanine that each raw test score yields.

Test Score Distribution Chart

The Z-score measures of how many standard deviations a raw score is from the average (mean) score. Raw scores above the average score will yeild a positive Z-score, while those below the average score will yeild a negative Z-score. The Z-score is calculated as [(Raw Score)-(Average Score)]/(Standard Deviation).

The T-score is similar to the Z-score in that it measures how far a raw test score is from the average test score. However, the T-score is a more expanded measure. The standard deviation from the mean is expanded by a factor of 10 and then added to 50. Thus, above-average scorers will have T-scores above 50, and below-average scorers will have T-scores below 50. The T-score is calculated as [(Z-Score)x10]+50.

Stanines Chart

The Stanine column is based on a rating system that looks at how many standard deviations a student's score is from the mean, and then places the student into one of the above categories.