Here is what the press release from the Pasadena Unified School District (PUSD) claims regarding the release of test scores by the state’s assessment system:
“Student achievement continued to rise in the PUSD, with John Muir High School registering growth, according to the 2011 Standardized Testing and Reporting (STAR) system.” (“PUSD Test Results Released – Four Years After Restructuring Muir High School Makes Gains”). http://pasadenausd.org/modules/cms/pages.phtml?pageid=241195&sessionid=81f139b19117bbcdac349955954e2024
But here is what the percentage of increase is as reported in the PUSD’s own data:
|
Test |
District Wide One-Year Change 2010-11 |
District Wide Five-Year Change 2007-2011 |
|
Percent of students at or above proficiency English Language Arts (ELA) including California Standards Test (CST) |
0% |
11% |
|
Rank student at or above proficiency ELA, CST and California Math Achievement (CMA) |
0% |
11% |
|
Percent of students at or above proficiency ELA, CST, CMA by 1-Yesr Change |
0% |
11% |
|
Percent of students at or above proficiency ELA, CST, CMA by 5-Yesr Change |
0% |
11% |
|
Percent of students at or above proficiency CMA, CST - MATH |
1% |
10% |
|
Math Summary Percent students proficient |
1% |
10% |
|
Math Summary Percent student proficient MATH 1-Year Change |
1% |
10% |
|
Math Summary Percent student proficient MATH 5-Year Change |
1% |
10% |
Source:
No doubt teachers, students, and school administrators have worked hard at PUSD to raise test scores. Over the last five years district wide test scores have risen 10% to 11%. But the past year test scores are up 0% to 1%. In other words, test scores have flat-lined. But that’s not what the above-referenced press release reports.
Muir High School Language Arts scores have reportedly increased 4% last year and 8% in the last five years, but only 31% of Muir students are language proficient and 69% are not proficient. By focusing on the percent of increase in test scores (8% -5 years) instead of the total percent proficient (31%) we see the tree but miss seeing the forest. Despite everyone’s hard work, there are no large gains at Muir and the rate of increase in scores is declining.
Averages, however, can be misleading if the statistical variance is not also reported as would be ethically required by an impartial statistician. The percentage of test scores may indeed increase but if, say, 5,000 students left the school district not by graduation, it may be that the increase just reflects that those who scored lower on tests left the district. A number of other explanations could explain test scores increasing other than by better performance. So test scores at PUSD went up 10% to 11% over 5-years doesn’t tell us very much.
The same could be said for PUSD’s claim that John Muir High School test scores “registered growth.” According to PUSD language arts scores rose 8% and math scores 7% over the past 5-years, or 2% per year. But test scores increased only 1% last year indicating a 50% drop in the rate of increase. With a typical statistical error rate of 5%, the 1% gain would wash out to zero.
As a good friend of mine put it when I asked him what he thought about PUSD's test scores he said "If you have a penny and I give you another penny, the rate of increase is 100%. But you still have two pennies. That is why PUSD's reporting of school test scores is not worth two cents.
I’m sure there are many individual stories of success at PUSD. But here is what should have been reported: 50% of students are not language proficient and 54% are not math proficient.
It is one thing to boast about gains in math and science in the classroom. But when it comes to reporting public school test score data statistical standards are abandoned. Standards and ethics are only for government to impose on the commercial sector and not to be imposed on government.
PUSD’s advocates will no doubt say that this writer has no moral credibility in reporting that in reality there have been no gains in test scores district wide unless I have volunteered as a tutor or provided hundreds of hours of volunteer service to the school district. This is a dodge from the reality at hand. This writer takes no delight in reporting that PUSD’s test scores are what they are – flat for the last year and modest over 5 years – and adjusted for a standard error of 5%, may indicate no gains at all.
American culture likes to report success whether one is a businessman, religious evangelist or school administrator. But reality is much harder and much more intractable. Without ethical standards for reporting school test scores, we can’t put much credibility in them. Is PUSD lying about test scores? No. They are just putting a positive spin on them.
Comments