Does PIAAC Make Accurate Assumptions About Future Skill Needs?

ETS released a new report used the PIAAC data to compare U.S. millennials with millennials in other wealthy countries. Coverage of the study in The Atlantic included an interesting overall critique of PIAAC from Tom Loveless of The Brookings Institution:

…Loveless, an education scholar with The Brookings Institution, said that while the PIAAC results aren’t surprising, the maker of the assessment “is unabashed about its ambitions in this regard … [it] believes it’s measuring skills that matter in the 21st century. Put me in the ‘I’m skeptical of that claim’ group.”

…[He] says that, empirically speaking, the PIAAC results don’t match up with existing data about the U.S. economy’s performance. But a bigger qualm Loveless has is the assumptions that the adult-competencies assessment makes about the skills workers will need going forward. “Let’s say I was alive in 1915 and I gave a test that predicted the job skills and future economic productivity of nations,” he said. “I just don’t see how anyone in 1915 could have foreseen the skills that would have been important for the rest of the 20th century, and I doubt that anyone’s doing that now for the 21st century.”

No One Could Have Predicted…

More discouraging news, this time from WRVO in New York:

New York replaced the GED because the test’s price tag was set to double this year. The new test gives students the same credentials – the equivalent of a high school diploma. Statewide pass rates are down by four percent after the switch, compared to 2012. The number of test-takers also fell by half.

First of all, that is a classic example of a buried lead, in paragraph form. I view a 4% pass rate drop as actually pretty good news, considering that they’ve switched to a completely different exam. The real news is that the number of test-takers has fallen by half. That is not an “also”—that is the story.

And to what do we attribute this dramatic decrease in the number of test-takers? In states that stuck with the GED, the drop off is attributed by critics to the higher cost and difficulty of the revised GED. In New York, they have a different problem:

Bruce Carmel, director at the Bronx Youth Center and a co-chair at the New York City Coalition for Adult Literacy, says there was a lot of misinformation about the new test.“You heard some people saying, ‘Oh, there’s no more GED,” he says. “So when people heard there was no more GED test, a lot of people thought it was over and you couldn’t get your high school equivalency diploma anymore.”

Sometimes it seems like if we had tried to come up with a plan to intentionally discourage adults from earning their high school equivalency we could not have done a better job. I know this is not in fact what anyone intended (including the test publishers), and I don’t want to the discount the heroic work that the adult education community has done to transition to these new exams. We’d be in even worse shape without their efforts. (And I wish the media would pay as much attention to that as they do to the poor numbers.) Nor is this a criticism of the decision in New York to replace the GED. But the dramatic drop we are experiencing around the country in the number of people seeking a high school equivalency diploma as a result of these changes should not have been unexpected. Some honest, no-finger-pointing reflection on how we ended up in such a situation might help us take steps to ensure that it doesn’t happen again. And while eventually this all may eventually shake out for the better, let’s not forget about the folks who gave up on high school equivalency during this transition, and what the future likely holds for them.

Michael Bloomberg: “Education Isn’t Going to Help Uneducated Adults”

Let’s hope this is not representative of the level of discourse one can expect at the Aspen Institute:

Moderator Jennifer Bradley, director of the Center for Urban Innovation at the Institute, then asked what the U.S. can do to get people out of poverty. Bloomberg responded that conventional wisdom points to education, but education isn’t going to help uneducated adults. Bradley later asked how government can offer basic fairness to the children “who have been failed.” (my emphasis)

Bloomberg claimed that 95 percent of murders fall into a specific category: male, minority and between the ages of 15 and 25. Cities need to get guns out of this group’s hands and keep them alive, he said.

As noted by Paul Campos, here, according to actual crime statistics, the real percentage appears to be 22.8.

Maybe this is an isolated case, but I worry that attitudes and ignorance like this, when espoused by elites, can have a significant adverse impact on efforts to improve adult educational opportunities for minority out-of-school youth and young adults. Curious what others think.

This Shouldn’t Be So Hard to Figure Out

Is it reasonable to still be making up our minds as to whether the new “tougher” GED makes good policy sense? This sentence from a recent Dow Jones Business News article on the dramatic drop in the number of GED test-takers got my attention:

Economists and policy makers are torn over whether a tougher test is good or bad.

You know who is likely not torn about it? The kid who can’t get a minimum wage job because he doesn’t have a diploma, and who can’t put the time in to pass the new, more difficult GED. Pretty sure they are going to go with “bad.” Randy Trask, the president of the GED Testing Service, says in the same article that the GED “was becoming irrelevant” before the changes, but for people without a high school diploma, it was and remains very relevant. I realize a high school diploma is not enough for most of the jobs that pay a decent wage, and that we want to encourage people to get into career pathway programs and integrated training and into college etc., but I’ll never understand why it would ever make sense as a matter of public policy to make high-school equivalency any harder than it needs to be.

“Some test takers may have the simple need to work at Starbucks, they don’t need to analyze a Shakespeare play,” said Larry Condelli of the Workforce and Lifelong Learning program at the not-for-profit American Institutes for Research. “Then again, if you give them a lesser education for a specific purpose, are you really helping them?”

This is not an unreasonable question. But at this point, it might be time to take the debate outside of the usual policy circles and ask actual test-takers and potential test takers what they think. If our adult education system was truly learner-centered, not only in terms of instruction, but policy as well—that is, if we had a system in which adult learners had a major voice in policy discussions and decisions—this would be a much easier question to answer. Lacking that, a survey of the target population might help us figure it out.

At the end of the day, this should not be a hard one to call. If, on balance, the new test ends up being more of a barrier to the people it’s designed to serve than an opportunity, it should and probably will be considered a failure. Arguing for more career pathways or integrated models or whether adult education works at all actually obfuscates what should be a fairly simple question, which is whether the test has or hasn’t created an unfair and unreasonable barrier to adults and out-of-school youth seeking high school equivalency. (It’s useful to remember, by the way, that many people who take the GED—maybe the majority—aren’t enrolled in or have any contact with the adult education system at all.)

Adult-learning instructor Marcia Leister has felt the impact of the new GED test at her technical college in Bellingham, Wash., a state where currently only the GED is offered. Of about 120 students she taught last year, about 10 people took the test, about a quarter of the number in a typical year, and only one person passed it, she said.

“My students are extremely frustrated by the new test,” she said. “They are losing hope.”

I think sometimes policy people (me included, when I’m wearing that hat) forget what it actually looks like on the ground in this business, and end up missing the obvious. We can disagree about a lot of things in adult education policy, but I don’t think any of us want to be in the losing hope business.