PIAAC

(Very) Long Awaited “Making Skills Everyone’s Business” Report Is Finally Released

/ The Old Crank

making-skills-coverThe Office of Career, Technical, and Adult Education (OCTAE) at the U.S. Department of Education has released a new report, Making Skills Everyone’s Business: A Call to Transform Adult Learning in the United States. The report, originally slated to be released in April of 2014, offers seven strategies, “grounded in evidence and informed by effective and emerging practices,” that hold promise for “improving the conditions that create and perpetuate poor literacy, numeracy, and problem solving.”

OCTAE has also produced a recorded video announcement about the report from Acting Assistant Secretary Johan E. Uvin.

Does PIAAC Make Accurate Assumptions About Future Skill Needs?

/ The Old Crank

ETS released a new report used the PIAAC data to compare U.S. millennials with millennials in other wealthy countries. Coverage of the study in The Atlantic included an interesting overall critique of PIAAC from Tom Loveless of The Brookings Institution:

…Loveless, an education scholar with The Brookings Institution, said that while the PIAAC results aren’t surprising, the maker of the assessment “is unabashed about its ambitions in this regard … [it] believes it’s measuring skills that matter in the 21st century. Put me in the ‘I’m skeptical of that claim’ group.”

…[He] says that, empirically speaking, the PIAAC results don’t match up with existing data about the U.S. economy’s performance. But a bigger qualm Loveless has is the assumptions that the adult-competencies assessment makes about the skills workers will need going forward. “Let’s say I was alive in 1915 and I gave a test that predicted the job skills and future economic productivity of nations,” he said. “I just don’t see how anyone in 1915 could have foreseen the skills that would have been important for the rest of the 20th century, and I doubt that anyone’s doing that now for the 21st century.”

Are We Too Quick to Accept PIAAC Findings at Face Value?

/ The Old Crank

Ralf St. Clair comments on last week’s PIAAC research conference:

At this meeting, lots of findings were discussed, but very little time was spent on methodology. The papers written by presenters were not available in advance (and mostly not at the meeting). One of the problems with PIAAC data is that it is not complete…

In many cases such data gaps are tackled through synthetic data, where the existing data is used to estimate what the missing data should be. One of the problems with this, of course, is that the missing data is essentially assumed to fit with what we have, and unexpected results will never arise.

Without understanding the details of how these types are issues are tackled it is difficult to assess the implications of some of the correlations found, which are often quite weak. Would they exist at all if we had the missing data? Would they run in different directions? What sorts of assumptions are being made throughout the research process that generates the results?

Yet throughout the meeting the findings were accepted at face value and the issues of the data set never fully discussed, even though it was a room full of people who could understand and even work out how to deal with them. As in so much of the activity that surrounds international surveys, the will to believe overwhelms the skepticism we must bring to these exercises. (my emphasis)

I would just add that critical scrutiny is particularly important with PIAAC since it appears that the adult education field (in the U.S. at least, can’t speak for other countries) has decided to embrace PIAAC as our primary foundational data source for policy decisions going forward.

I recommend reading the entire post.

Long-Awaited PIAAC Response Report Resurfaces

/ The Old Crank / 1 Comment

In case you missed it, this morning the White House announced the latest round of TAACCCT grantees. If you read the entire release, you may have noticed this curious related announcement:

In addition, the Department of Education is releasing a new report on the importance of building foundational skills in a job-specific context.

Ø  Department of Education Report on Transform Adult Learning through Work. The Department of Education is releasing a new report with recommendations to transform adult learning in the United States. After months of public engagement with a variety of stakeholders around the country, the recommendations for public-private partnership include strategies that engage employers to support upskilling of more entry-level workers while on the job, encourage the use of assessments and innovative learning tools to improve access to targeted career guidance for youth and adults, and promote better alignment and coordination of public and private programs so that youth and adults experience seamless services. The report highlights unique opportunities for implementing these recommendations as a result of the changed legislative environment made possible by the passage of the Workforce Innovation and Opportunity Act in July 2014.

I confirmed today with Department staff that the report referenced above is the long-awaited PIAAC response report or action plan that was announced last fall and then previewed at last spring’s adult education state directors meeting  However, contrary to the White House statement, the report was not in fact released today. It is finished but still in the “awaiting clearance” stage. No one could give me a date for actual release but the expectation is that it will be out sometime this fall. I know a lot of us in the field were wondering about the status of this report—some were speculating it might not ever be released—so I thought it was worth mentioning. It lives!