Does PIAAC Make Accurate Assumptions About Future Skill Needs?

ETS released a new report used the PIAAC data to compare U.S. millennials with millennials in other wealthy countries. Coverage of the study in The Atlantic included an interesting overall critique of PIAAC from Tom Loveless of The Brookings Institution:

…Loveless, an education scholar with The Brookings Institution, said that while the PIAAC results aren’t surprising, the maker of the assessment “is unabashed about its ambitions in this regard … [it] believes it’s measuring skills that matter in the 21st century. Put me in the ‘I’m skeptical of that claim’ group.”

…[He] says that, empirically speaking, the PIAAC results don’t match up with existing data about the U.S. economy’s performance. But a bigger qualm Loveless has is the assumptions that the adult-competencies assessment makes about the skills workers will need going forward. “Let’s say I was alive in 1915 and I gave a test that predicted the job skills and future economic productivity of nations,” he said. “I just don’t see how anyone in 1915 could have foreseen the skills that would have been important for the rest of the 20th century, and I doubt that anyone’s doing that now for the 21st century.”


Earlier this week, a group of civil rights groups and education advocates released a set of shared principles for the reauthorization of the Elementary and Secondary Education Act (ESEA), and I would draw your attention principle number two, just as an FYI:

States and districts [should] ensure that all Title I schools encourage and promote meaningful engagement and input of all parents/guardians – regardless of their participation or influence in school board elections – including those who are not proficient in English, or who have disabilities or limited education/literacy – in their children’s education and in school activities and decision-making. Schools communicate and provide information and data in ways that are accessible to all parents (e.g., written, oral, translated). (my emphasis)

An ESEA reauthorization bill is expected to be unveiled in the Senate this week.

Rebranding Family Literacy

Two articles in the National Journal last week provide further hope that the rebranding/refashioning of family literacy, a once powerful and influential approach to adult education that fell out of fashion for some reason over the last decade or so, is gaining traction.

The first piece, by Fawn Johnson, examines the “two-generation” approach to literacy practiced by the Briya Public Charter School here in Washington, D.C., a program that provides a preschool for 3- and 4-year-olds while their parents are offered English-language instruction, basic computer skills, and parenting classes—as well as skills that will help them nurture their child’s developing literacy skills. What’s interesting, though not mentioned specifically in the piece, is that Briya’s roots are as a federally funded Even Start program, a federal initiative that not so long ago provided upwards of 150,000 families (that’s from memory—I can’t locate the exact figures at the moment) with such services in programs across the country. This “two-gen” approach certainly had fallen out of favor among federal policymakers by the end of the last decade, as Even Start funding was cut several times by Congress, continually proposed for elimination by President Obama in his budgets, and finally eliminated for good in 2012.

Thankfully, Mary’s Center, which started the program, developed strategies that made their program less dependent on federal funds, beginning with going after charter school funding back in 2006 and forming the Even Start/ESF Public Charter School, which later evolved into Briya. While it’s good news that Briya survived and even grew despite the cut, many (most?) Even Start programs have shut their doors. If the family-focused approach is indeed back in fashion, it’s important to understand that we let a lot of other programs like Briya wither and die over the last several years. Maybe its time to work together to revive federal support for such initiatives.

The second article, by Alana Semuels of The Atlantic, examines the two-generation model championed by the Annie E. Casey Foundation, focusing on a program in Mechanicsville, Georgia where they are piloting this approach. Here again, as Semuels’ points out, the general concept is not new (the paper cited twice in the article is actually from 1995), and this model appears to focus less on education services for the parents as much on other services and assistance. But the basic idea appears similar to the Briya approach.

The term “family literacy” started to go out of fashion around the same time that Even Start was under attack (the National Center for Family Literacy actually changed their name to the National Center for Families Learning a few years ago). Sadly, the phrases “family literacy” and “Even Start” don’t appear once in either article. But I doubt there are many substantial differences between the approach they are taking at Briya today and the approach they took when it was an Even Start program.

Whatever it’s called, and however much of the approach is actually “new,” the embrace (or re-embrace)  of family-focused, dual adult/child literacy approaches by policy and media influencers is long overdue. It’s especially heartening to read a quote like this one from Anne Mosle, the executive director of Ascend, at the Aspen Institute, in the Johnson piece: “For all the strides we’ve made in investing in early education, we can’t put all of the weight on the back of the child.”

If you scroll back through this blog you’ll find no shortage of posts lamenting what appeared to me to be a frustrating lack of understanding—most notably, in the pre-K movement of recent years—of the critical role of parents in childhood literacy development (I even wrote an op-ed about it, many, many years ago), and this renewed interest in linking the two again may present some interesting new opportunities to advance the adult education cause in the coming years. (Although there is still a lot of work to do to connect the dots—neither article conveys any sense that an adult education system actually exists, let alone the role it has played and continues to play in these efforts.)

Are We Too Quick to Accept PIAAC Findings at Face Value?

Ralf St. Clair comments on last week’s PIAAC research conference:

At this meeting, lots of findings were discussed, but very little time was spent on methodology. The papers written by presenters were not available in advance (and mostly not at the meeting). One of the problems with PIAAC data is that it is not complete…

In many cases such data gaps are tackled through synthetic data, where the existing data is used to estimate what the missing data should be. One of the problems with this, of course, is that the missing data is essentially assumed to fit with what we have, and unexpected results will never arise.

Without understanding the details of how these types are issues are tackled it is difficult to assess the implications of some of the correlations found, which are often quite weak. Would they exist at all if we had the missing data? Would they run in different directions? What sorts of assumptions are being made throughout the research process that generates the results?

Yet throughout the meeting the findings were accepted at face value and the issues of the data set never fully discussed, even though it was a room full of people who could understand and even work out how to deal with them. As in so much of the activity that surrounds international surveys, the will to believe overwhelms the skepticism we must bring to these exercises. (my emphasis)

I would just add that critical scrutiny is particularly important with PIAAC since it appears that the adult education field (in the U.S. at least, can’t speak for other countries) has decided to embrace PIAAC as our primary foundational data source for policy decisions going forward.

I recommend reading the entire post.