Like most things, the pandemic has disrupted standardized testing of students in Canada and beyond. But it must resume as soon as possible—and preferably in a revamped and modernized way.

The Biden administration recently ordered testing to resume. Canada doesn’t have a federal education department, so each province makes its own decisions. Ontario has suspended annual Grade 3 and 6 reading and math tests for another year, but Alberta’s and British Columbia’s Grade 4 and 7 tests are back on track for now.

Yet these differences—when provinces resume testing—are less important than differences between the provincial tests themselves. When testing resumes in the United States, students in Grades 4, 8 and 12 will continue to take the same tests every year, with results posted on the National Assessment of Educational Progress (NAEP) website for everyone to see, compare and download. Canadian students, often at different grade levels, take their own provincial tests. Consequently, results in one province can’t be compared to those in any other. And even then, results in some provinces are not always easy to find and are typically hard to compare.

We do have one national achievement test—the Pan-Canadian Assessment Program (PCAP). Given to random samples of Grade 8 students in each province, PCAP estimates achievement in reading, math and science every three years. This is certainly better than nothing, but it falls far short of the coverage, quality and reach of the U.S. NAEP.

Moreover, a new study of math performance published by the Fraser Institute raises questions about the accuracy of PCAP results over time. According to the latest PCAP results, math performance has improved in all provinces except Ontario. Yet according to the Programme for International Student Assessment (PISA), the gold standard for comparing school and student performance around the world, math scores have fallen in all provinces. The two tests nevertheless agree that Quebec students have consistently outperformed their peers in all other provinces over the past two decades, with Ontario and Alberta in second and third place.

More broadly, Canadian math scores have steadily fallen compared to other counties. According to PISA results, from 2003 to 2009, Canada and Japan had similarly high math scores but by 2018 Canada had fallen significantly behind Japan and had been overtaken by Korea, Estonia and the Netherlands. We continue to outperform the U.S. but the gap has narrowed.

However, given their sampling limitations, neither PISA or PCAP allow fine-grained comparisons within provinces, like the NAEP in the U.S. As such, a new comprehensive Canadian assessment program would allow parents, education professionals and politicians across Canada to compare performance between school boards, cities, suburban or rural areas and public, separate, French immersion and independent schools. Indeed, it’s time for a serious conversation about a national system of student assessment to measure the performance of students at the same grade levels in all kinds of schools in each province against the same standards, ideally keyed to international benchmarks. The NAEP provides a compelling template, with comparable results on websites and summary results published in easily understood annual report cards.

Unfortunately, when the pandemic ends and schools return to normal, some will resist a return to testing, even Canada’s acutely imperfect hodge-podge of non-comparable provincial tests. There have already been calls from teacher unions to scale back or eliminate student testing in Ontario. But running today’s complex billion-dollar school systems without objective, comparable and scalable performance measures is like sailing across the Pacific without GPS, radar or even a compass. We might eventually reach land by steering for the setting sun, but have no idea where we will be, how far we’ve gone or how others are doing.

As part of our COVID recovery, let’s replace our outmoded and limited testing programs with a modern system of national student assessment.