On March 28, 2019, the U.S. Department of Education (DOE) Institute of Education Sciences (IES) published a notice (84 FR 11755) in the Federal Register seeking public comment on IES’s proposed priorities.
The stated mission of the IES is to "provide scientific evidence on which to ground education practice and policy and to share this information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public." The director of IES is required to propose priorities for IES, which must be submitted to its board for review after a period of public comment.
MIND Research Institute has submitted its comments, which will also be available on the docket (ED-2019-IES-0017) along with comments submitted by other organizations, agencies, and entities. Comments are due by May 28, 2019.
MIND’s comments focus on the following proposed IES goals:
“Edtech effectiveness research is tricky to navigate but is coming off as deceptively simple,” said Andrew Coulson, Chief Data Science Officer at MIND Research Institute. “Well intentioned regulations and non-regulatory guidance have, in practice, created a simple checkbox mentality. If a product has ‘one good study,’ the box is considered checked, and the decision maker moves on."
But at MIND, we believe that educators shouldn’t rely solely on one piece of evidence, even under the ESSA tier system. An edtech program could have been rated as having "strong" evidence based on a single randomized control trial, done many years ago, on a version of a product that no longer exists.
To change this dynamic, MIND’s comment suggests that educator decision makers should be empowered to expect more from edtech research - to not only look to see that one study exists, but also to take a deeper look at what the research is really saying, and whether the studied solution will make an impact for their students.
The MIND comment further asserts that rather than relying on one “gold standard” study, we should be looking at a large number of studies, using recent program versions, garnering repeatable results, over many varied districts.
“Under the current ESSA tier system, only a randomized control trial (RCT) carries the weight of tier 1 ‘strong’ evidence,” said Coulson. “But quasi-experimental studies - tier 2 ‘moderate’ evidence - make it possible to conduct a vast number of studies, and therefore provide extremely valuable data for decision makers. With more studies, educators can evaluate repeatability, and identify consistent patterns over multiple years, across grade levels, and especially across different types of districts and assessments.”
At MIND, we also believe there needs to be much more non-academic conversation and education about research, and that all formal research findings can and should should be unpacked and translated to become useful to decision makers.
For more about our methodology, visit stmath.com/impact.
Liz Neiman is Vice President of Engagement at MIND, leading the marketing team's plans and activities to promote MIND's initiatives and impact. Besides education and gaming, her interests include music of all kinds (from musical theater to heavy metal), cooking and baking, and fashion. Follow her on Twitter @lizneiman.
.png)
Comment