The core qualities that drive success in the 21st century can be learned but not necessarily taught. That is the most critical insight in journalist Paul Tough’s excellent book, “Helping Children Succeed,” which came out this summer. It is an important conclusion, in part because it takes issue with more conventional school reform, which has tended to focus on “fixing” the students, the parents, and, most recently, the teachers inhabiting that environment.
Instead of concentrating on individual actors, Tough’s book showcases the interventions – some school-based, many not – that have altered the very conditions in which children grow up. Building independence and self-efficacy through rigorous but meaningful academic work is key. Doing so, the research shows, has successfully lifted up children in poverty, imbuing in them the same sort of academic ambition and resilience that has traditionally been the province of higher income neighborhoods.
But as remarkable as these programs are, sadly many of them live on the margin of the K-12 and surrounding systems. They are niche programs removed from the mainstream and delivered to a lucky few recipients. Along with Tough, those of us who are grantmakers to these programs worry that this work, at least to date, is reaching too few students and at too slow a pace.
For his next book, I am hoping Tough will write “Helping Children Succeed at Scale.” This is a difficult and complex proposition. But if I were to venture a guess, I would predict that Tough’s insight about the conditions for student learning to be successful are also true for adults.
In other words, to construct powerful learning environments, teachers and school leaders also need to work under conditions in which they are challenged, autonomous and accountable. As it currently stands, teachers chafe under school constraints, districts complain about state mandates and states, in turn, about federal strictures and their inability to chart their own educational destinies in light of the detailed prescriptions of No Child Left Behind.
Fortunately, all of that is beginning to change. Or at least the possibility of change exists. No Child Left Behind is over. It has been replaced by new federal legislation with a much less evocative title – the Every Student Succeeds Act, which does not envision an entirely new education system.
But Every Student Succeeds Act does reduce some of the barriers, remove the more onerous and impractical provisions of No Child Left Behind, and maybe most importantly, it begins to restore a degree of agency to state education systems.
My hope is that this is the moment where the beautiful work of our grantees in crafting the sort of vibrant learning environments that Tough describes can begin to find purchase. Where states can create the system conditions for districts to do their best work and districts can enable schools to innovate for their students.
Several respected policy organizations, including the Council of Chief State School Officers, the Alliance for Excellent Education and the Learning Policy Institute, are focused on forging state education systems that encourage a broader view of student progress. States are beginning to recognize both academic excellence and the non-academic skills that Tough so eloquently describes. And they are focused on capacity building at every level so that the adults in the K-12 system have the resources, support and authority to be successful.
I don’t think that any of us are naïve about how challenging this will be to bring about. Policy change can at best enable - but not create - the learning environments that Tough and others seek, particularly for our most disadvantaged students. This hard work will fall to schools, teachers and the students themselves.
That said, for a very long time, the education programs Tough so beautifully describes, like EL Education, have succeeded despite a myriad of hurdles from many quarters. They have kept “two sets of books” – holding themselves accountable for the deeper learning competencies that will help their children thrive and enduring a multiplicity of challenges that have very little to do with this.
Beginning to develop coherence and alignment between the work of great educators and the environments, in which they find themselves, is both an important next step and near-term opportunity.
December 10, 2015 — By Christopher Shearer and Barbara Chow
Today, President Obama signed into law the Every Student Succeeds Act (ESSA), which replaces the controversial No Child Left Behind (NCLB). The bipartisan passage of ESSA propels the nation into a significant pendulum swing that will shift us away from what some characterized as the long arm of the flawed NCLB accountability system over to what others fear might become a Wild West of state control. Perhaps the most seismic result of the new law will be state revisions of their accountability systems. And, given the demands of the global economy and modern civic life, getting accountability innovation right is a major, and timely, issue—states must seize the chance to ensure all kids get beyond the “3 Rs” to acquire the more robust knowledge, skills, and learning mindsets they’ll need to succeed.
Accountability and innovation are two words that sound like polar opposites. Accountability has a sort of weightiness; it is a word that, at its best, conveys responsibility but, at its worst, conveys rote compliance and guilt. Innovation, in contrast, seems unchecked by convention, buoyant—but also risks encouraging “anything goes” for the sake of experimentation. Pairing these words—and the messages they invoke—in the context of education policy might be surprising. But this is precisely what we at the Hewlett Foundation are striving for as we look to the future under ESSA.
States that have felt constrained by federal policy will now be able to experiment to sort out for themselves what they want their education systems to produce. Some folks fear that we return to an era without an agreed-upon “floor” of expectations for outcomes and progress—or that we’ll allow backsliding to a time when achievement for all students was not an important-enough part of the discussion. The challenge—and opportunity—facing state leaders at this juncture is how to combine the best of what we mean by “accountability” with the best of what we mean by “innovation” to create new systems and supports that more successfully reinforce and act on what they value.
Much can be learned through such a merger, particularly about how an education system can support innovation while holding itself accountable for even better, more equitable outcomes—shifting from a static compliance model toward capacity building and continuous improvement. In this moment of change, we are hopeful states will lead towards a vision of schooling that supports all of the things we know about what children need to succeed. This vision should include rigorous academic achievement but also vital interpersonal and intrapersonal competencies: what we refer to together as the “deeper learning” outcomes that students must master to be successful in the global economy and today’s civic life.
How can newly-empowered leaders unite innovation and accountability as the pendulum swings their way? Inspired by our nonprofit partners—from state education agencies and policy design experts to researchers and educators—we offer the following specific actions that states can take:
Align Accountability Goals with College, Career, and Civic Readiness—Including Deeper Learning Outcomes: As states advance real student readiness as the goal of their education systems, they should take the step to define “readiness” as deeper learning and make this the “north star” goal for their accountability systems, including a full range of knowledge, skills, and dispositions. Setting clearly-articulated and overarching definitions of student success now is important to guide where states set the bar for school accountability—both in terms of student performance on the current crop of new assessments and on those indicators that will develop over time as we improve the achievement measures for such hard-to-test areas as collaboration and social and emotional learning.
Include Multiple Measures for Accountability: If states stipulate that they care about a range of outcomes, they should take this opportunity to adopt multiple measures of student learning. High-quality summative assessments of student learning—like those PARCC and Smarter Balanced have created—remain an essential component. But they need to be complemented with measures that reflect the full breadth and depth of whether students are ready for college, career, and citizenship—providing information on a range of cognitive, interpersonal, and intrapersonal knowledge and skills including rigorous academic content, critical thinking and problem solving, collaboration and teamwork, expert written and oral communication, learning how to learn, and development of an academic/learning mindset. Notably, states should include performance assessments that better measure and advance student application of knowledge and higher-order skills. States could also get beyond lip service to local capacity by making room for local assessments.
Report a Data Dashboard to Drive Improvement: States should use a dashboard system—a limited number of important indicator metrics and “warning lights” for critical aspects of accountability—to report and interpret progress on these multiple measures rather than only collapsing them into a single score or performance index. As a part of this disaggregated information on student outcomes, states could further empower ongoing action for school improvement by considering inclusion of equitable resources, information on opportunities to learn, and data on the climate for learning, among others.
These three reform areas will only succeed in practice if states pursue them through a policy of continuous improvement, committing fully to equity, to evaluating their success or failure, and intentionally fostering local innovation with an eye toward fairly scaling up effective improvements. States must ensure that revised designs do not simply shift state and local systems from one static “box” of accountability to another unadaptable box—especially as new measures of currently “hard to measure” student outcomes are developed. They should embrace ongoing innovation tied to clearly-articulated goals rather than to innovation for its own sake. Finally, new accountability systems should be connected to a real, statewide system of supports designed to help all schools improve—with a particular focus on mechanisms to enhance the professional capacity and learning of educators themselves.
Passage of the Every Student Succeeds Act will kick off a time of momentous change. As state leaders combine accountability and innovation, they must remember that schooling serves all students and provides a public good. Indeed, accountability systems were first developed precisely to ensure that the most at-risk children receive the supports they need and that their achievement is transparently reported. We must not lose this goal. As states implement improved accountability models, which make new room for a richer and more demanding array of student outcomes, we can and should keep equity top of mind in both design and implementation.
We believe that these key reforms—basing accountability on a clearer definition of desired outcomes, adopting multiple measures of student achievement on these outcomes, and using dashboard reporting—are critical to success in this transitional moment. States can and should combine innovation and accountability by advancing the deeper learning competencies necessary for all students to be successful in college, career, and life.
Textbooks aren’t new or novel, and they certainly don’t represent the cutting edge of the Open Educational Resources (OER) movement that the Hewlett Foundation has long supported. But it’s precisely because of the way they are deeply embedded in the education system that they have the potential to shift OER from the exception to the default in school districts and colleges across the United States. And make no mistake: OER is poised for widespread adoption, with research showing that nearly 10% of K-12 teachers and over 10% of higher education faculty regularly use OER in some form. That realization, which heavily informed the strategic planning process we began when TJ Bliss and Dana Schmidt came on board as program officers for OER, formed the basis of our revised strategy for OER. This strategy will focus on supporting the development and adoption of open textbooks over the next three to five years while also maintaining a robust infrastructure for the OER field until it becomes self-sustaining. We’ve gotten enough questions about this decision from grantees and colleagues that we thought it was worth explaining in more detail with this blog post.
Three big trends are driving our push for open textbooks. First, millions of college students today are crushed by debt brought about by the high cost of education. Open textbooks can play an important role in solving this problem (particularly for community college students, where textbook costs can be as much as 40% of the total cost of a student’s education) by reducing the costs of instructional materials. Many current digital support tools (like learning management systems and assessment platforms) are bundled with printed texts that students can’t afford, but the vast majority of educators and students still need the content from those textbooks. At the same time, as a result of years of investment by private foundations and government, a supply of less expensive, high-quality open materials does exist, and they are more flexible than proprietary textbooks and more easily updated. They’re better for students, and more empowering for faculty.
Second, while new digital support tools are rapidly appearing in the market, educational technology depends on quality content to meet curricular requirements. The deployment of open content in these new tools can lower overall development costs, which can then speed time to market and keep costs low for students. Supporting the development of open textbooks is a key requirement to achieving the digital promise of tomorrow.
Third, schools with the largest populations of disadvantaged students have neither the infrastructure nor the funds to take advantage of the latest education technology. These schools still purchase textbooks and are likely to do so for many years to come. Open textbooks, while looking toward the future, meet the needs these schools and students have today.
Indeed, there is a desperate need for open textbooks. In the K–12 world, the gap between textbooks aligned with high standards and what is actually available has widened considerably. For example, EdReports, a new organization that we have supported to serve as a type of Consumer Reports for education, has documented a significant lack of alignment between the most-used math textbooks and the new Common Core State Standards they are supposed to cover. In community colleges, textbooks make up to 30 percent of the costs of a student’s education. In many cases textbooks are so expensive that students simply forego their purchase, diminishing their educational experience and chances of success.
By contrast, openly licensed materials are hitting the mark and filling a market niche. EngageNY has developed openly licensed educational materials that have been vetted for quality by the authors of the Common Core State Standards. These resources have been downloaded over 24 million times and teachers around the country have independently cited them as providing valuable support in the shift to new standards. Similarly, OpenStax College, which creates openly licensed textbooks for college courses with the highest enrollment, has already experienced significant market penetration, achieving mainstream use in 20 percent of all Title IV institutions despite virtually no marketing.
So open textbooks are clearly good for the education system as a whole, but is this the best focus for OER? After all, improvements in materials alone will be much less effective without better and more open pedagogy, and textbooks are fairly U.S.-centric while OER is a global movement. We recognize that there are many other cutting edge ways in which OER could contribute to solving serious problems in the education world. So why continue supporting open textbooks when it is very possible that “the textbook” will not be the primary form of content delivery in the digital future?
Looking at this question from the vantage of the OER movement, we believe that it is precisely because textbooks are not new or novel that they carry within them the seeds of change that could help OER cross the innovation chasm and jumpstart widespread mainstream adoption—moving solidly into "the wood" in the pencil metaphor of school technology integration pictured above. In K–12, states and districts allocate funds to purchase textbooks, while in higher education, faculty assign textbooks for students to buy. In our home state of California, textbooks serve a critical articulation function between community colleges and four-year colleges. Textbooks also serve as a means of fulfilling K–12 educational equity requirements as dictated by the landmark Williams settlement. In other words, textbooks are solidly within the workflow of the education system. And focusing on something that offers a solution to problems that are widely recognized within the education system itself represents the best opportunity for mainstream adoption of OER.
As we’ve considered the range of options where OER could make a meaningful difference, we’ve been weighing four core areas:
Benefits: Is this a problem area that is likely to lead to tangible and persistent benefits for potential users?
Potential to succeed: Is the context favorable? Is there demonstrated demand? Is there an evidence base documenting the benefits? Is there field capacity?
Philanthropy’s unique role: Are there market failures that neither public nor private resources are positioned to address? Is there enough momentum to make a potential OER solution possible but not so much momentum that it will happen on its own?
Resources for execution: Do we at the Hewlett Foundation have the internal capacity in both staff and resources to invest sufficiently in a particular problem’s success? Are other funders interested in this topic?
In our view, a focus on open textbooks in the near-term meets all of these criteria. The benefits are widespread. For example, a recent U.S. PIRG report estimates that if every full-time undergraduate student had just one of their traditional textbooks replaced with an open textbook each year, it could save students nationally almost $1.5 billion in textbook costs.
Evidence shows that OER can reduce costs while maintaining education quality. A rigorous study published in Education Researcher last year found that students who used open science textbooks in Utah high schools performed better than students using proprietary science textbooks. Amazingly, the open textbooks cost about $5 per student while the proprietary textbooks cost $75 per student, which demonstrates remarkable learning outcomes per dollar. Research on OER in higher education is also fairly consistent in finding that OER decreases costs without negatively impacting learning. For more information on the mounting evidence, see the “Review Project” by the Open Education Group, which provides a summary of all known empirical research on the impacts of OER adoption, or check out the “OER Evidence Report” recently published by the OER Research Hub.
Philanthropy can play a catalytic role in pushing OER as a solution to the real problems the education system is experiencing today. Given the growing availability of OER supply, the costs of advancing open textbooks is relatively small as it consists of making policy efforts, filling gaps in supply, marketing, and building awareness.
We recognize that there is a long list of other areas related to OER where our funding could make a significant contribution. These are all within our sights. But for the next few years, we believe open textbooks represent the greatest promise for mainstream adoption of OER.
This post is the first in a three-part series on the relationship between Deeper Learning and the Common Core standards. The other posts in the series will be published later this week –Ed.
Lately, a lot of people have been asking me about the relationship between the Hewlett Foundation’s work to promote “Deeper Learning” and the Common Core State Standards. Some of this is because the Common Core has become much more politicized and controversial; some of this is because there is confusion about whether or not deeper learning complements—or somehow competes with—the standards; and some of this is because the new Common Core assessments are being field-tested and folks are unclear about what exactly is being measured and if the tests are any better.
Here’s how we think about it at Hewlett’s Education Program:
Deeper learning is a shorthand description of the student “competencies” (or mix of academic content knowledge and higher-order skills) that are most critical for success in the rapidly changing world of work and civic life. Ample research has clarified that those skills include critical thinking and problem solving, effective communication, collaboration, learning to learn, and developing an academic mindset. When this potent mix of skills and dispositions are applied to a mastery of core academic content, deeper learning is to educational outcomes what jet fuel is to airplanes.
With its goal of “fewer, higher, clearer” standards for both English Language Arts and Math, the Core moves the ball forward by providing educators with a useful yardstick for measuring both a student’s understanding of fundamental principles and concepts, and the higher-order thinking skills that are key components of Deeper Learning . To date, 43 states have voluntarily adopted the Common Core standards in an historical show of commitment to educational excellence.
The Common Core and deeper learning are connected because many of skills emphasized by deeper learning will be advanced through successful implementation of the Core. Independent research from a number of prominent experts confirms this. Notably, a research paper from Achieve (a key architect of the Common Core standards) and an analysis of how Deeper Learning and CSSS interact from researcher David Conley (one of the country’s foremost experts on college readiness) have both documented the strong relationship between the deeper learning competencies and the Common Core standards.
For us, the Common Core represents a rare opportunity in US education reform to ensure at scale that students are exposed to a more rigorous education that will better prepare them with the knowledge, skills, and learning mindsets they’ll need for success in college, career, and civic life. For these reasons, the Hewlett Foundation has been deeply supportive of the Common Core standards, with a particular focus on better assessments as a way to signal and support school systems for continuous improvement.
But we recognize that, like any education reform, the Common Core can either be implemented well or it can be implemented poorly. If done well, teachers will be given the freedom, support, space, and time to adapt instruction to meet the individual needs of students and to develop their higher order thinking skills, and the Common Core will be a runaway success. If, instead, the Common Core is viewed as a “flavor of the month” fad, if it stifles teacher creativity and risk taking, if it becomes another “top down” reform tied to high stakes accountability, or if it simply results in no real changes in practice, then the historic adoption of common standards by the vast majority of the country will go down as a squandered opportunity in the annals of education history.
As we look across the country, common core implementation has been highly variable—in some cases, it looks like the best case scenario outlined above and in others a less rosy picture has emerged. To be fair to all students and teachers it is critical that many more districts and schools become deeply engaged with the new standards. That’s why we think it is so important to support effective Common Core implementation. Schools need to align high quality curriculum with teacher professional development and to deliver the standards with fidelity to deeper learning. Expeditionary Learning’s excellent English Language Arts model curricula—selected by New York for all of its schools in grades 3 to 5, is a good example. Chicago’s Polaris Charter Academy is another, offering living proof that the Common Core can spark exuberant and creative teaching and learning in some of the country’s most challenged communities.
The Common Core is not a magic potion that will cure all of the ills of the education system. Nor will it deliver all of the skills that students will need to prosper in a complex and changing world. But it represents a significant, scalable step forward for deeper learning that offers schools flexible guidelines, improved measures, and the opportunity to raise the ceiling of student achievement.
If we hope to step up as a nation to improve public education, there is no question that standards must be raised for what students know and can do. We believe that the Common Core State Standards provide an excellent set of academic benchmarks. If they are implemented with integrity and creativity in service of the ultimate goal—powerful deeper learning for all students—we believe they can be a key part of the solution.
It is graduation season: caps and gowns, relieved students, and happy parents across the country.
I graduated from Troy High School in Fullerton, California many years ago—more, in fact, than I care to either count or reveal, but suffice it to say that when I donned my cap and gown, Tony Orlando and Dawn had the number one song in the country, the Berlin Wall was still intact, the closest thing anyone had to a personal computer was a calculator, and people still thumbed through gargantuan white and yellow tomes to find phone numbers.
My son Lucas graduated from high school only six years ago, just as the Great Recession was ratcheting up, and, while much has changed in the span of days that separates my son’s graduation from my own, one thing that remains remarkably constant is the classroom experience in most of our nation’s high schools.
Why is that a problem?
Here’s why: while my son, like me, was lucky to finish college and land a job in his field straightaway—whew!—he and his classmates are unlikely to have a career path that is remotely similar to mine or that of my peers. After graduate school, I worked for nearly three decades for essentially the same employer—that curious industry known as the federal government—both at the White House and in Congress, before leaving to head the Hewlett Foundation’s Education Program.
According to a 2012 study by the Associated Press, however, 53 percent of today’s college graduates are unemployed or underemployed, the highest in more than a decade. Students born today will also probably have between ten and fourteen jobs before their thirty-eighth birthday. Further complicating things for this generation, according to the Department of Labor, is that most of the jobs that Lucas and his friends will likely apply for in the future—the good ones anyway—haven’t even been invented yet.
Coupled with automation and outsourcing, these trends have redrawn the map of the American labor market, requiring today’s high schools to reinvent the employees, employers, and citizens of tomorrow. To keep pace with the seismic environmental and social changes that are recalibrating America’s future, our children will need a different set of skills than we acquired in high school.
When I graduated from high school, it was possible to get a job that would last a lifetime, as long as you possessed a strong work ethic and a basic procedural understanding of the world. Lucas’ generation will almost certainly have to go a step further, and convince prospective employers that they can solve unforeseen problems, which don’t appear on any standardized test or surface in a job interview. They will have to demonstrate a certain nimbleness of mind—and possess an educational portfolio that indicates a certain kind of depth.
To help address the needs of today’s graduates, the Hewlett Foundation’s Education Program has invested, since March 2010, in an emergent initiative known as Deeper Learning, which we believe best equips high school graduates with the tools—or in educators’ lingua franca, “competencies”—they need to navigate the twenty-first century’s shifting and uncertain terrain.
So, what is “deeper learning”? First, perhaps, I should note what deeper learning is not: it does not represent an abandonment of the basic principle that high school graduates should master core academic content such as reading, writing, and math. Indeed, deeper learning is a bit like baking a cake, with core content as a key ingredient, but the final product relies on a mix of desired educational outcomes that emphasize the ability to think critically and solve complex problems; communicate effectively; and collaborate, learn how to learn, and value learning (or what we in education call an “academic mindset”). These higher-order thinking skills and learning dispositions enhance mastery of content and allow students to apply their knowledge to new situations.
Until recently, the tools and evidence base available to reformers in this arena were limited. But a new book, written by a Bay Area educational consultant, Monica Martinez, and a research study released this month by one of our grantees, the American Institutes for Research (AIR), provide us with the strongest evidence yet of deeper learning’s promise.
By eschewing a one-size-fits-all curriculum, teachers in the eight deeper learning schools studied were able to tailor the educational experience to their students’ unique needs and ambitions, and identify “the spark—a subject, idea or project that makes a student light up.”
That classroom success is beginning to materialize in ways that we can measure. In the recent AIR study, researchers found statistically significant increases—standard deviations of between 0.19 and 0.50—in all eight indices they established to measure opportunities for learning outcomes, including “communication, complex problem solving, creative thinking, and real world connections.”
What’s more, AIR found that the deeper learning schools they studied produce significantly higher on-time graduation rates. If non-network schools graduate sixty students on time, schools in the deeper learning network graduate sixty-nine students within four years.
These differences seem admittedly modest, but research of educational interventions typically show no change at all, so I hope you share the enthusiasm we felt at the Foundation to learn that our hopes for deeper learning are much more than mere wishful thinking.
They’re on to something here.
So, with another graduation season upon us, we should think hard about what our kids can actually do with their degrees. If they receive only a traditional education, do well on multiple-choice tests like the ones I took, and do not go on to postsecondary education, they will likely move very quickly from celebrating commencement to struggling to find a job and keep it. If, however, they can demonstrate that they can solve unstructured, complex problems using new information—made possible because they have mastered deeper learning competencies—they are much more likely to be successful in college, land that first job, and fashion a rewarding—and remunerative—career as an adult.
Changing education to deliver deeper learning won’t be easy for teachers, administrators, or policymakers. But deeper learning gives us a sense of where the finish line is, and while getting there will require a concerted effort from everyone—policymakers, teachers, parents, and schoolchildren—perhaps we should be mindful, now, more than ever, that America didn’t become America by failing to rise to a challenge.
Earlier this month, I was in Philadelphia for the annual American Education Research Association conference—the largest gathering of education researchers in the country. It is, in a word, overwhelming: roughly 13,000 education researchers sporting name tags and blue bags, looking exhausted or annoyed; scurrying through the vast maze of conference center hallways searching for sessions, colleagues, coffee.
Three things struck me about the whole event and the state of education research. First, with thousands of researchers examining some aspect of education that they are hoping no one else has studied, it seems to me that a lot should be known about education. The amount of research and breadth of topics is staggering. I randomly sat down at a roundtable discussion (number 21 of 40 or so tables) and it turned out that five researchers were studying the question of how teachers use assessment data in their instructional practices. This is exactly what I’d hoped to hear more about and I just stumbled into the conversation.
But I also discovered that while much is investigated, nothing is very definitive. So of the five research studies, a few were still in early stages, others had conducted some research but found “no effects,” and so on. Even with the “no effects” research, we all (including the researcher) hypothesized that perhaps this tweak or that would improve the results and they should try again. I’ve found this often happens in education research. Maybe it is appropriate that research is never really finished, but it is nonetheless frustrating when the primary recommendation is usually “more research is needed.”
Every once in a great while, something is actually concluded. We have decided, for example, that class size reduction, once considered the great silver bullet, does not consistently correlate with higher student achievement. Good to know. At least we don’t need to spend money on that any more. But even here, it turns out that class size reduction under the right circumstances does produce effects. So, the answer to the question, “does this work?” is most accurately answered by “it depends.” Context, culture, fidelity of implementation, and the mysteries of the human brain all figure into the answer and the inexplicable alchemy of it all make it difficult to create many generalizable and useable action steps emerging from research.
Look, I get it. No self-respecting researcher is ever going to make an unqualified statement that X always causes Y. That doesn’t describe reality, particularly in complex systems populated by unpredictable humans. But research is being funded in large part because we want to understand how to get better at what we are doing. To improve our education system we will need to take action and we’d like to do this based on evidence.
So, what to do? I have three suggestions. One, make it easier to find the research. One way to do this would be to create a taxonomy that categorizes all of those thousands of research studies and a clearinghouse with a really good search engine that stores them. Two, we need more meta-analyses that sum up what is known and give practitioners guidance (as best we know) about what should be done and what should be avoided. Lastly, we need open anonymized data (that protects student privacy) so that all of the millions of data points that are emanating from online education can be accessible to those thousands of researchers so they can learn stuff and speak with greater authority about what they know.
And we need more coffee stands in convention centers.