The Tennessee State Board of Education sits at the forefront of evidenced-based policy making in education and is a pioneer in developing educator preparation report cards. In her role as Director of Policy and Research, Amy Owen is responsible for policy research and analysis and strategic planning. Ms. Owen explains the value of educator preparation report cards, the challenges of developing them, and Hoonuit’s role in helping their organization continually enhance the value of their reporting.
Hoonuit’s community engagement experts have been working with the Tennessee State Board of Education (TSBE) for four years – initially as Tembo and now as part of Hoonuit. TSBE is charged with developing rules and policies for all of K-12 education in Tennessee.
A key function of the TSBE is to produce an annual document called the educator preparation report card, which rates each of the approximately 40 educator preparation providers approved in Tennessee. Each provider is rated on a number of different metrics, ranging from the quality of their candidates to whether or not their candidates accept and are retained at jobs in Tennessee public schools. And most importantly, how they are impacting student learning.
“What do the evaluation and observation scores of EPP grads look like? Do they affect student achievement and student growth? These are important questions that we answer,” said Amy Owen, Director of Policy and Research at TSBE. “This is the State Board’s responsibility because we also approve all education prep providers on a seven-year cycle. Several years ago our legislature wanted to make sure there was an annual way to check on the quality of each EPP (educator preparation provider) and share that information clearly with stakeholders in the public and in the policymaking community.”
The educator preparation report card has been required by Tennessee state law since 2007. It was originally produced by the Tennessee Higher Education Commission, which oversees Tennessee public colleges, universities, and community colleges. “They didn’t have a clear way to display this information. It was a 300-page PDF document with numerous tables that provided great information, but it was challenging to understand without a strong background in statistics,” said Owen.
When the responsibility for producing the report cards switched over to the State Board of Education in 2015, Owen’s predecessor worked on a new version with the help of Tembo. At that point, TSBE made a service-oriented decision to expand the scope of potential users.
“It really is important that if we have all of this data and information at the state level, that we do something with it for the betterment of the public. We’re not just sitting on it, we’re making it something that people can understand to help make good decisions for themselves,” Owen explained.
“Here in Tennessee, we really focus on evidence-based policy making and having this data clearly available for the public in a way that not only a legislator could understand it, but also somebody who’s 18 and deciding between two colleges. If you want to be a teacher, you can compare your choices side-by-side and figure out what would the best fit. Or, if you are somebody who already has a bachelor’s degree and you want to go into teaching as a new career, you could be selecting programs that have a master’s in your specific area for some kind of post-bachelor program. So I think there’s the policymaking reasons and then there’s the reasons of trying to attract people into the teaching profession and get them into a place to be a good fit for them as they move forward in their career,” said Owen.
One unique advantage to Owen and her colleagues is the amount of student growth data available to them in Tennessee. “It is called TVAAS which is the Tennessee Value-Added Assessment System, where you can compare students across the state and how much they grew in a year in their key subject areas. We are not just looking at their achievement, if they are proficient or not, but their growth. Did they improve? Whether they started high or low. As part of our law, we actually go back and track the people who graduated from ed prep programs for the last three years and see if they are having a positive impact on student growth,” said Owen.
Tennessee’s legislators took a strong stance on TVAAS, which has given the State Board a clear charge with the report card. The legislature’s authorization made it a requirement that this data be included in the report cards. In turn, TVAAS data is a strong influencer in TSBE’s provider impact ratings.
One of the tremendous values of TSBE’s report cards is their ability to look at outcomes. “Not just that the EPP checked the box, that they had a high enough GPA requirement and that people took a licensure assessment, but do they actually do what we care about? Which is to have a positive impact for kids in the classroom. I know that’s something when I’ve talked to other states, they’ve been envious. They were asking, how do you have the ability to link this impact data to EPPs, and how has the state developed buy-in for this process?” said Owen.
Owen describes two key challenges in developing educator preparation report cards. The first is making sense of mountains of data so that the larger public will understand it. Hoonuit works with Owen and her team to develop and execute strategies to effectively translate complex data into meaningful information. There is also a heavy emphasis on data quality. Erika Leicht, Senior Research Associate at TSBE, ensures that all the data in the report cards are clear and accurate. She validates data with partners at the Tennessee Department of Education and provides EPPs the opportunity the check their data before it goes public.
The other challenge is developing an accountability system for many different audiences who may have different beliefs about what is important. “The accountability system has a lot of different people who are affected by it. In Tennessee, we have public providers like UT-Knoxville or Tennessee Tech. We have private providers like Vanderbilt. We have Teach for America and similar programs. We also have smaller mission specific colleges,” explained Owen.
“Our goal is to have an abundance of strong teachers available to teach in our Tennessee public schools and we’ve done some tweaking on weighting of metrics. We’ve had places where we’ve provided additional information where we say not only how many folks were teaching in Tennessee public schools, but how many folks got a job in teaching period,” says Owen.
TSBE has been steadfast in their belief that the report cards focus on student outcomes. “The state board members have been a consistent voice in advocating for strong student-centered education policies,” said Owen. “Historically the models for accountability have asked if somebody has the right degree? Did somebody have the right GPA getting in? It’s very much about the inputs into the system. It’s been harder to look and say, did all of those things lead to a good outcome for our students? That’s really what we want to focus on with the report card and where the majority of our points are weighted within the report card framework. How are the teachers observed? What’s going on in their classroom? What are their principals saying about their skills and their areas of need? And what’s the actual impact on student growth, because that’s the ballgame for us. That’s why we’re all here.”
Each year, TSBE looks to make enhancements to the report cards and this year was no exception. An advisory council comprised of representatives from EPPs as well as district leaders, legislators, and other stakeholders helps TSBE identify opportunities for change and improvement. Entering the fourth year of the project with Hoonuit, Owen and her team adjusted their accountability system, added domains and metrics, and upgraded to the latest version of Hoonuit’s web platform. The current design was built with more inherent explanations that contextualize information and puts the values in the proper context.
“We really kept that design consistent for three years, and then this past year we made it even more accessible and user-friendly. We’re excited that it’s easier to compare EPPs to each other. So if I’m a student and I know I want to go to a public in-state institution, I can select those that have the area of endorsement I want and look at them side by sides. I think that’s really useful. We have new data on candidate satisfaction with their preparation and the rate at which they pass various licensure exams, which is also important for candidates to know,” she explained.
True to her research roots, Owen led her team through a reevaluation of many key metrics. “We went over each of our metrics and went through education research to verify if there is reliable evidence to suggest that things like scoring retention – how long is somebody actually teaching in a Tennessee public school – is truly important to score? Does it matter? And the research says yes, teachers tend to improve very much in their first three years in the classroom. So it is important that they are retained and they stay and continue to improve.”
The growth mindset at TSBE and their dedication to meeting the different needs of their audiences will lead to more enhancements down the road. “We are already trying to think about how we can continue to share information about the report cards throughout the year because different people need it at different times,” she said. “People might be making decisions about colleges to attend in the spring, or a legislator might want to know about it going into session in January.”
TSBE is also dedicated to expanding the reach of the website. “We’re especially interested in, and actively exploring ways to get more prospective teachers to be aware of this resource as they make their higher ed decisions,” said Owen.
Hoonuit is proud to be a part of TSBE’s success, and we’re looking forward to seeing their continual growth and leadership around all aspects of educator preparedness and reporting.