How Teachers Help Texas Create Its State Exams

Cindy Riney is a veteran history teacher and AP history teacher in the Montgomery Independent School District north of Houston. She has served on Texas Education Agency (TEA) panels that review test questions for the annual State of Texas Assessment of Academic Readiness (STAAR) exams. Riney, chair of her school’s social studies department, has served on a panel reviewing the state’s history standards.

Riney explained to Anne Wicks, the George W. Bush Institute’s Ann Kimball Johnson Director of Education Reform Initiative, and William McKenzie, the Bush Institute’s senior editorial advisor, how test questions are reviewed, refined, or rejected in Texas. She discussed the importance of teachers being part of the selection – and rejection – of test questions. And she reported how she uses testing in her practice as a teacher to help students become proficient in U.S. history.

Why did you decide to participate in a STAAR review panel?

It was more to understand the process itself. Some questions seemed to have very little to do with AP U.S. History and regular U.S. History. Some seemed more like a science question.

I didn’t know that teachers were involved until our counselor asked if I would like to get involved. I was thrilled and immediately applied.

I had been complaining about the STAAR test, not testing in general, because some part of the STAAR test seemed random. I wondered how I was to know that I was supposed to teach a particular part of the state’s standards. I wanted to see the connection between the TEKS (Texas Essential Knowledge and Skills) and the test.

I first sat on a committee devoted to streamlining the state’s standards. Then, I was invited to do the testing process, where we actually review test questions. Both of those committees are valuable and cleared up a lot of information for me. I don’t think many teachers know they can be involved in this process.

What was the process like when you were reviewing questions for the STAAR test?

There are lots of questions and it is a two-to-three day process. Someone from the Educational Testing Service (ETS) [Editor’s Note: ETS is Texas’ testing vendor] writes the questions, and the panel of teachers goes over them question-by-question. The process is tedious because everybody has an opportunity to speak on the question.

We look for validity, we look for bias, and we look for clarity. We also look at the answer choices to see if there are any confusing choices for kids.

 

We look for validity, we look for bias, and we look for clarity. We also look at the answer choices to see if there are any confusing choices for kids.

 

For example, we talk over a question if it contains a word that someone who teaches in a heavily Hispanic district knows their kids would not understand. Some social studies experts from the Texas Education Agency (TEA) also make comments. I have found them very knowledgeable and professional.

Then, we decide whether this question is a keeper, whether we’re going to edit it and review again, or whether we are going to toss it. It’s a tiring process, but everyone in that room is very committed to putting forth fair questions.

I was a skeptic before I got involved, but I was very impressed with the process.

 

I was a skeptic before I got involved, but I was very impressed with the process.

 

When reviewing these questions, is the goal to reach consensus when you finalize a question?

Yes, everyone has an opportunity to speak. Sometimes it’s a debate. If there’s too much disagreement, we put the question on the back burner and re-look at it on the last day. If there’s no agreement, then it’s out of there.

Has being part of this process changed how you teach?

I don’t think it’s made me teach any differently, but it’s made me a lot more comfortable with the test itself. I don’t feel like I teach the test. There are a lot of standards, and the U.S. History test covers all of them. We were told at our school to teach the important ones, so the kids pass the test and move on. But the U.S. History test covers everything.

I have used testing in the sense that all my tests look like the STAAR exam. I make a test bank for every historical era and then pull questions from STAAR tests. That has helped my students. The STAAR test isn’t a surprise, although some surprises occur every year.

 

I don’t think it’s made me teach any differently, but it’s made me a lot more comfortable with the test itself. I don’t feel like I teach the test.

 

It has been helpful that the state has released the tests. I disagreed with them in the past for not releasing them. Parents deserve a chance to look at the questions; kids can look at it later. It helps teachers to see what type of questions are asked.

One reason I wanted to be involved with this whole process, not just looking at test questions, is that some wording in the TEKS is too broad. For example, in U.S. History they will say “Thou shalt know people who forwarded civil rights, such as,” and then give you a list of people. When they use the term “such as” they could use anyone on Earth. That’s too open-ended for teachers.

What recommendations would you make for a principal or superintendent about the connection between the STAAR test and your classroom practice?

TEA now has put so much out on their website. Even if they don’t release a question, you can ask certain questions be released to you, even if the test isn’t released. Somebody at the district level needs to promote that. A lot of teachers don’t know what’s out there to help them.

I would recommend that teachers use the test questions. So many test questions are out there you don’t have to worry about somebody stealing the test and cheating the next year. It is time-consuming to go through them, though. If the district has a social studies coordinator or someone else who could comb through them, they could really back up teachers.

In my opinion, kids in a class that is tested are going to learn a lot more than in one that’s not. I’m the social studies department chair, and I can tell you that I am conscious of the TEKS. I remind our history teachers that they need to stay on track. You can’t just teach World War II the entire year.

 

In my opinion, kids in a class that is tested are going to learn a lot more than in one that’s not.

 

How do you see the alignment between the TEKS and the STAAR test? Are they fairly in line these days?

Yes, mostly. They do throw in some strange things, but it’s more rigorous. It’s not just testing: Who was the first president of the United States? It requires some thinking.

That’s also a better way to assess teachers. I know people don’t like that, but I do. I think it makes me a better teacher that I have to stay on the ball. I keep a list of the TEKS for every unit, and every year I double-check them to see if I covered them.

If they ask a question differently than how I have taught it, I want to know that. Did I cover this? But my kids are going to pass, and that would be the same for any teacher or school with a review program.

We do daily warmups starting second semester that cover both exam content and skills. We try to be finished with our course content by the end of the last week in April, so we can have two weeks to review material and take a practice exam or two.We do a warm-up exam, plus we have district benchmark exams every six weeks.

There are opportunities to review but I would recommend that schools not do too much testing. Otherwise, all we are doing is teaching to the test. Maybe just do a short benchmark test in one grading period. That keeps kids on their toes and it makes reviewing at the end less overwhelming for kids.

I look at the kids who failed English I and English II STAAR tests as my target group. Those are the kids who have trouble with reading. Since the U.S. History STAAR test is now a lot of reading passages, that’s my target group for extra help. The kids who are weaker readers are the ones who struggle. I would recommend districts target poor readers for remediation.

Yes, the struggle doesn’t end for high school graduates who are poor readers when they leave your campus. It will impact them, whether it comes to reading a ballot, a job application, or mortgage loan information.

What do you wish other educators knew about how STAAR is created and how to use it?

I’ve had so many colleagues say that this is just about ETS making money, that those people at TEA don’t know anything, and that they aren’t even teachers. But the folks I met at TEA, especially those in our social studies group, were incredibly knowledgeable and committed.

I had heard so many negative things that I was surprised to see how open they were to working with teachers. They really want teacher input. I don’t think teachers understand that. They are not trying to dictate everything that we do.

They may have been former educators, too. And they are committed to making these tests reflect what we’re supposed to be teaching students.

I was really impressed with the process, and a lot of teachers may not know about it. Districts have an obligation to involve their teachers, to let them know about this. It’s important to get teachers involved in the process.