Let’s examine our mental models on assessment

In his book Schools That Learn: A Fifth Discipline Fieldbook for Educators, Parents, and Everyone Who Cares about Education, Peter Senge (2012) talks about mental models and how we need to examine them in order to change. What keeps us from changing, many times, is that we do not think about why we think the way we do; we don’t examine our view of the world. An aspect that really needs reexamining and change is our mental models on assessment. Why do most people think the way they do?

In my experience with teachers, parents, and students, I have come to realize that there are many contextual factors that lead us to think the way we do in Brazil regarding assessment. They are related to how our society is organized, how education is viewed, and the great value we place on testing and school and student ranking. Because space in universities is limited, there needs to be a high-stakes, reliable selection process. Lack of human and financial resources precludes a system that considers multiple factors, such as grade records, statements of purpose, and letters of recommendation in addition to a standardized test, as is done in other countries such as the United States. Thus, we use a single, one-shot measure, our “Vestibular” and now our “ENEM”, to select students. The washback effect is that we tend to replicate this model in schools, where tests still have a prominent role and are developed according to the same model, in order to prepare students for the “big test”. It’s not that other types of assessments are not used, but they are less valued; what really counts is the big test. This creates in our minds a general assumption that only big tests can really measure learning. Not only that – it inculcates in our minds the idea that good tests are ones that look like these standardized tests, which are developed according to psychometric rules that make it imperative that there be items within a range of difficulty that goes from extremely difficult (only a very small proportion of very bright students can get them right) to very easy. The purpose is to discriminate the more capable students and find ways not to allow students who merely guessed the answers to be selected. This competition to get into university forges a competition among schools. Those that place more students in competitive courses in public universities are considered the best ones. School rankings are produced so that parents can identify the most effective schools. And then come the student rankings inside the schools. They need to know who the students they need to invest in are, the ones who will produce the good results that they envision – the ones who will make it to the billboards showing how many students achieve very high results.

In English Language Teaching (ELT), we are also highly influenced by standardized proficiency tests and are led to believe that we should follow their model for classroom assessment when, in fact, they are completely different things and serve completely different purposes. For one thing, international exams are proficiency exams, not achievement tests. They are meant to determine a candidate’s general proficiency and not to ascertain whether a student has learned what was taught – the purpose of an achievement test. Also, due to their high -stakes nature, proficiency tests have certain types of items that can ensure a high level of reliability, whereas achievement tests do not have to follow the same parameters and can encompass a wider variety of items, especially open-ended ones. Nevertheless, I frequently hear teachers suggesting that the assessments we give to our students be like the ones in proficiency exams. What they need to be like, truly, is the types of instructional activities that students are engaged in in the classroom. Oh, but then, when we do that, we think that the assessment is too easy! “Students just need to perform in an assessment what they perform in class, not at a higher level, with a little more difficulty? They are going to think that the school is ‘weak’!”

We are led to think that good schools are those that put a lot of stress on students, that produce tests that will always bring an element of surprise and, thus, encourage students to study really hard for them, trying to figure out what the new “catch” will be. In our society, good schools are also those that do not give a lot of chances because, let’s face it, the world does not give us a second chance. Also, they must emphasize tests, especially long and difficult ones, so that students can be prepared for the “many tests” they will have to take throughout their lives. Yes, the school can also include other types of assessments, such as projects, in their curriculum, but they can’t substitute or be worth the same points as the big tests. We are also obsessed with grades. We think that an instrument that does not produce a number is not as important as one that does. Even very well written and detailed descriptions of levels of performance are regarded as less “official” than a numerical score. Also, reliable assessments are those that are as “teacher-proof” as possible. Teachers’ daily observations of student performance are never as important as the tests. What matters is what the student shows on that one day, and not what they show daily. I have a close relative who is running the risk of failing math because his mind goes blank during the tests. He forgets everything he knows, literally. His math teacher says that the student knows all the content; he is sure of that. But that doesn’t count. What counts is the test. Only the test can really determine if he knows his math, not his teacher who observes him every day, not his private tutor who says he doesn’t need tutoring because he knows the content.

This mainstream mental model of assessment is incongruent with today’s world. In order to change it, we must understand it first, unpack it. The aspects that I brought up here are just the beginning, the tip of the iceberg. My aim was to focus on a basic set of assumptions, namely, that…

• a good test must be long and difficult;
• a good test is one that accumulates a lot of content. Shorter and more frequent tests are not as good as big tests, so even though they can be used, they can’t replace the big test;
• a good test needs to discriminate the students. If everyone goes well, it’s because the instrument is not effective;
• giving too many second chances on tests makes the school seem weak and ineffective;
• assessment systems without grades are not effective.

What other beliefs would you suggest we examine in order to change our mental model?