Conundrum: Reading and Listening Assessments

 I need to think through our KS3 Reading and Listening assessments in the light of the Ofsted Research Review.

At the moment we have regular vocabulary tests, assessed written work and speaking. And Listening and Reading tests. We use tests based on past papers from the FCSE. Many years ago we used to enter pupils for the FCSE, and when National Curriculum Levels disappeared, we carried on using some of the tests to ensure continuity and comparability.

The things from the Ofsted Review that I am thinking about are: 

Pupils should be tested on what they have learned and how well they know it.

Tests are an important part of showing pupils their progress and making them feel successful.

Our KS3 tests "content" and "skills" separately. The language they know, and what they can do with it. We have vocabulary tests that test how well pupils are learning their words and structures. And these also model how testing is an important part of the process of learning. Then we have end of unit Reading and Listening tests. These have the language pupils have been learning in context. They also have some unknown words and structures. We encourage pupils to use Reading strategies, Listening strategies and Exam strategies to deal with this.

This is the first decision we are going to have to make. Is it worthwhile and reasonable to have this type of test? An immediate answer would be that pupils do cope, pupils find the right answer, pupils use skills and strategies. So it seems right to expect them to do it.

This throws up a series of further questions. We have to examine whether these are the things we want to be testing. Are we simply separating out the pupils with strong underlying literacy skills and labelling them as "doing well in languages"? And vice versa. Not doing well.

Or are these literacy skills at the heart of what it means to make progress in a language? Next question: are we developing literacy by having this sort of test? Do our lessons show that we know how to teach pupils how to read texts with unfamiliar words? Do we do this through explicit strategies, or through lots of practice? If we were to stop doing this kind of testing, would we be stepping away from supporting pupils' literacy?

Some of the skills may be exam skills rather than literacy skills. Using the question to lead you to the answer. Eliminating the wrong answers to arrive at the correct option even though you didn't "know" it. The Ofsted Review specifically rejects the validity of this. I am not so sure.

Then the question of "what pupils have been taught." Is this so clearly defined? Do we want to martial our content so that all pupils are expected to know the same words thoroughly? But nothing outside the defined list? Or is learning a language part of something much bigger? All the words you have ever met including in songs or stories or articles, or because you looked it up in a dictionary or because you play video games in French...

Does that same argument apply to the "skills"? The willingness to take risks and speculate on meaning; the confidence to build a hypothesis on what you think something says; the clarity of vision to be able to change your hypothesis when it is not confirmed by the rest of the context? Are these things that can be developed by lots of exposure to texts? 

From a social justice point of view, is it fairer to make sure everyone achieves the same, even if that means restricting others? Or not?

This leads on to the second element I picked up from the Ofsted Review. Tests should be part of what makes pupils feel successful and confident in their progress. Our current tests do discriminate between pupils who can cope with longer texts, unfamiliar language, new content, and those who can't. Is the whole idea of testing in this way inappropriate? Should we simply be using tests to make sure pupils have learned what we taught them. And if they haven't learned it, working harder on making sure they do. Should we design tests where all pupils achieve full marks, by simply testing them on things we know they have learned?

By this stage of writing a post, I would expect to be coming to some answers. It's not happening yet. What would you advise?

Ok. So a day later and I've had time to think about it and some useful conversations on twitter:

I am not going to resolve the principles, but I am going to be more aware of them. So I can review our assessments on a case by case pragmatic basis and address the balance.

I think it has become clearer in my mind where the balance has to be struck:

tests need to discriminate between pupils to give us a range of marks

versus

tests need to help teachers and pupils have a feeling of success and progress

tests should test what we have been learning and not primarily be a test of literacy skills

versus

literacy is central to learning a language and should be developed and assessed 

what pupils see in Reading and Listening rehearses the structures we use in Speaking and Writing

versus

pupils also see texts and learn to recognise high frequency words that may not yet be part of their core repertoire for Speaking and Writing

These questions won't go away and don't need to be resolved. They need pragmatic and careful balancing. That's our job.