Revisiting Principles of Assessment for the Four Language Skills

TOPIC 2: REVISITING PRINCIPLES OF ASSESSMENT FOR THE FOUR LANGUAGE
SKILLS

2.1 Principles of Assessment for the Four Language Skills

In this topic, we are going to revise the basic principles of assessment which are:

Validity Reliability Fairness Flexibility

We are also going to revisit:
The principles of assessing receptive skills
The principles of assessing productive skills

2.1.1 Validity
Validity is a crucial element of assessment and evaluation as tests are meaningless without it.

Validity

• The ability of the test to measure what it intends to measure.

Imagine that a driving test asks you about banking and finance terms. Would that test be valid?
To produce a valid assessment, teachers must be clear about what they are assessing. Teachers
should always assess what have been taught and reasonably anticipate what the students may
know.

There are different types of validity:

Face validity
• The test appears to measure what it is supposed to measure.
• A test has face validity if:
1. directions are clear
2. organized in a logical way
3. no surprises
4. time appropriate
5. appropriate in difficulty

Construct validity
• The fit between the underlying theories and methodology of language learning and the
type of assessment.

Concurrent validity
• The test results are supported by other concurrent performance beyond the assessment
itself.

• Statements of concurrent validity indicate the extent to which test scores may be used to
estimate an individual’s present standing on a criterion.

Predictive validity
• The test predicts future performance.

• To assess a test-taker’s likelihood of future success.

2.1.2 Reliability

Reliability

• The ability of the test to be repeated and produce consistent results.

Think about stepping on a weighing scale and getting a different reading every time. Would that
scale be reliable? The scale readings would be pointless as they cannot be interpreted in a
meaningful way. A reliable assessment will produce the same results for similar groups of
students although marked by different examiners.

These are some ways to design a reliable assessment:

Keep
instructions
simple and

provide
examples.

For testing Designing a Use language
productive skills reliable that is similar to

(writing or assessment the daily
speaking), have language used in
two examiners
and use standard class so not to
written criteria. confuse
students.

Ask a fellow
teacher to try
completing the
assessment

before
administering it to

the students.

There are various factors that contributes to the reliability of a test such as:

Student-related reliability • fatigue, sickness, anxiety

Rater reliability • subjectivity, bias and human error

Test administration reliability • the conditions in which a test is administered

Test reliability • the nature of a test.
- long tests can cause fatigue

Reliability can contribute to validity. One commonly used illustration for validity and reliability is
archery.

A test is valid and reliable in the context of:
1. the students who sit for it
2. the setting in which it is administered

The appraisal of examiners and the fairness to students can be safeguarded by the
implementation of well-characterized assessments. Therefore, it is essential for test scores to
have high degree of validity and reliability to ensure fairness to the students and to maintain
institutional integrity.
2.1.3 Fairness

Fairness is one aspect of the four Principles of Assessment. For an assessment to be fair, it
should not discriminate against any subgroup of students or provide advantages to other groups.
It should also be fair to those who rely on the results:

assessment should be achievable by all

there should be a range of assessment methods

no bias towards any student

consistent, timely and constructive feedback should be provided to students

Teachers need to keep fairness in mind when designing assessment. Assessment is fair when
the assessment process is clearly understood by the students and agreed by both teachers and
students. Fair assessment also addresses the students’ needs and characteristics.
Understood and agreed by the students
The assessment process should be explained to the students to ensure they understand and
agree to the process. Teachers can also check and confirm if students need any support in
understanding the requirements of the assessment.
Addressing students’ needs and characteristics
It is recommended that teachers make reasonable adjustment to meet the students’ needs and
characteristics. If the students have special needs and characteristics which require reasonable
adjustments, then teachers must ensure that they:

utilise a range of determine the
assessment adjustments to be
methods
made by
consulting the
students and if
necessary, a

specialist

review the clearly
assessment and document the
determine that adjustments
adjustments will made as part of
not compromise the assessment

the outcome record

REMEMBER ! Testing writing with a question where students do not have enough background
knowledge is unfair.

Testing speaking where they are expected to respond to a reading passage they
cannot understand will not be a good test of their speaking skills.

2.1.4 Flexibility

Flexibility in assessment involves consideration of the various needs of the parties involved in
the assessment process. When designing an assessment tool, teachers need to:

identify the target contextualise the assessment to students’ realistic environment
group and in real-life application

(designing tasks which require students to apply the knowledge
learnt in real-life situation)

Flexibility in assessment can further motivate students and engage them in the learning process
by giving them some control over their learning. There are multiple ways to be flexible with
assessments while challenging students. Described below includes flexibility in assessment
timelines, weighting, and formats.

Incorporation of flexible deadlines and timelines in several ways, such as:

• allowing students to choose their own deadlines from a set of options or within
a set of criteria.

Timelines • for an assignment tied to weekly course content, having students choose which
week’s content they would like to focus on.

• creating an online assessment that can be completed over a window of time,
offering multiple attempts to complete an assessment, and/or allocating more
time to all students to complete an assessment.

Another aspect of flexibility is in the weighting of assessments. Flexibility in
weighting might look like:

• offering several quizzes/tests and excluding the lowest mark from the final
grade, providing optional quizzes for students who would like to reduce the
Weighting weighting of a final exam, or adding the weight from a missed or poorly
completed test to the final exam.

• allowing the option of completing 2 smaller assignments or 1 larger
assignment.

Many format variations can be offered to encourage and support students’
individual passions and strengths. For example:

Format • allowing an assignment to be submitted as a written text, podcast, video, or
oral presentations (Fuller, Healey, Bradley & Hall, 2004)

• offering short answer questions as an alternative to multiple choice questions

• challenging students to design a campaign of their choosing to share what
they’ve learned from the course with the broader community.

Principles of Assessing Receptive and Productive Skills

The ability to use English in a variety of contexts involves multiple language skills and therefore
testing the four skills enhances the accuracy of a test. It is common for language abilities across
the four skills to be interrelated. However, such relationships are not strong enough to allow
measurement of one skill to substitute for another. Since we emphasise the importance of testing,
it is crucial for us to do it right.

2.1.5 Principles of Assessing Receptive Skills

In order to assess students’ receptive skills, teachers need to develop assessment tasks that
allow students to demonstrate real-world listening and reading skills that will be useful in
interactions outside of the classroom. We listen and read to do something: to engage socially, to
complete tasks, or to find answers to our own questions. These purposes can be adapted when
developing assessment tasks, to provide students with relevant and motivating opportunities to
demonstrate comprehension.

What are we trying to How do we assess receptive skills?
measure?

The students’ ability to Deciding what to assess
receive, decode and Selecting appropriate text
comprehend texts in the Selecting appropriate assessment format
target language using
listening or reading skills.

This involves processing
information, constructing
meaning with the help of
prior knowledge and
displaying
comprehension.

Assessing only listening or reading abilities
and nothing else

Obtaining reliable scores

DECIDING WHAT TO ASSESS
Consider the subskills involved:

Reading:
- identifying main idea
- identifying supporting details
- understanding text organisation
- differentiating fact and opinion

Listening: - listen and predict
- listening for gist - selective listening
- listening for main ideas
- listening for details
- listen and infer

SELECTING APPROPRIATE ASSESSING ONLY
ASSESSMENT FORMAT LISTENING OR READING
ABILITIES AND NOTHING
- multiple choice questions ELSE
- sentence completion/short answer
- information transfer - avoid assessing other
- True/False/not mentioned elements such as creativity,
spelling and grammar

SELECTING APPROPRIATE TEXT OBTAINING RELIABLE SCORES
• text type:
- reports - What is the passing score? How do you decide?
- advertisement - Is the marking scheme complete and accurate?
- story/poem - Is there only one correct answer?
- speech - Are the examiners reliable?
- Do examiners agree to ignore errors in mechanical
• topic of the text accuracy? (spelling, grammar)
- unbiased
- uncontroversial
- is cultural/background knowledge
needed to comprehend the text

• length of the text & time allocation
- suitable according to students’
level

• lexis/grammar/speed (listening)
- suitable according to students’
level

2.1.6 Principles of Assessing Productive Skills

In order to assess students’ productive skills teachers will need to develop assessment tasks that
allow students to demonstrate real-world speaking and writing skills which will be useful in their
interactions outside of the classroom.

What are we trying to How do we assess productive
measure? skills?

The students’ ability to Setting appropriate tasks
interact in the target
language in spoken or Assessing only speaking or writing
written form. abilities and nothing else

This involves
comprehension as well as

production (Hughes,
1991).

Restricting students’ responses

Selecting appropriate assessment
format

Obtaining reliable scores

SETTING APPROPRIATE TASKS ASSESSING ONLY SPEAKING OR
WRITING ABILITIES AND
Consider these questions: NOTHING ELSE

• What is the communicative function I want to elicit Avoid the assessment of:
from the student? (expressing… giving…
describing…) • opinions
• creativity or imagination
• What is the form of interaction expected? (letter, • general knowledge
postcard, email, voice chat, interview, oral • other skills (reading, listening)
presentation)

• What is the context for the interaction? (formal,
informal, real, unreal, school context, the street,
home, abroad…)

RESTRICTING STUDENTS’ RESPONSES SELECTING APPROPRIATE
• define tasks in detail (telling exactly what ASSESSMENT FORMAT
you expect from the students)
Formats for speaking:
• establish boundaries (controlling ideas
that will stop students from going too far • oral questionnaire
astray) • free interviews
• peer-interaction/role-plays
• specify the minimum requirements for • picture-cued tasks (describing a
production (number of pages, how many picture, structuring a story, giving
minutes) directions, giving instructions)

• provide information or cues in the form of Formats for writing:
notes, examples or pictures • essay questions
• guided writing (a letter, postcard,
email, note, form, advertisement)
• picture-cued tasks (describing a
picture, structuring a story, giving
directions, giving instructions)
• recount/retelling a story, a situation

OBTAINING RELIABLE SCORES

Determine a well-structured criteria based on descriptors, for
example:

• 39-50 marks: Excellent idea organization & language use
• 26-38 marks: Good idea organization & language use
• 13-25 marks: Average idea organization & language use
• 0-12 marks: Poor idea organization & language use

Have at least two examiners to mark/evaluate

Reference:

ALTA Language Service. (2020). What are validity and reliability? Retrieved from
https://www.altalang.com/language-testing/validity/

Bosson, A. (2013). Testing receptive skills-reading. Retrieved from
https://prezi.com/hompl8c3biuo/testing-receptive-skills-reading/

Chambers, D. (2016). Basic principles of assessment: reliability and validity. Retrieved from
https://slideplayer.com/slide/5299746/

Cornerstones of assessment: assessment and international exams in TEFL. (n.d). Retrieved
from https://en.ppt-online.org/324796

Craig, L. (2018). Language assessment. Retrieved from https://slideplayer.com/slide/11981919/

EnglishPost.org. (2020). Reliability, validity and practicality. Retrieved from
https://englishpost.org/reliability-validity-and-practicality/

Galaczi, E. (2018). Benefits of testing the four skills (reading, listening, writing and speaking).
Retrieved from https://www.cambridgeenglish.org/blog/benefits-of-testing-the-four-skills/

Integrating CLB in assessment into your ESL classroom. (n.d). Developing receptive skills
assessment tasks. Retrieved from https://iclba.language.ca/chapter-4-developing-
receptive-skills-assessment-tasks/#top

Integrating CLB in assessment into your ESL classroom. (n.d). Developing productive skills
assessment tasks. Retrieved from https://iclba.language.ca/chapter-3-developing-
productive-skills-assessment-tasks/

Magdalena, J. (2009). Testing productive skills. Retrieved from
https://www.slideshare.net/songoten77/testing-productive-skills

Malaysian Examination Syndicate. (2019). Modul e-Kompetensi Pentaksiran Guru
(e-KOMPeG). Retrieved from https://app.schoology.com/course/2315182463/materials

McMaster University. (2017). Flexibility in assessment. Retrieved from
https://flexforward.pressbooks.com/chapter/flexibility/

Welton, S. (2017). Principles of assessment – part 2 (fairness). Retrieved from
https://ittacademy.net.au/principles-of-assessment-part-2/

Welton, S. (2017). Principles of assessment – part 3 (flexibility). Retrieved from
https://ittacademy.net.au/principles-of-assessment-part-3/