Test Difficulty in Speaking Tests

A.    Background of study

The difficulty of speaking tasks has only recently become a topic of investigation in language testing. This has been prompted by work on discourse variability in second language acquisition (SLA) research, new classificatory systems for describing tasks, and the advent of statistical techniques that enable the prediction of task difficulty.

B.     The supporting Theory

1.      In second language acquisition (SLA) research the classification of tasks in order to better understand their impact upon language learning dates to the early 1980s (Crookes and Gass, 1993: 1–2).

2.      The only language testing studies to find large significant differences between how learners perform on tasks are those where the tasks are maximally different (as in Bachman et al., 1995; Fulcher, 1996b) and employ multiple rating scales.

C.     Approaches to task difficulty in second language acquisition

The assumption underlying present SLA influenced approaches to studying speaking tasks is that there is variation in test-taker performance by task characteristics or conditions, and that this variation leads to different scores (or estimates of speaking ‘ability’) under different conditions.

D.    Approaches to task difficulty in speaking tests

Skehan has suggested various (psycholinguistic) categories that will affect task difficulty:

·      Familiar information: The more familiar the information on which a task is based, the more influent the performance will be.

·      Structured tasks: Where the task is based on a clear sequential structure there will be significantly greater influency and accuracy.

·      Complex and numerous operations: The greater the number of online operations and transformation of material that are needed, the more difficult the task. This may impact upon greater complexity, but at the expense of accuracy and fluency.

·      Complexity of knowledge base: The more open the knowledge base on which a task draws, the more complex will be the language produced.

·      Differentiated outcomes: As a task outcome requires more differentiated justification, the complexity of the language will increase.

E.     A pragmatic approach: rationale and assumptions

It is concerned with how speakers perform differentially across tasks when pragmatic task conditions are systematically manipulated. This approach links the concept of task difficulty to the cultural and social demands made by the task and – by systematically manipulating pragmatic task features to increase or reduce the demands of the task – it is possible to investigate how difficult the task is for different L1 speakers. L1 (their language) cultural  background and pragmatic task features have an impact upon task difficulty.

F.     Politeness in pragmatics

The concept which indirectness and politeness are virtually indistinguishable, but the model is common in pragmatics: negative politeness is about making the option of refusing a request more acceptable.

(See table 1)

G.    Method

1.      Tasks and administration

The first task required the student to approach a professor and ask to borrow a book that they needed to write an assignment. The second asked the student to imagine that they worked part time in an office, and had to leave for 20 minutes to do a job outside. In the third task, the student was asked to imagine that they were moving accommodation, and had to ask a neighbour to help move luggage and furniture. In the next task the student was on a bus with a child and needed to request that a passenger move seats so that they could sit next to the child. In task 5, the student was asked to imagine that they had run out of money and needed to ask for a pay advance from their boss at the place where they had a part-time job. The final task required the student to ask a newcomer at work (subordinate) if they could borrow their laptop for a morning.

2.      Subjects

Spanish-speaking students : 23 and 32 years old

English-speaking students : 18 and 24 years old

3.      Scoring

After two weeks, all the students were also asked to complete a questionnaire for each of the six tasks, in which they were asked to assign a probability of success to samples of requests for the situation drawn from the performances on the test.

4.      Reliability

The correlation acts as a reliability check in the research, as the students are essentially being asked to make the same judgement twice using two different methods.  it indicates that the estimates of success were consistent across a short period of time and across two methods of eliciting judgements of success from the speakers.

5.      Analysis

A simple comparison of means for the categories of L1, social power and imposition.

(see table 2)

H.    Result

The results indicate that the task conditions of social power and imposition are significant, and the L1 background of the test-takers is also significant.

Situation 1:

English: I was just wondering if you have the book and if I could borrow it?

Spanish: Me puedes prestas el libro?

               ‘Could I borrow the book?’

Situation 3:

English: I was just wondering if you had some spare time on your hands,

  could you help me?

Spanish: Te importar‡´a ayudarme con la mudanza?

   ‘Would you mind helping me move?’

Situation 4:

English: A: Excuse me I’ve got a child with me I was wondering would it

be possible if we could sit in these two seats and you could

move to another place?

               B: Umm I yep certainly I’ll just get up and move round to where

there’s another space

Spanish: A: e:::h Iperdone I e:::h I no le importa sentars:::e Ien otro lado?

I para yo poder sentarme con el nin˜o i en el mismo asiento?

(‘Do you mind sitting somewhere else?’)

              B: no Ino me importa I sie´ntese

              A: ah I vale muchas gracias

Situation 6:

English: Is there any chance today that it might be possible for me to use

your laptop for a while?

Spanish: Me prestas el portatil hoy?

   Can I borrow your laptop today?’

I.       Conclusions

English and spanish have different culture. Sometimes, there is a sentence that in english is impolite but in spanish it is polite. This is influence the social power and imposition. Whereas L1 (the first language), social power and imposition influence the task difficulty. So, L1 (the first language), social power and imposition have an important role for this.

Lampiran :

Table 1. A pragmatic scale                                        

 

1)  Answer the phone                                                  Less indirect Less polite

2)  I want you to answer the phone

3)  Will you answer the phone?

4)  Can you answer the phone?

5)  Would you mind answering the phone?

6)  Could you possibly answer the phone?

7)  etc.                                                                         More indirect More polite

Table 2. First language as a main effect

Situation                                  English    English    Spanish    Spanish    Cohen’s

Mean       sd            mean         sd              d

1) borrow book                       6.26        2.09          6.93          2.36         .30

2) cover telephone calls           7.65        1.76          7.84          1.92         .10

3) help with moving                5.60        2.13          6.56          2.16         .45

4) swap bus seats                    7.03        1.84          7.74          1.92         .38

5) ask for pay advance            5.46        2.01          5.24           2.46        .10

6) borrow laptop                     6.11        1.98          7.26           1.92        .59