Test Administration

Please note: The information below assumes the physical presence of an administrator with the candidate. If this is not the case then the candidate will need to be talked through the points below over the phone or maybe receive an email from the company explaining the purpose of the test. The phone call/email should include points such as the need for the candidate to do the test in an appropriate environment and without distractions. It is also helpful to explain a feedback report is available.

Poor administration of tests undermines the value of test scores and thus makes interpretation difficult.

The link between high quality administration and high quality scores is represented by the formula:

In other words a candidate’s score at the end of a test represents some combination of his or her true standing on that aptitude, test error and administration error. Well constructed tests such as Business Intelligence are very carefully designed and introduce only an small amount of error. This means the main barrier to establishing an individual’s true level on an attribute such as numerical reasoning is poor administration. Anything said or done during the administration of a test which causes confusion, anxiety, hostility or uncooperativeness in candidates increases the proportion of Ead relative to Ts found in Sty.

Administration Skills

A test session divides into parts - Rapport Building and Administration Script.

It is difficult to overestimate the importance of five minutes spent motivating and relaxing candidates.

Assessment as a Social Relationship

Like interviewing, testing also involves a complex social relationship. Both tester and candidates bring a set of differing expectations into the situation.

Setting the Scene

Careful preliminary work is the key to successful administration. The object of testing is to extract an individual’s true score (as close to the true score as is achievable by the test) on an attribute or set of attributes.

To achieve this a tester must standardize the situation with respect to:

• Motivation
  
• Expectation of success
  
• Interpretation of the task
  

In other words, candidates must be willing to do their best, must not fear the test, and must understand what is expected of them.

Always convey to candidates the impression that they are more important than the tests and test scores - This should be evident in your tone of voice, your polite manner, and your willingness to deal with questions.

Conduct a short discussion about tests - This will enable you to say something about the nature of testing. You can, for example, stress the difference between the sort of tests taken at school and psychological tests - the former being assessments of how much an individuals knows, the latter being a assessment of what someone’s potential is.

Ensure confidentiality - Assure candidates that access to test scores is restricted.

Explain why tests are used in the company - It is usually worth saying to candidates that tests help make selection fairer. They put people on an equal footing and are objective. It is also worth stressing not just the general benefits of testing but the specific purpose of the test. In other words stressing the relationship between the demands of Business Intelligence and the job. For example:

“As you know one aspect of the job you are applying for involves dealing with tables containing information about sales. This test assesses how easy you will find that part of the job.”

This increases the face validity of the exercise.

Explain how tests are used in the company - It is usually the case that tests are used alongside other pieces of information - education, experience, references and interview. Test scores form a part of this jigsaw. For example:

“As you will be aware in order to make the fairest decisions we are collecting a good deal of information over the next few hours about whether a career with us is right for you. This exercise forms a part of the assessment and sits alongside all the other evidence of whether working with us is the right career move for you.”

Environmental considerations

Additionally, you will need to consider these obvious, but frequently neglected requirements.

Adequate space and computing facilities for candidates - There should be room for candidates to complete the exercises without their responses being seen by other candidates, and without their being able to see the responses of others. All candidates should have a PC which is in good working order, i.e not likely to “freeze”, crash or slow down during the test. In addition to this it is imperative you ensure that all candidates have their user id and passwords to hand, as they will need this to gain access to the test which you have registered them for. All candidates should have a calculator.

Adequate heating, lighting and ventilation - Candidates will not be able to do their best if the room is too hot, too cold, too stuffy, or if they cannot see properly.

Adequate toilet facilities - Candidates should not however, leave the room during the test.

Freedom from interruptions - Candidates should not be distracted by people entering and leaving the testing room, noisy groups outside or telephones ringing. Make sure mobile telephones are switched off.

Adequate acoustics - Make sure that people at the back of the room can hear you. Nervousness can make us hear things less well, as well as making us less willing to admit that we have not heard something.

Problems with scene setting

Failure to create a rapport and to motivate candidates will amplify what is termed test anxiety in candidates. Behaviourally, test anxiety may manifest itself in the following ways.

• Inattention during the giving of directions
  
• Lack of co-operation
  
• Refusal to take the test
  
• Fault finding
  
• Failure to complete the test
  
• Restlessness
  

Where this occurs, ‘repair work’ with the group will obviously be essential if the exercise is to be a valid one. It is possible to describe a few general problems which undermine the tester-candidate relationship. Having read through them you should consider which sort of tester you are likely to be and which problems, if any, are likely to apply to yourself.

Tester is unwilling or unable to offer adequate emotional support to candidates - This is perhaps the least common problem but is not unknown. It occurs when the tester, for whatever reason, uses the testing situation to improve his or her own self-image. An inadequate tester perceives a testing session as an opportunity to improve his or her own self-image at the expense of the relationship with the candidate, usually by comparing the candidate unfavourably against themselves. Where this is happening, behaviour towards candidates is likely to be characterized by one or all of the following: hostility, rudeness, contempt and aloofness. Obviously, these behaviours can only increase a candidate’s test anxiety, and reduce motivation and the expectation of success. The key problem here is not creating an atmosphere where candidates feel they can ‘own up’ to getting an example wrong. This means candidates start the test at a severe disadvantage.

Tester may offer too much support - This stems from testers having a very high need for affiliation. This means it is essentially the relationship element of work that motivates individuals. Testers whose need for affiliation is higher than other work related needs may see testing as a threat to their need fulfilment. Having developed a relationship with candidates you may see the testing sessions as an obstacle to developing this relationship further. This may cause you to violate the requirements of testing procedures by:

Adding to the ideas contained in the test instructions - It is imperative that, once scene setting has ended, no further dialogue is added to the standardized instructions contained within the test direct system. This is because the norms - scores against which you will be comparing the scores of your candidates - were collected using tests administered exactly as described in the test direct system. If you add to, or alter these you are, in effect, creating a different test, and cannot then legitimately compare your candidate’s score with those found in the norms tables.

Downgrading the importance of the exercise - You may feel tempted to play down the exercise - ‘it’s something we ask you to do, don’t worry about it - we don’t take too much notice of the results’. If you do not feel confident about the value of the exercise, candidates will undoubtedly also see it as a waste of time. You will, in addition, be denying candidates access to valid test results which could otherwise be of value both to the organisation and to themselves.

A central objective of this session is to make you feel confident about the value of testing, and committed to high standards of administration. To the extent that you are both confident with and committed to testing, the problems outlined above will diminish.

Tester offers support differentially - We all have ‘blind spots’ - people we find it difficult to like or relate to. When we encounter these people in our testing sessions we may be less willing to provide them with support, and reduce their test anxiety. Analyse your own attitudes to determine which types of people constitute your own blind spot. Should you encounter candidates like this in a testing session, make doubly sure you are not treating them differently. For example, make sure you answer their questions as adequately and politely as you do questions coming from people you view more favourably.

Critical Incidents - When things go wrong - It is not possible to provide a complete list of possible critical incidents; nor is it possible to prescribe how to deal with each of them. There is always an element of ‘thinking on your feet’ when testing. The occurrence of critical incidents should, however, be kept to a minimum if you adhere to the guidelines described. Additionally, it is always worth indicating that you are willing to discuss points which cannot be discussed before or during the test at the end of the session.

Administering the test

Having set the scene your next task is to administer the test. It is imperative that you follow the instructions exactly as given in this manual. This is an essential requirement. Once you have finished setting the scene and candidates have started the program you must not add any further instructions. If you do not you are creating a different test. Remember, the norms data for the test were collected using the test instructions exactly as they are in the test direct system. If you add or omit something you will not only be unable legitimately utilise the norms contained within the test direct system. To illustrate this further, imagine presenting Business Intelligence The norms tables have been constructed by testing groups of subjects who have not been advised about guessing. We assume that, like other factors, the propensity to guess -how sure an individual needs to be before guessing a response - is normally distributed (in other words, that some individuals will rarely if ever guess; others will guess very frequently; and most people will guess occasionally). Now, if a candidate in your group asks whether it is all right to guess, and you say that it is, you will have significantly changed the characteristics of your group compared to groups contributing to the norm table in the manual. Guessing in your group is now no longer normally distributed. People who rarely if ever guessed may be more willing to do so, and people who guessed occasionally may be prepared to guess more frequently.

To summarize:

• Always follow the directions as described
  
  
• Where questions are invited, never add ideas or information not already contained in the instructions
  
  
• Be thorough familiar with the test you are administering

Typical Questions

Is there a pass mark? - There is a mark we like to see you get - but you would not have got this far if there wasn’t a very good chance of you achieving it.

How much weight do you put on this? - The test is just part of the picture. We obviously take other information into account.

If I get one wrong do you take a mark off? - I’m afraid I cannot discuss how we mark the test.

Can I have another go if I do not do too well? - On this kind of test it does not make much difference. You may do a little bit better, you may do a little bit worse but we take that into account anyway.

I’m not feeling 100% today - If you really feel unwell don’t do it. But we do have a margin of error that allows for people not performing at their best.

Why can’t you just go on our CVs and/or application forms? - We like to supplement the interview and we find this test gives us a good idea of how well you are going to cope with certain elements of the job.

Implementing the Test

• Ensure there is one PC for every candidate and that all are in good working order.
• Ensure that all PC’s are set to the “candidate logon” part of the test direct website.
• Ensure all candidates have their user id and password and a calculator.

If this is the first or only test being used during the testing session, welcome candidates into the room and give the introductory briefing that you have prepared. Remember to include the following points

• Introduce yourself to candidates (if appropriate) and thank them for attending.
  
• Discuss the nature of the test(s) being used and explain why it/they are being used in this particular assessment situation.
  
• The length of the testing session and the nature of any breaks (if appropriate)
  
• Ensure confidentiality
  
• Give information regarding the level of feedback which can be expected
  

When you have made sure that candidates are comfortable and answer any general queries about the testing session, then say:

"From now on we shall be doing the 'Test Name' test. Please do not talk from now on. To ensure that everyone is treated equally and fairly all instructions will now be given to you after you log on with your user id and password, which were provided for at the start of the session. Please enter your user id and password now and follow the instructions given by the test direct website."

When all candidates have finished the test you may proceed to the next test (If applicable) having allowed candidates a short break. If this is the last or only test ensure that you thank candidates for their participation and answer any queries re: feedback or any other issues arising from the test taking process.