|
||||||||||||||||
Who Cares?for Test Administration instructions click here Why Use Who Cares?Who Cares? increases the probability of making effective selection decisions by providing job relevant information. Who Cares? assesses the key knowledge, skills and attitudes which predict effective performance in care settings. And so it provides a useful supplement to other sources of information which may be available about the candidate. Identifying individuals who are most likely to provide the level of care required by service users is an important part of the Home Manager's role. Mistakes can be costly in terms of wasted resources. But selection mistakes also have an impact on care users, their families, other members of the team and ultimately even on the reputation of the Home. Obviously, the Home Manager can utilize traditional assessment methodologies and interview applicants. Or, he or she can utilize other sources of evidence such as the career history of the candidate. However research examining the effectiveness of interview assessments of future performance indicates our impressions of candidates, formed very quickly during interviews, can be misleading. And the research which tests the effectiveness of references suggest these provide information which is little better than chance at identifying the most effective applicant. Additional versions of Who Cares? have been designed for a number of organisations in the care sector to more specifically simulate job functions. Rationale and Development Rationale - Who Cares? is a psychometric test. The word psychometric has its roots in the Greek words psyche and metric. These literally translated mean the mind and measurement. One useful distinction to make between psychometric tools is tests of function and tests of trait. Test of traits - personality and intelligence for example - are rooted in the European tradition of assessing broad, underlying qualities which are assumed to represent fundamental ways in which we differ. These qualities are considered sufficiently pervasive to be useful in predicting job performance across a range of job functions. So differences in effectiveness in care workers might be systematically related to differences in, for example, their sensitivity, affiliativeness (warmth) and rule consciousness. The relationship between the first two would be anticipated because we assume effective care workers ought to identify with residents and their needs. We would anticipate a relationship between rule consciousness and effectiveness because effective care workers will be those who are willing to respect systems, procedures and frameworks such as care plans and be prepared to work within them. The problem with a trait approach is the relationship between trait scores and differences in effectiveness is likely to be at best quite modest. This is because they are, by definition, measures of broad underlying traits. They are useful at indicating an individual's status at a level of psychological generality. But they are less good at predicting very specific behaviours such as those which might differentiate the effective and less effective care worker. The function approach, much more prevalent in the United States, attempts to bring the assessment much closer to the job function. Who Cares? is very much a function test. It attempts to capture the knowledge, skills and attitudes which differentiate the effective from the less effective care worker. Development / Job Analysis The Who Cares? test is a series of questions which have been designed on the basis of a thorough job analysis of care workers' duties and responsibilities . The job analysis provided the basic information about the nature of the care worker role. However, it was necessary to complement this task focus with techniques such as critical incidents and repertory grid which provide more person centred information. This enabled a thorough exploration of what personal qualities differentiate the effective from the ineffective care worker and the events and circumstances where these differences were manifested. Critical incident interviewing is based on a technique developed during the Second World War in the air force. It was used to explore the underlying causes of times when things went particularly well or particularly badly during bombing raids. And although it has its roots in a military setting it is equally applicable in organisations. In a care setting what kinds of incident mark an individual off as being particularly effective? Conversely what kinds of behaviour are particularly inappropriate and suggest an individual is ineffective in the role? Similarly, repertory grid requires a job expert or manager to articulate the key distinctions and differentiations he or she would make between care workers. This is accomplished by presenting him or her with a series of three names of individuals. With each triad of names the manager is asked, "tell me one important way in which any two are the same and different from the third." This prompts the manager to reveal a way he or she would differentiate between individuals. And the impact of these differences on the home, particularly the care of service users, and the contexts in which these differences are manifested can be explored further. So rather than testing for underlying traits such as sensitivity, Who Cares? is intended to measure the knowledge, skills and attitudes which are more closely related to the differences between Support Workers' effectiveness. In addition, in being based on a job analysis the test would realistically reflect the day-to-day experiences of a Support Worker. Questions are therefore mostly scenario based and where they are not they tap the key attitudinal differences identified in the job analysis. In addition to being a measure of function, Who Cares? is also, to use another of the general distinctions often made to differentiate between tests, a measure of Maximum performance as opposed to a measure of Typical performance. This means there are right and wrong answers. Or more accurately there are behaviours and attitudes which are deemed by a consensus of home managers to be appropriate and inappropriate. The correctness of responses was based on the critical incidents which emerged during the job analysis. Question Trialing The critical incident and repertory grid interviews with home managers provided a very sound basis for question or item construction. Not only did this process reveal the key differences between effective and less effective care workers it also indicated the contexts in which these differences reveal themselves. An initial pool of questions was constructed and trialed with a group of care workers and members of the general public. As a result of the initial trialing, items were removed which did not discriminate effectively. In other words, where the correct response to items was either too obvious or too obscure- resulting in either very low or very high rates of correct or incorrect responses - the question was removed. This meant questions where the correct answer was simply the most socially desirable were dropped, ensuring Who Cares? is resistant to the effects of distortion. What Who Cares? Really Measures Whilst Who Cares? is not a test of any well defined psychological trait the question of what underpins the Who Cares? test is still nonetheless a pertinent one. One way of exploring this issue is Factor Analysis. This technique, developed early in the last century but refined constantly by successive generations of statisticians and psychologists, offers a way of identifying deeper structures in data. It was, for example, the basis for the claim that intelligence was essentially one thing or factor. This is because measures or tests which involve intelligent functioning of one form or another- using words or numbers, interpreting diagrams all seem to correlate with one another. If an individual performs well on one kind of activity the chances are he or she will perform well on another. So a statistical factor is a way of summarizing consistencies in data- patterns of association which are observable and which are manifestations of deeper structures. If we Factor Analyse responses to the Who Cares? inventory three interpretable factors appear which indicate what the inventory is actually measuring. Factor 1 Professionalism - High Scorers on this factor understand the importance of working within care plans. So this factor can be interpreted as evaluating the candidates professional attitude, responsibility, and clinical reasoning in complex situations. For example, respecting a care plan for a Service User even if this involves denying the gratification of the Services User's immediate needs. Factor 2 Service User Orientation - High scores on this factor like being of service to Service Users. They are willing to subordinate their own interests to those of the Service User. They are those who display empathy, tolerance and responsibility towards Service Users. Factor 3 Organisational Awareness - High scorers on this factor are those who recognize their wider responsibilities to the home and organisation. They are those who realize they need to work with other support workers to achieve collective goals. They are aware they are part of a team. These three Factors measure key aspects of the Support Workers role. They indicate the test is not measuring a unitary trait but three important ways in which effectiveness can be manifested and differentiated in the Support Workers role. The Final Edition Items were retained if as suggested above they appeared neither too easy nor too difficult. They had to pass another two basic tests of fitness I. They were not affected by ethnicity or gender II. They correlated with the final score. Questions which did not seem to fit with other questions were dropped. The remaining questions formed the final 'trial edition' which was then standardised to establish the distribution of scores. The test was standardised by selecting a large range of Care Workers from different organisations. There was also a group of individuals included who were not involved in care work. The samples were of mixed age and gender. (Mean age = 42, SD = 11.5, gender = 82% female) Validity The concept of validity in more ordinary English addresses the issue of relevance. Validity data is essentially any data which enables us to assess the meaning and the relevance of a test score. There are several ways of attempting to answer this question. One easy way to assess the validity of a test is simply to look at it. Does an initial inspection of the test suggest it is relevant to the context in which it is supposed to be used? This form of validity is sometimes referred to as face validity. This form of validity is particularly important in securing the cooperation of candidates. If a test is face valid candidates can understand immediately why it is part of the selection process. It encourages rapport and motivation. And importantly it reduces the chances of challenge to the process. A Face Valid test by definition is likely to be perceived as defensible. Who Cares? easily passes the face validity test. In being based on a job analysis its questions are obviously grounded in care work. Similarly, the origin of Who Cares? will mean it is more likely to pass another somewhat more complex form of validity - content validity. In content validity there is an initial assumption of a 'domain of skills' which describe the Care Workers role. The notion of content validity concerns whether the questions adequately cover this 'domain' or 'universe' of skills. It also covers other issues such as whether there is distracting content, a very high reading difficulty level, for example. Other forms of Validity include construct and criterion. The former includes data which helps interpret and refine our understanding of what is being measured. The identification of three interpretable statistical 'factors'- described above- provides an important insight into what the test measures. The relationship between these three factors is as follows:
As the table shows, factors appear to be relatively independent of each other. The smallest relationship, indeed a negative one, perhaps as would be expected, is between Service User Orientation and Professionalism indicating the tension which sometimes arises between on the one hand identifying with service users responding to their needs, and working within care plans which might involve not gratifying some needs - for instance if a service user is overweight and the care plan is attempting to address this issue. However the stronger relationship between service user orientation and organizational awareness is to be expected because they are both manifestations of the extent to which the candidate understands how to relate effectively to others both residents and co-workers. Criterion validity is probably what most people assume validity to be about - the ability of the test to predict real life outcomes. At present the criterion related information is, as is often the case with a new test, is tentative. The test was able to discriminate between a sample of 73 Care Workers rated as effective and adult members of the general public. The probability of the difference in scores being due to chance factors was less than one in a thousand. Using discriminant function analysis which predicts membership of a group from test scores Who Cares? was able to predict correctly 97% of the time which group individuals belonged to. On-Line Administration It is possible to administer, score and interpret the Who Cares? test online. For this option the key features of effective test administration are the same as with the traditional paper medium. The candidate has to understand how the exercise fits into the process, has to be motivated and interested. And so the value of time spent setting the scene cannot be overestimated. The main difference on-line is the candidate is sitting at a computer terminal rather than at a desk. It important the candidate is free from interruption and noise. How To Use the Test Results Interpretation Who Cares? On line instantly produces a report both for the organization and the candidate. The report consists of what is termed normative information - your candidate compared with other care workers. This normative information is provided for each of the scales and the overall score. Being based on a much larger sample of the candidates knowledge skills and attitudes it should be given more weight. In fact, if screening applicants ahead of interview the total score is sufficiently robust to use it to select those interviewees to bring forward for interview. The graphic conveys information about candidates in terms of percentile ranks. Percentiles indicate the percentage of candidates who fall below a particular score. A score which falls on the 65th percentile, for example, means 65% of the sample group got a score the same or worse then this candidate. And 35% of candidates would do better. A typical candidate will achieve a percentile between 16 and 84. Percentiles have the advantage of being readily understood and being universally applicable. Users should remember, however, that percentiles are not equal units. Differences between percentiles around the median are somewhat exaggerated, and at the extremes are therefore understated. The norm group against which candidate scores are interpreted is a sample of 135 individuals employed in care work. Who Cares? Narrative The three factors in Who Cares? - Service User Orientation, Professionalism and Organisational Awareness. Besides giving an overall score, the test is designed in factors to give an identifiable profile and give relevant areas to probe in interview. Organisational Awareness As stated previously, this factor is concerned with the candidates awareness his or her work is part of a team effort. The items in this factor test the candidate by identifying his or her probable course of action in situations where an understanding of the impact of actions on the entire home is necessary. It also involves evaluating the candidates deference to authority as well as knowledge of policy in this area. Professionalism This factor comprises items containing situations which require good organisation and prioritisation and a willingness to work within care plans. Correct choices indicate the skill to be thorough, calm, and a responsible part of the team. However, a high score in this area can also indicate unwillingness to help other team members. Service User Orientation Questions here assess the candidates interest in service users. High scorers are those who identify the best course of action to help Service Users. High scorers are those who are generous with their time and so willing to sacrifice their own interests to help others, even when it is not part of their job. It also includes candidates with good attitudes towards caring and conducting their tasks responsibly. for Test Administration instructions click here
|
