What is the pass rate for my psychometric tests?
This question was posed recently by a student to a Careers Adviser at Salford University. The Careers Adviser did not know the answer so he contacted all the other University Careers Services to gather the thoughts of Careers Advisers from all round the country. Of course, the real answer to the question begins: ‘It all depends…’ but the views and experiences from all around the UK make interesting reading. Thanks to the collected wisdom of the Careers Advisers of AGCAS (Association of Graduate Careers Advisory Services), and only read on if you are able to cope with deeply ambiguous data.
Whatever you make of all this you can get a strong predictor of your own performance in psychometrics and a chance to discuss what your results might mean by signing up to one of our practice sessions.
Views of Careers Advisers
1
A Tale of Woe
’I think the reason that this question has provoked interest is down to the lack of transparency within this aspect of the recruitment process. Many test developers will say that the tests that they develop are normative tests and that there is no pass of fail in the exam sense. All well and good but in terms of selection they are in effect being used in this manner (or at least that is how the student/graduate sees it – “They said that I failed the tests” is one oft-quoted line) i.e. the applicant has failed to reach a particular standard.
Now you can bang on about the nature of the tests being normative but at the end of the day an applicant will “pass or fail” on the basis of falling below a particular percentile or scoring above. This of course may vary from company to company and from job to job, so why can’t a recruiter just say that we are looking for individuals who score above this or that level in the tests, particularly those who test at the very beginning of the selection process?
I know that the response to this will be, “well it depends” for all the reasons listed below but I think you can see the point of the graduate who says they “failed” the tests and don’t actually know what their results were/are or received feedback on their performance – it all seems a bit mysterious.
If this bugs me – think about it from the graduate’s perspective! Feel free to disagree with this if you will but I’ve heard too many tales of woe from graduates for it not to.’
The question I asked was:
“Does anybody out there know if there is a typical cut-off score that recruiters use to select/reject applicants applying to graduate positions?
I realise that the answer is probably “Well that depends on the employer”, but does anyone have any concrete examples to offer just to get an idea of the range?
Apologies if the question sounds a little simplistic but I’ve just seen a graduate who has sat a number of selection tests and has asked the question of recruiters and got no answer. In fact, I’ve asked the question myself of some recruiters and received no answer. So…wouldn’t it just be nice to know?”
2
Pre-screening versus Assessment Centres
“I guess, as you say, it depends on how they are used.
If used prior to an assessment centre to reduce the pile of eligible applicants a cut off may be applied consistently e.g. anything less than a C on SHL numerical reasoning for an accountancy role would be rejected………… I would have thought C on SHL for any job would be typical, and for some roles it would be higher.
However, if used as part of an assessment centre, the cut off might be waived if the candidate had performed well in any other exercises measuring the same ability/competency.”
3
Top Half? Top Quarter? Bottom Quarter?
“I did a testing module as part of my CIPD (Chartered Institue of Personnel and Development) and it really does vary. I think a number use it to cut off the bottom half / quarter, some about the middle and some use it to cream off the upper quartile. SHL may be willing to provide a rough idea, but employers tend to want to keep it confidential how they are using the tests.”
4
City Firms
“I have worked as a graduate recruiter and yes it does depend on the employer. Both City businesses I worked for used the verbal and numerical tests (MGIB) and acceptable scores ranged from 40%ile to 60%ile. However, the scores in some cases were used in conjunction with feedback from interviews and assessment centres. However, in my experience they have also been used for pre-selection (i.e. don’t pass don’t get an interview)”
5
One Firm’s Practice
“I know a couple of years ago Hilton International were using Gap3 SHL tests,numerical reasoning and verbal reasoning and had a cut off of 30% on one and 50% on the other, but didn’t mind which was which.”
6
Commercial Confidentiality
“My own view is that any cut offs should be absolutely individualised to the Company/organisation’s needs. That being the case differing cut offs say in SHL material could be regarded as commercially confidential, although I do
recall seeing someone who had allegedly failed a Maths test with a large firm of accountants because he hadn’t achieved the 95th percentile!”
7
Case Law
“Most reputable firms who understand testing will base their “cut-off” point, at least initially, on performance against norms. The “pass mark” therefore depends as much on the norm group being used for comparison as it does on the employer. So if you are applying, say, to a firm of actuaries that is using norm data specifically from trainee actuaries, you may be competing with a much more able norm group than at a firm where very general graduate norms are used. Whilst eventually a “pass mark” may emerge, for reputable firms it normally has its basis in performance against a norm.
The stage in the selection process that tests are used is also
significant. If you are applying to firms who only test at final selection stage, tests will be a far less significant part of the process than when tests are used to “screen” applicants early in the process (however, even in this situation, employers are often only looking for average scores).
How norms are used, varies enormously from firm to firm, but on the few occasions when I have managed to find out about the policies of individual firms, I have found that most firms are only hoping that candidates will perform at about the 50th percentile level - referring to the earlier point, this obviously depends hugely on the norm group being used.
I have found that where tests are used only at final stage, employers will often bend over backwards to help candidates who have performed well in all other aspects of the process.
I have dealt with several (at least six) candidates for jobs with big accountancy firms who have been sent away to practice, having performed appallingly in the tests but well in everything else.In all but one case, the candidates did lots of practice, retook the tests, did just as badly (as you’d expect them to!), but were offered jobs anyway!
8
Bank Recruitment
“In a prior life I used to be a graduate recruiter for a major clearing bank. Although that was some 5/6 years ago, at that time, we did not impose a cut-off point for psychometric tests – the tests were merely part of the process and results were looked at alongside other evidence, application
form, performance at first and second interviews etc.
There was no pass/fail mark for the tests themselves but the results were used either to raise queries about candidates with the interviewers or were looked at for confirmatory evidence for issues that became apparent in other parts of the assessment process.”
9
More Confusion
“If its any use I can tell you that I used the 20th percentile as the cut-off when I was recruiting, but I heard one employer recently say that they looked for a minimum of 60th percentile.”
10
Variations in same firm between functions
“Don’t know if what I’ve been told is ‘typical’ and they might vary across function and each year but three examples I know of were all at 40th percentile for verbal and numerical.
These were big graduate recruiters using SHL tests or very similar to GAP series 3. All general business functions and three different industries so seems to be a good guide though I do stress to students that the same companies may look for higher scores for some specific functions. And that
I’ve never heard what investment banks and management consultancies use for their cut-offs so I’d imagine they’d be higher or hard to compare as the numeracy tests are often harder than GAP series.
Of course also depends on when the tests are used and the candidate. When I visited a confectionery company a couple of years ago I don’t think they revealed the level they look for, though gave impression above average, but they did say they’d taken one candidate in the 15th percentile for numeracy. This wasn’t common of course but the tests were at the assessment centre and the test results didn’t match her performance in other tasks which was outstanding across the board. So in her case they made an exception and took the risk, which shows that unless the tests are online at an early stage, the cut-off maybe flexible.”
11
Engineering Versus Finance
“I’ve been on a number of employer visits in recent months, mostly in the engineering sector, and the general concensus in the companies I have visited seems to be that the cut off point is anything below the 50th percentile. As recruitment of suitable engineering graduates into training programmes seems always to create a few difficulties, this figure may well be lower than that for, say, business or finance graduates competing for commercial training, but I have no direct experience to back this suspicion up!”
12
Top Whack
“I know that recruiters I have spoken to who are happy to divulge this information tend to look for people at around the 90th percentile though most insist that they take a wider view of the applicant than aptitude test results alone.”
13
Retail Bank
“Before becoming a careers adviser I used to work
in graduate recruitment for a retail bank. We used ability tests – verbal and numerical reasoning tests – alongside first interviews, after an initial screening on the application forms submitted. This was for recruitment to a generalist graduate training programme.
We used a cut off point that was not too demanding. It was based on research and analysis that had been done previously that indicated people performing in the tests below a certain point would subsequently struggle in the professional banking qualification exams that were an integral part of the graduate training programme. The norm group that we used was based on previous applicants who had sat the tests (as such
these were people who had already passed initial screening) as part of our recruitment process over the previous few years (we, via SHL whose tests we used, updated our norm groups every couple of years). The cut off point we used, if I remember correctly, was the 16th percentile, or
a STEN score of 3. Candidates had to get more than this. Anything less and however well the candidates had interviewed they would not progress to the next, and final, selection stage which was the Assessment Centre.
We did the tests and the interviews on the same day. Each year it was only a relatively small number of people who were affected in this way, but there were some.
I’m sure, as you suggest, different employers use different norm groups and cut off points, and use tests at different times in their selection processes. No doubt some are taking those scoring in the highest percentiles. Others will perhaps have a minimum cut-off point. Others may well retain the tests scores and use them, probably in a weighted form, alongside all the other evidence they collect during the selection process. Knowing how the tests might be used can perhaps help candidates, but even knowing what the cut-off score was for a particular employer wouldn’t really help a candidate applying for them as there are so many ‘uncertainties’ in the process – what the norm group is, how many questions you actually need to get right, how many you have actually got right!
What I would say is that anyone who had interviewed really well but was borderline on the tests I would go back and look at the test answer sheets themselves to see the percentage they had got right, out of how many they had attempted. Where, say, the cut off point was 13 questions
correctly answered if the candidate had only attempted 14 and got 13 right, whilst another had got 13 out of 35 right, I would be tempted to give the former one the benefit of the doubt and put them through to the next stage on the basis that they had adopted a strategy of going slower, but concentrating on getting the ones attempted right. For us that wasn’t necessarily a bad thing. For other employers it might be!?!
I couldn’t really comment much on other employer strategies and processes, but I’m guessing that they are probably pretty similar to ones I have highlighted.”
14
How long is a piece of string?
“There are not officially ‘cut off’ scores for psychometric tests. Different tests are measuring different kinds of aptitudes and abilities ,in the case of personality tests then there are no ‘right’ or ‘wrong’ test results at all. In the case of aptitude or ability tests, individual’s test results are compared with a set of norms, which indicate how that individual compares with a group of people that fit the profile the employer is keen to employ. If they fall within a given range, then the result should indicate that they would be able to do a particular job.
However, test results should not be used in isolation – they should make up only a part of the selection process. It is unlikely that an employer would employ someone based on test results alone, although they may reject someone on that basis. An ideal selection test should be objective and be able to discriminate between candidates on the basis of their ability. It should also closely match the key elements of the job. All tests have different scoring methods and different norm values, so it is difficult to generalise.
BPS good practice guidelines suggest that all test takers should get feedback on their test results, so perhaps the student in question could ask for written or verbal feedback on their performance.
For more information, have a look at the BPS site www.psychtesting.org.uk “
