In this post, Shelagh Green, Director for Careers and Employability at The University of Edinburgh, discusses the influence of ChatGPT and other related AI on the changing landscape of job hunting and recruitments. Shelagh informs and cautions student users on the potential benefits and common pitfalls associated with the use of AI tools in job search. This post belongs to the Hot topic series: Moving forward with ChatGPT.
The launch of ChatGPT has created a new buzz around the opportunities and challenges of using AI in recruitment, as it has in many sectors. AI has been used in the UK graduate recruitment market for over a decade yet currently remains in the minority, with only 10% of Institute for Student Employment (ISE) members saying they use it. Organisations are using AI to pre-screen candidates, to analyse video interviews, and to screen CVs. There is variation across sectors, but as the ISE tends to represent larger organisations with the capacity to invest in technology, it provides a reasonable indication of the current situation, many of our current students will encounter. However some data suggests that many companies are investing in recruitment automation, and developments such as ChatGPT could change the landscape.
Speaking to one recruiter in the tech sector recently who include significant text based assessment in their recruitment, they saw ChatGPT being used by applicants as soon as it was launched. Tell-tale signs included factual inaccuracies, an absence of context in answers and the consequence of prompt engineering, where the response is only as a good as the instruction or request made. Once the company website indicated they won’t accept ChatGPT generated answers, the numbers immediately dropped (and to be on the safe side they are using ZeroGPT(a ChatGPT detector tool)).
The decline in applicants making use of ChatGPT would suggest that most applicants are in fact honest. And when the company explored with applicants who they suspected of using it, many students immediately admitted this and on the whole were trying to use it positively, e.g. to improve grammar, as non-native speakers. They also found that students tried to use it too quickly which created poor outputs that were then easy to spot. This is well known in IT (GIGO-Garbage In Garbage Out) and in recruitment – investment in high quality applications is what gets results.
The personal nature of recruitment may make it more challenging to game the system, but it would be naïve to think that the influence of AI in graduate recruitment will be limited.
Recruiters see benefits and challenges in harnessing emerging technologies.
The biggest benefit seems to be in managing large volumes of data and sifting applications, contributing to communication with candidates, e.g. chatbots informing applicants about where they are in the recruitment process, and if used well, creating a more level playing field. Unilever used a system which didn’t use name, educational background or work experience, this increased female candidates by 16%. Some of these benefits also free staff time for higher value activities and human interaction. Some organisation have seen it as a way of screening fake CVs or claims about experience….which could leave us in the position of AI screening AI generated content, which may well only benefit the tech suppliers!
Concerns cluster around the robustness of the algorithms, evidence base and candidate perceptions. If algorithms are based on skewed data there is a risk of baking in inequalities, there is concern that cultural differences aren’t adequately addressed. To get round the GIGO problem, upfront investment is required and for many organisations, the business case for this is weak due to lack of evidence on the benefits derived. One factor in this is understanding candidate’s perception of fairness from the use of AI and the possible lack of emotional connection in the recruitment process. In fact it could be seen as a differentiating factor for business, such as recruitment email signatures that state this wasn’t written by a bot.
Wherever graduate recruitment team currently sit within this argument, I expect the autumn 2023 recruitment and assessment methodologies to reflect the emergence of ChatGPT, possibly with increased emphasis on psychometric and aptitude testing and elements of authentic assessments, via assessment centre exercise. And we all have a role in supporting our students to navigate this.
Students may need to consider issues such as copyright from data that is being scraped and be aware that their personal data may become part of what is out there to scrape. Digital, data and privacy literacy is essential. But along with this students can really value and promote the communication skills, critical thinking and creativity they develop with us. Communication is vital in every organisation, there is no originality in ChatGPT, asking the right question becomes ever more important, as does the ability to understand what is factual.
The emergence of ChatGPT and the like won’t make our students immoral – it provides a new tool for those who want to cheat to do so. The return in investment argument remains; applicants who put most effort in will be most successful, and that includes in how they use assistive tools such as AI. I would always say be honest in the application process as you will get caught out eventually – and as I heard from one recruiter recently, “if ChatGPT can get you a job – the chances are it can do your job”.
Shelagh Green
Shelagh Green is Director for Careers and Employability at The University of Edinburgh, leading the work of the Careers Service and supporting the institution in its strategic ambition to maximise students potential and to ensure that our graduates go on to achieve success in whatever they do, wherever they go. Outside of the University, Shelagh is a past President of AGCAS and has served on Scottish Government Careers Strategy Group. She is always open to suggestion, ideas and collaboration to enhance our support for students.