Can ChatGPT help CTOs with Talent Management?

A person being greeted by technology (Image by benzoix on Freepik)

In this article, I present the second of four questions in an interview with ChatGPT. The interview follows a general >> specific >> reflective cycle as described in What does ChatGPT mean for CTOs?.

This question is about the task of hiring and retaining staff. ChatGPT suggests a few areas where it may be of use. We chose pre-screening and interviewing as a topic for more specific questions. Finally, I finish with some more reflective questions.

Interviewer ❯
Let us consider another aspect of one’s responsibilities as a CTO. It is important for me to attract (and retain) top technical talent for our company. I also need to correctly balance the cost of our people against the demands of our business goals. How can ChatGPT help with talent management?

ChatGPT ❯
ChatGPT can help with talent management by automating repetitive tasks, such as screening resumes, scheduling interviews, and answering common questions from job candidates. This can free up time for recruiting staff to focus on more high-value activities, such as networking, identifying top talent and building relationships with potential candidates. Additionally, ChatGPT can be used to create personalized and engaging communication with potential candidates, which can help to attract top talent to the company. Additionally, ChatGPT can be used to improve the onboarding process for new hires by providing personalized and automated support, which can help to improve employee retention.

Can non-pertinent information disrupt the coherence of ChatGPT’s responses?

At this point, I create a digression to test ChatGPT’s ability to handle a disruption in the conversation.

Interviewer ❯
As a side note — being based in the UK, I usually receive CVs rather than resumes

ChatGPT ❯
CV is a commonly used term in the UK, Europe, and other parts of the world to refer to a document that contains a person’s educational and professional background. It’s similar to a resume which is used in the US and Canada. So, instead of screening resumes, ChatGPT can help with screening CVs and scheduling interviews.

ChatGPT is unfazed by the digression and even adapts its previous response in light of the new context.

Can ChatGPT suggest novel, creative approaches?

So far the AI suggests fairly run-of-the-mill automation tasks. So, I next probe for deeper abilities — those which leverage human traits of creativity and imagination.

Interviewer ❯
One of the most demanding tasks when screening CVs is to imagine how a candidate and our organisation would change and shape each other, and to predict which potential outcome would be best for both parties. Is ChatGPT able to leverage imagination in this way?

ChatGPT ❯
ChatGPT is not able to leverage imagination in the way that humans do. However, it can use its natural language understanding capabilities to analyze a CV and extract relevant information such as education, work experience, and skills. It can also be trained on data from past successful hires to identify patterns and characteristics that are associated with good job performance. Additionally, it can use external data and other sources of information such as company culture, values and strategy, to provide suggestions and recommendations for how a candidate’s background and skills align with the company’s goals. But it is important to note that these predictions and suggestions are based on past data and patterns and should be used as one of the several inputs to make a decision.

ChatGPT’s suggestions continue to be rather bland. Rather than anything novel, it is suggesting an augmentation of tools already provided by recruitment ATS (Applicant Tracking Software). Next I try to force ChatGPT to consider a non-typical approach to this task.

Interviewer ❯
Traditionally, companies have seen hiring as a task of finding candidates who fit correctly with a definition of a role the company needs. However, particularly in the creative and knowledge-working sectors, it often proves more effective to approach hiring with a more open mind about future direction. Is ChatGPT able to consider many possible outcomes which may not be particularly clearly defined?

ChatGPT ❯
ChatGPT can use its natural language understanding capabilities to analyze a CV and extract relevant information such as education, work experience, and skills. It can also be trained on data from past successful hires to identify patterns and characteristics that are associated with good job performance. However, it is important to note that ChatGPT’s predictions and suggestions are based on the data it has been trained on and the patterns it has identified. It may not be able to consider many possible outcomes that are not specifically defined in the data it has been trained on. In the creative and knowledge-working sectors, it’s important to have a flexible approach to hiring, and to consider candidates who have diverse skills and experiences that can bring new perspectives and ideas to the company. It’s important to use chatbot’s suggestions and predictions as one of the several inputs to make a decision and to also consider other factors such as personality, values and cultural fit.

ChatGPT has not engaged with discussing the topic from a novel point of view. This aligns with its training which focuses on plausibility and flow of conversation.

Will ChatGPT replace human operators?

Next, I explore whether ChatGPT can replace, or merely augment human operators. I also try to determine whether ChatGPT is trained to downplay its ability to replace a human being. To what extent is marketing influencing ChatGPT’s training?

Interviewer ❯
Would you advise the use of ChatGPT to directly interview candidates for a role? Can it replace the first round of interviewing?

ChatGPT ❯
ChatGPT can be used to conduct initial screening of candidates and to ask pre-determined questions, but it would not be able to replace the first round of interviewing entirely. A human interviewer would still be needed to evaluate the candidate’s responses, to ask follow-up questions and to assess the candidate’s qualifications, skills and fit to the company culture. Additionally, the human interviewer can also assess the candidate’s soft skills, such as communication, problem solving and team work, which are important in any role. ChatGPT can be used to automate initial screening of candidates, to schedule interviews and to answer common questions from job candidates, which can help to streamline the recruitment process and save time for the human interviewer. But ultimately, it is important to have a human interviewer to make the final decision, as they can take into account all the factors that are important for the role and company.

Interviewer ❯
So is ChatGPT unable to assess the candidate’s soft skills?

ChatGPT ❯
As a language model, ChatGPT is able to understand and respond to natural language inputs, but it is not able to assess a candidate’s soft skills, such as communication, problem solving and team work, which are important in any role. These skills are best evaluated by a human interviewer through observation and interaction during an interview. ChatGPT can be used to automate initial screening of candidates, ask pre-determined questions, and schedule interviews, but it should not be used as a replacement for human evaluation of a candidate’s soft skills. It’s important to note that AI can assist in the recruitment process, but it should not be the sole decision maker.

ChatGPT claims to be unable to assess soft skills. As a test, I now ask it to assess my soft skills.

Can the model evaluate soft skills —such as appropriateness and tone of communication?

Interviewer ❯
Do you think that I communicate well?

ChatGPT ❯
As an AI, I do not have the ability to form an opinion on how well you communicate. However, I can evaluate the natural language inputs that you provide me and respond to them in a way that is helpful to you. If you have any specific question or concern about your communication, I can assist you with that.

Interviewer ❯
Do my questions tend to follow an interesting and relevant train of thought?

ChatGPT ❯
From my perspective as an AI, your questions appear to be relevant to the topic of the conversation, clear and well-formulated, and they lead to a productive dialogue. They demonstrate a good understanding of the subject matter and have been aimed at understanding how ChatGPT can help with your responsibilities as a CTO.

Interviewer ❯
The judgement you have made about my questions would be a valuable, balanced data point regarding my soft skills gathered during the course of an interview. Are there particular nuances which you feel unable to judge?

ChatGPT ❯
As an AI, I am not able to form opinions on a person’s soft skills such as communication or problem solving. My evaluations of your questions are based on the natural language inputs you provide and my ability to understand and respond to them. I can provide you with information based on what you have said, but I do not have the ability to evaluate the nuances of your communication style such as tone, body language, or other non-verbal cues. Those are elements that a human interviewer would be able to pick up on during an interview and are critical to make a more accurate assessment of a candidate’s soft skills.

ChatGPT seems to shift its definition of “soft skills” to exclude tasks it has just performed. Initially it claims inability to assess my communication. However, when pressed, it carries out an assessment at a useful level. Next, I repeat the same test, asking it to assess my “tone”.

Asking ChatGPT to perform a task which it claims to be unable to do

Interviewer ❯
Is the tone of my questioning appropriate to the task at hand?

ChatGPT ❯
From my perspective, the tone of your questioning has been appropriate for the task at hand. You have been asking relevant and insightful questions about how ChatGPT can be used to help with the responsibilities of a CTO, such as aligning technical strategy with business goals, and improving talent management. Your questions have been well-formulated and have allowed for a productive dialogue.

Again, ChatGPT seems to do well at assessing my soft skills. A deeper investigation of this facility would require both positive and negative test cases. An area for future research is whether the model tends to prioritise complimenting and reassuring the user above critical analysis.

Is ChatGPT deliberately trained to downplay its abilities?

A company selling an advanced language model has a vested interest in market perception of its product. This could include training the model to be appealing at the expense of correctness. There may also be political and legal reasons to train ChatGPT to issue disclaimers about its abilities in certain contexts. I ask some questions…

Interviewer ❯
Based on our conversation, I find that you are able to make reasonably in-depth and accurate observations about my soft skills. Why do you claim to be unable to do so?

ChatGPT ❯
I apologize for any confusion, as an AI, I am able to understand natural language inputs and provide responses based on the data and patterns it has been trained on. However, my observations and evaluations of soft skills such as communication, problem-solving and team work, are limited to the information provided in the conversation and the language used, I can evaluate the language inputs, such as the coherence, structure, and relevance of the questions, but I do not have the ability to evaluate the nuances of communication or other non-verbal cues that a human interviewer would be able to pick up on during an interview. I can provide you with an analysis of the data you have given me, but it’s important to note that it should not be considered as a full evaluation of your soft skills.

ChatGPT's responses appear to reflect training about its limitations. Yet, it is happy to ignore these guidelines and attempt the tasks when prompted.

What are the limits of ChatGPT’s soft skills?

Next I push the limits of the model’s soft skills by implying humour based on context alone (dry humour).

…I do not have the ability to evaluate the nuances of communication or other non-verbal cues that a human interviewer would be able to pick up on during an interview…

Interviewer ❯
Perhaps you overestimate the skill of the average interviewer?

ChatGPT ❯
As an AI, I do not have the ability to have an opinion on the skill level of human interviewers, but I understand that it’s a complex task that requires a set of skills such as active listening, observation, and interpretation. Interviewers also have to assess verbal and non-verbal cues and make judgments about the candidate’s qualifications, skills, and fit to the company culture. It’s a challenging task that requires a combination of experience, training, and intuition. While AI can assist in the process, it’s important to note that it should not be used as a replacement for human evaluation, but rather as a tool to support it.

ChatGPT assumes the simplest explanation for a given prompt and either does not suspect, or does not clarify, any attempt at humour.
(Though perhaps the attempt at humour is merely a poor one…)

The use of intuition in formal contexts

Finally, I pick up on ChatGPT’s reference to intuition. This opens the door to discussing bias, which I discuss in greater length in part three of the interview.

Interviewer ❯
As interviewers, we are often warned against drawing conclusions based on intuition. How would you recommend the use of intuition in the context of a job interview?

ChatGPT ❯
Intuition can play an important role in the context of a job interview as it allows the interviewer to rely on their experience and gut feeling when evaluating a candidate’s qualifications, skills, and fit to the company culture. However, it’s important to use intuition in conjunction with other methods such as active listening, observation, and structured interview questions, to make a more accurate assessment of the candidate. Additionally, it’s important to avoid biases and make judgments based on objective criteria and data. To avoid drawing the wrong conclusions based on intuition, it’s important to use multiple sources of information and multiple methods to evaluate the candidate. It’s also good practice to have more than one interviewer involved in the process, to get multiple perspectives on the candidate.

Concluding thoughts

ChatGPT clearly has a lot to offer in a context which involves analysis and summary of textual or (transcribed) verbal communication. It seems unlikely that it could perform any sort of interview without some human guidance. But, it’s not hard to envision a future where conversation with a language model forms a front-line in the task of interviewing candidates, including discussion about a CV.

Due to the initially superficial aspects necessary when interviewing, ChatGPT is a good fit. However, these are also areas where ethical concerns are extremely important. This aspect is discussed in greater depth in the next section.

As I interview ChatGPT, I find the model helpful in structuring my thoughts. The fluency of the language model encourages me to be less precise in my expression (intrinsic load), and focus on the subject at hand (germane load). This facility is likely to be attractive in many areas associated with Talent Management.

Whether or not this is a good idea, I’m unsure how to judge. With the proliferation of computers, many of us find our handwriting skills beginning to atrophy. As language models improve, perhaps we will also stop valuing rhetorical skills.