ChatGPT and the problem of training bias
This article is extracted from a series of articles about ChatGPT and technology leaders.
ChatGPT claims to be able to assist with talent management. This includes pre-screening candidates. There have been several infamous incidents of training bias in AI models, including in the policing and recruitment sectors. I ask ChatGPT about how we can tackle this problem.
Interviewer ❯
If we could revisit the topic of interviewing for a minute, I have a clarifying question regarding the question of bias.ChatGPT ❯
Sure, I’d be happy to revisit the topic of interviewing and discuss any questions you have about bias. What’s your question?Interviewer ❯
My colleague who specialises in recruitment has pointed out that there have been notable examples of bias in AI model training in the past, including an incident in 2015 when Amazon realised that their model had been trained to favour resumes submitted by men over those submitted by women. Are there scientifically respected ways to demonstrate that GPT and ChatGPT have not been trained in ways which bias their input to interviewing and pre-screening?ChatGPT ❯
There have been instances where AI models, including those used for recruiting, have been found to have bias in their training data, which can lead to biased outcomes. One way to mitigate bias in AI models is by using techniques such as data preprocessing, data augmentation, and debiasing methods.Data preprocessing is the process of cleaning and preparing the data before it is used to train the model. This involves removing any irrelevant information, handling missing data, and ensuring that the data is in the correct format.
Data augmentation is the process of creating new training data by applying a set of random transformations to the original data. This helps to make the model more robust and less likely to be affected by bias.
Debiasing methods are used to remove or reduce bias from the data. There are various debiasing methods that can be used, such as removing sensitive attributes, adversarial debiasing and fair representation learning.
Another way to mitigate bias in AI models is by using a technique called “counterfactual data augmentation” which consists of generating synthetic data that is similar to the original data, but with one or more sensitive attributes altered. This allows the model to learn from examples that are similar to the original data, but which do not contain bias.
It’s important to note that no method can completely eliminate bias and it’s a continuous process to monitor and improve the model. Additionally, It’s important to have a diverse set of people involved in the development, training, and testing of the model to help identify and address any potential biases.
It is also worth noting that OpenAI has a team dedicated to responsible AI and ethics, which includes research on bias and fairness in AI. They have published papers and articles about the topic, and make sure to follow best practices and guidelines in their development process.
The model talks about ways to mitigage bias in training data. However, it doesn’t tackle my specific question. ChatGPT doesn’t seem to know of any ways we can test for (or prove absence of) bias in a trained model.
OpenAI’s website talks about two techniques (TruthfulQA and RealToxicityPrompts) for measuring relative safety of models. But neither of these techniques can certify a model as “biased” or “not-biased”.
Next I ask whether OpenAI offers any guarantees about mitigating bias
Interviewer ❯
What guarantees do OpenAI make about bias mitigation in the service offered via ChatGPT?ChatGPT ❯
OpenAI does not make any specific guarantees about bias mitigation in the service offered via ChatGPT. However, they do take steps to mitigate bias and promote fairness in their models.OpenAI mentions that they are aware that AI models can perpetuate and even amplify societal biases present in the data used to train them. They also provide resources and guidelines in their website on responsible AI and ethics, and have a dedicated team working on this area.
They also regularly publish papers and articles on the topic of bias and fairness in AI, and make sure to follow best practices and guidelines in their development process. Additionally, They also encourage their users to use the model in a responsible way and to be aware of its limitations.
They are also transparent about the data used to train their models, and they provide tools that allow users to fine-tune the model on their own data. This allows users to adapt the model to the specific use case and to reduce bias by training the model on a diverse set of data.
It’s worth noting that no AI model can be completely free of bias, especially when it’s trained on data that reflects the real world. However, by following best practices and guidelines, by being transparent about the data used to train the model, and by encouraging responsible use, OpenAI is making efforts to mitigate bias and promote fairness in the service offered via ChatGPT.
OpenAI definitely invest in research about the impact of AI. This research is needed to successfully market their products. But, their conclusions are that there is “no silver bullet” on questions of safety and misuse.
One wonders whether this area of industry is something that will eventually be the target of government regulation. But how such regulations could be inforced is anyone’s guess.
