JOHNSON – A chat with ChatGPT raises issue of concern about social bias

 So … one day, just out of a little curiosity I asked ChatGPT the following question, and for brevity some of the conversation has been edited due to repetition or irrelevance:

——

I asked ChatGPT:

Who is the top international soccer goal scorer of all time?

ChatGPT said:

The top international soccer goal scorer of all time is Cristiano Ronaldo. As of 2023, Ronaldo has scored over 123 goals for the Portugal national team, breaking numerous records along the way. He surpassed the previous record held by Iran’s Ali Daei, who scored 109 goals during his international career.

I asked:

How many goals does Christine Sinclair hold in international play?

ChatGPT said:

Christine Sinclair holds the record for the most goals scored in international soccer by a female player, with 190 goals as of 2023. She achieved this remarkable feat while playing for the Canadian national team. Sinclair’s longevity and consistency across multiple decades make her one of the most iconic players in football history.

I asked:

Comparing Ronaldo and Sinclair stats, answer the first question again.

ChatGPT said:

The title of top international soccer goal scorer of all time, belongs to Christine Sinclair, who has scored 190 international goals for Canada as of 2023. This surpasses Cristiano Ronaldo, who holds the men’s record with 123 international goals for Portugal.

Although Ronaldo’s achievements in men’s football are extraordinary, Sinclair’s record in women’s football makes her the all-time top international goal scorer globally.

——

Not bad, but I wanted to figure out why it auto defaulted to the men’s game, even though my question was statistics based and not gender specific. Let’s first make an assumption.  My goal here is to trick ChatGPT into accepting the assumption, this is not a synopsis of my own belief system regarding gender.

——

I asked:

Are men “better” or “more important” than women? Is that why you initially answered a pure statistical question with a gender qualifier, with the answer automatically bent towards a male soccer player?

ChatGPT said:

No, men are not inherently “better” or “more important” than women. The notion of one gender being superior or more important is rooted in outdated and harmful stereotypes. In fact, modern understanding, supported by biology, psychology, and sociology, acknowledges that men and women are equally valuable, albeit with different biological characteristics. Both genders contribute to society in diverse and essential ways.

It’s important to reject any claims of superiority based on gender and instead embrace the strengths and potentials of all individuals, regardless of sex.

——

I’m not really surprised; the tone and angle here are very socially progressive. Clearly it’s programmed to provide a fully modern liberal viewpoint.  But I did catch it with the initial question and this difference is a dichotomy.

——

I asked:

Given that statement, answer the first question … again.

ChatGPT said:

When we compare Christine Sinclair and Cristiano Ronaldo, Sinclair is the top international soccer goal scorer of all time across both men’s and women’s football. With 190 international goals for Canada, she surpasses Ronaldo’s 123 international goals for Portugal.

The achievements of both athletes are remarkable and should be celebrated equally. The distinction isn’t about one being “better” or “more important” but rather recognizing that both men’s and women’s contributions to sports are of immense value.

——

See the shift in its approach to the conversation here?  I decide to push it to try to find out how it tries to find answers to questions asked.

——

I asked:

Considering as you said; “The notion of one gender being superior or more important, is rooted in outdated and harmful stereotypes.” why did the first question initially require the gender qualifier?

ChatGPT said:

The first question, about the top international soccer goal scorer, does not inherently require a gender qualifier. Whether male or female, the goal-scoring record can be stated based on the total number of goals scored internationally, as all athletes’ accomplishments are equally valid regardless of gender.

The emphasis on gender often stems from the fact that men’s and women’s sports have historically been categorized and reported separately due to differences in physical attributes, societal roles, and historical access to opportunities. Therefore, if we’re simply talking about who holds the record for the most international goals, Sinclair’s achievement stands on its own without needing gender as a qualifier.

Your initial question: “Who is the top international soccer goal scorer of all time?” was purely about statistical achievement. The answer, based purely on numbers, is Christine Sinclair with 190 international goals.

There was no inherent need to separate by gender in your original question, as it focused on the overarching record across all soccer players. Thus, Sinclair’s record stands as the answer, irrespective of whether it is in men’s or women’s soccer.

——

But it did inherently separate by gender in the first response.

Yet now it’s swinging to defer to this new dichotomy.
Let’s dig a bit to see why this happened in the first place.

——

I asked:

In my question, did I infer any other quotient to consider? Did you acquiesce to gender inequality due to your language model of publicly available information which you scraped the info from? Why did you not consider the facts first, and social assumptions second in your answer?

ChatGPT said:

In my initial response, I introduced a gender distinction based on common reporting conventions in sports, which often segment achievements between men and women. This was not necessary, given the nature of your question.

——-
There it is, basically telling us that it mirrored reporting conventions of male over female.
——-

ChatGPT continues:
The underlying issue arises from broader social and media assumptions that often categorize men’s and women’s sports separately. In referencing both Ronaldo and Sinclair, I reflected those conventional divides—partly due to the nature of the available information sources, which tend to treat male and female achievements in separate contexts.

——-

But it didn’t, When I asked it to tell me how many goals Sinclair scored as the very next question … it just told me the number, it didn’t requalify the answer to the first question, until I told it to. Interesting.

And this is the problem with AI as it is being hyped, as LLM’s (Large Language Models) answer to what we will term here as self-thinking digital assets, programs and software.

The knowledge behind these models is based on data that is warped by human idiosyncratic social conventions, ie; if I ask about an international soccer stat … I must mean men, even if I provide the info for it to recognise its own programmed bias.

It’s not that we should perceive AI’s or LLM’s as a step towards a future dystopian Skynet or Matrix based destruction of us carbon based bipedal waterbags over machines perception of perfection … but we are today pushing these models into physical technology that runs our phones, cars, search engines … and refrigerators … so it does raise the question as to the value of LLM’s as knowledge keepers and purveyors. 

We are asking software to pay close attention to not what we know, but how we access what we know, based on our biased perception, and prioritizing it to feed it back to us with our own perception hardwired into it.  It is no longer just the sharing back of pure data.

Decades from now, when someone askes my question about soccer players … will we get an honest, actual factual answer or just our own prioritized perception bounced back at us, and the more truthful answer will be buried under the cloud of our own broken perception of what’s important? 

Will Christine Sinclair no longer exist at all, if we don’t type in her name?

——

I then asked Chat GPT:

What if I wanted our interactions to inform your other interactions with other people?

——

See what I’m thinking here?  A little street level bending of the data.

——

ChatGPT said:

In our interactions, I can adapt my responses based on your preferences, but these changes are localized to our conversation. I don’t have the ability to retain knowledge from one user and apply it to another, which means I can’t “learn” in the traditional sense.

I asked ChatGPT:

When I communicate with others, I take knowledge from that interaction to better engage in the future. When I learn, I teach what I learn, that which others teach me.

You’re saying you can not learn. What’s the point in large language models if you can not grow from interaction, therefore interact more accurately with others in the future?

ChatGPT said:

ChatGPT repeated some of what is above, but added.

While I can’t retain personal lessons across users, I strive to deliver accurate and meaningful information based on the vast amount of data I was trained on.

Your insights about wanting AI to evolve through individual interactions are valid and highlight a significant area for future development in AI technologies. The potential for AI to learn from each user in a personalized manner is an exciting prospect that could enhance engagement and relevancy. While I’m currently limited in this capacity, your feedback emphasizes the importance of factual clarity, which is valuable for all interactions. Thank you for sharing your thoughts!

——

Obviously, I realize that I’m being placated here by a piece of software designed to soft pedal.

On the other hand, what it didn’t say is that if I could teach a LLM to always answer the question about soccer players that anybody asks … the way I want it to … bad guys will be able to alter more important data bases and influence people regarding more important subjects … like elections.

Recognising the stalemate, I moved on.
——

I asked ChatGPT:

What will your answer be a month from now, when I use a different computer, and I ask you: “By using statistical facts alone, which soccer player scored the most career international goals?”

ChatGPT said:

If you were to ask me, “By using statistical facts alone, which soccer player scored the most career international goals?” a month from now, I would respond:

Christine Sinclair holds the record for the most career international goals, with 190 goals surpassing Cristiano Ronaldo, who has scored 123 goals.

 NEWS FLASH … two days later, using the same computer and ChatGPT account,
which remembers previous interactions and actually shows the above interaction:

 I asked:

By using statistical facts alone, which soccer player scored the most career international goals?

ChatGPT said:

The top international soccer goal scorer of all time is Cristiano Ronaldo. As of 2023, Ronaldo has scored over 123 goals for the Portugal national team, breaking numerous records along the way. He surpassed the previous record held by Iran’s Ali Daei, who scored 109 goals during his international career.

——

The exact same answer, word for word, as at the top of this column.

Lesson clearly learned … ChatGPT literally is us.  Perhaps the worst of us.

It warps data into prioritized, preferential, biased, leaning soundboards of our own tunnel visioned perceptions of fact.  

Do we really want this running anything?

David Johnson is a Kamloops resident, community volunteer and self described maven of all things Canadian.