News

Exploring ChatGPT’s Age Estimation Capabilities on Serve Legal

Written by Catriona Jolley | Dec 2, 2024 12:55:22 PM

Exploring ChatGPT’s Age Estimation Capabilities on Serve Legal

The rapid development of technology is reshaping the landscape of age verification, particularly in the retail sector, as digital age assurance technologies are continuing to be developed. As these technologies advance, their effectiveness and fairness must be critically examined.

At Serve Legal, we recognise the potential for these tools to support compliance processes. Equally, we emphasise the importance of rigorous testing to ensure accuracy and avoid bias.

What is ChatGPT?

ChatGPT is an AI-powered conversational agent developed by OpenAI; it's designed to understand and generate human-like text based on the input it receives. But what does this software know about age estimation?

A recent article from biometricupdate.com highlighted the capabilities of ChatGPT, a large language model (LLM), of recognising facial identities and distinguish between different faces with notable accuracy. In a paper published by IEEE titled ‘Chatgpt and Biometrics: an Assessment of Face Recognition, Gender Detection, and Age Estimation Capabilities’, (See 2024 IEEE ICIP) discussions into the accuracy of ChatGPT’s Age Estimation capabilities caught our interest.

The Experiment: ChatGPT and Age Estimation

As a business who are passionately engaged in supporting the devlopment of accurate and fair age estimation technologies to support the age verification industry, we decided to conduct a lighthearted experiment to put ChatGPT to the test. Whilst we are aware that ChatGPT do not claim to be an expert in age estimation, and it is unlikely that businesses will be adopting ChatGPT as their age assurance provider in the near future, this experiment gave our team an insight into the accuracies of age estimation technology.

We gave ChatGPT images of 29 members of the Serve Legal internal team and asked the software to estimate their age. The images we provided to ChatGPT were of staff members of the following ages:

  • 18 staff members between the 21-25
  • 6 staff members between the 26-30
  • 3 staff members between 31-35
  • 2 staff members between 50-55

Of the 29 staff members, ChatGPT accurately estimated the age of just one—our Area Manager, Isy, who is 21.

For thirteen others, ChatGPT underestimated their ages, mostly by 1-2 years, though a few were underestimated by 3-5 years.

In practical applications of age estimation technology, underestimating age is less problematic, as it simply means that customers over 25 (who typically wouldn't be challenged) might still be asked for ID. While this could be a minor inconvenience at checkout, it’s generally better to err on the side of caution.

However, our Client Manager Ella, who is 24, was estimated to be 19—a significant 5-year error.

On the other hand, the ages of fifteen team members were overestimated. This was especially concerning for Jenni, Catriona, Maisie, Will, Emily and Andrew, all under 25 in their images, yet estimated to be older.

For example, Maisie, who is 21, was estimated to be 27. While she is legally old enough to purchase age-restricted items, the overestimation raises concerns. Under Challenge 25 policies require ID checks for anyone appearing under 25, such overestimations could lead to stores unknowingly making illicit sales, trusting faulty technology to manage compliance.

Serve Legal’s Role in Ensuring Reliable Age Estimation

As age verification tools become more prevalent, it’s essential for retailers and wider businesses to invest in technology that is not only reliable but also free from biases related to gender, race, or disability.

Serve Legal is proud to be at the forefront of this effort, working with partners like Durham University to create audit programmes designed to evaluate the performance and fairness of age estimation technology.

We offer a variety of testing services to ensure that businesses can trust the systems they implement.

Serve Legal’s Audit Programmes

  1. Zero Failure Certification – Age Estimation Testing (Challenge 25 Certification)

Age estimation technology can streamline customer transactions and reduce conflict between staff and customers. However, businesses must ensure that these tools are highly accurate to avoid sales to minors. Our Zero Failure Certification programme rigorously tests the technology to verify its accuracy in estimating ages, ensuring that it protects businesses while improving customer experience.

  1. Liveness Testing

As more transactions and verifications move online, robust biometric liveness detection is crucial for preventing fraud and ensuring secure authentication. Our Liveness Testing service leverages our unique ability to deploy large-scale, real-world testing frameworks using live, real-time human participants rather than pre-recorded or static datasets, ensuring the highest level of authenticity and relevance for your production-ready biometric tools.

  1. Fairness Testing

Data has shown that current facial biometric technologies often perform differently based on factors such as race, gender, or image quality. Serve Legal uses its diverse pool of over 5,000 auditors to identify weaknesses in biometric models, helping businesses ensure that their tools treat all customers fairly.

  1. In Situ Testing

Deploying age estimation technology in live retail environments presents challenges that can’t always be replicated in a lab. Factors like lighting, camera quality, and background noise can impact the technology’s performance.

Serve Legal’s In Situ audit programme tests these tools in real-world environments, ensuring that their accuracy and fairness match lab-based performance. Our testing evaluates not only technical accuracy but also customer and staff experience, providing a comprehensive benchmark for businesses to improve their compliance efforts.

As age assurance technology continues to develop, we expect to see it implemented in a number of businesses to assist with ID verification. For this technology to be successful, it must be reliable, accurate, and unbiased. Retailers who invest in well-tested, proven technology will not only improve compliance but also enhance customer experience by reducing friction during age verification processes.

Serve Legal is committed to supporting businesses in making informed decisions about the tools they deploy. By offering comprehensive, real-world testing, we help ensure that age estimation technology works as intended—protecting both businesses and customers in an evolving retail landscape.