Ethical retraining – the ethics of how often we update our models

When we think about the amazing power we have as humans, the complex brain operations we employ for things such as weighing up different choices or deciding whether or not we can trust someone, we may find it hard or impossible to believe that we could ever use machines to do even a fraction of what our minds can do. Most of us make choices, selections, and judgments without fully understanding the mechanism that powers those experiences. However, when it comes to ML, with the exception of neural networks, we can understand the underlying mechanisms that power certain determinations and classifications. We love the idea that ML can mirror our own ability to come to conclusions and that we can employ our critical thinking skills to make sure that process is as free from bias as possible.

The power of AI/ML allows us to automate repetitive, boring, uninspiring actions. We’d rather have content moderators, for instance, be replaced with algorithms so that humans don’t have to suffer through flagging disturbing content on the internet on a daily basis. However, ML models, for all their wonderful abilities, aren’t able to reason the way we can. Automated structures that are biased or that degrade over time have the power to cause a lot of harm when they’re deployed in a way that directly impacts humans and when that deployment isn’t closely and regularly monitored for performance. The harm that can cause at scale, across all live deployments of AI/ML, is what keeps ethicists and futurists up at night.

However, part of the danger with AI/ML is in the automation process itself. The types of drift we went over in the prior section impact how models derive meaning from the training data they learn from. Generative AI models in particular might be especially sensitive to this kind of stagnation because their purposes are so generalized. Since new data is constantly coming out with each passing day, the temptation to keep up with training them is very high. But so are the costs. Typically, LLMs are pre-trained during certain periods and those pre-trained models are then packaged and sold to companies, which then do their own training with their own specific data samples to create a more specialized model.

It is worth noting, however, that though ChatGPT has been out for some time, its last knowledge update as of March 2024 was actually January 2022. Though the English language itself may not have changed significantly since January 2022, the world as we know it has. People are increasingly using LLMs like ChatGPT for learning about the world around them and they’re interpreting the outputs from these models as fact. Let’s take, for instance, an AI PM using ChatGPT and taking its outputs as fact. It could impact their role and responsibilities by contributing to decision making bias, leading to decisions being made on inaccurate information. Decisions based on flawed information could lead to misguided product features, misaligned goals, or ineffective strategies.

The ethics with regard to generative AI models, particularly LLMs, need to account for the increased dependency on these models. I recently met with a recruiter who was telling me about schools like Southern New Hampshire University laying off members of their staff because they were using tools like ChatGPT to generate course content. While we can say that they likely (hopefully) have human editors who check the content of their courses for accuracy, if they feel confident enough to displace workers with LLMs, that doesn’t diminish the importance of the stewards of LLMs to ethically maintain their models optimally considering the increased dependency on them.

The current state of accountability

Even when performance and maintenance appear normal, that doesn’t mean that the models aren’t taking liberties, resulting in real-world harm for the end user or for human beings that could be impacted downstream from the end user, whether or not they actually interact with the models themselves. A common example of this is the pervasive and unnecessary use of facial recognition software.

In February 2022, President Biden signed two pieces of legislation into law that expanded on AI accountability in the US: Artificial Intelligence for the Military Act of 2021 and the AICT Act of 2021. Will Griffin from Fortune magazine writes “While this legislation falls far short of the calls for regulation consistent with the European Union model and desired by many in the A.I. ethics community, it plants the seeds of a thoughtful and inevitable A.I. ethics regulatory regime.” It’s important to remember that AI ethics and regulations vary depending on where you live. In the US, we still lag behind European standards both in terms of legislation that’s put in place to rein in AI misconduct and in terms of how we enforce existing laws.

On October 30th, 2023, President Biden signed an executive order on “Safe, Secure and Trustworthy Artificial Intelligence” in an effort to minimize the risks of AI. The new standards directed by this executive order require developers working on AI systems to transparently share test results and “critical information” with the US government and embolden the National Institute of Standards and Technology (NIST) to set testing and safety standards before AI systems are released, restrict AI-engineered biological materials, and limit AI-generated content with watermarks. Other notable aspects of the executive order revolve around protecting the privacy of American citizens’ data, battling algorithmic bias and discrimination, consumer protections, mitigating worker harms brought on by AI, and the promotion of AI innovation and competition.

While executive orders are nice to have, their effects can vary and it’s unclear how they’ll be enforced moving forward. They’re a step in the right direction, and the executive order on October 30th was lengthy and circumspect, but that’s precisely what leaves them open to interpretation. Without legislation and hard regulations that are exact in definition, they can be rendered almost meaningless. Executive orders are also vulnerable to subsequent presidents coming in and limiting their language or revoking them altogether if they don’t align with the current administration’s priorities.

AI is still considered a wild west legislatively speaking, and we will likely see strides being made toward further defining the scope for how AI can interact with us as we see more and more use cases for AI products expand during this decade. AI PMs need to be aware of how legislation is changing at a local and global level to better prepare for potential changes to how AI products are conceptualized, built, and managed. Recently, the US made strides toward publishing a blueprint for an AI Bill of Rights that covers the following areas:

• Safe and effective systems
• Algorithmic discrimination protections
• Data privacy notices and explanation
• Human alternatives, consideration, and fallback

For now, we will use the European standards for framing how AI/ML product managers should think about their products because, even without deliberate laws that enforce AI ethics, entrepreneurs and technologists still face risks, such as losing customers, receiving bad press, or being taken to court, as a result of their algorithmic choices.

The European Commission outlines the following four key areas as ethical principles:

• Respect for human autonomy: “AI systems should not unjustifiably subordinate, coerce, deceive, manipulate, condition or herd humans. Instead, they should be designed to augment, complement and empower human cognitive, social and cultural skills. The allocation of functions between humans and AI systems should follow human-centric design principles and leave meaningful opportunity for human choice.”
• Prevention of harm: “AI systems should neither cause nor exacerbate harm or otherwise adversely affect human beings. This entails the protection of human dignity as well as mental and physical integrity.”
• Fairness: “While we acknowledge that there are many different interpretations of fairness, we believe that fairness has both a substantive and a procedural dimension. The substantive dimension implies a commitment to: ensuring equal and just distribution of both benefits and costs, and ensuring that individuals and groups are free from unfair bias, discrimination and stigmatisation.”
• Explicability: “This means that processes need to be transparent, the capabilities and purpose of AI systems openly communicated, and decisions – to the extent possible – explainable to those directly and indirectly affected. Without such information, a decision cannot be duly contested. An explanation as to why a model has generated a particular output or decision (and what combination of input factors contributed to that) is not always possible. These cases are referred to as ‘black box’ algorithms and require special attention.”

  For more details, you can refer to A framework for assessing AI ethics with applications to cybersecurity by Danilo Bruschi and Nicla Diomede at https://doi.org/10.1007/s43681-022-00162-8.

On February 2nd 2024, EU member state representatives voted unanimously to approve the EU AI Act, making it the first comprehensive legal framework for AI anywhere in the world. Its impacts won’t just be specific to EU member countries but to countless citizens all across the world. The AI Act will first group AI systems into various categories:

• Minimal risk
• Limited risk
• High risk
• Unacceptable risk

This is a concept significantly lacking in any of today’s guardrails. Depending on the level or category of your AI system, responsibilities would include things like risk assessments, technical documentation and record keeping, transparency and disclosure, and compliance in accordance with the frameworks it will include. This is also a piece of legislation with teeth, with significant financial fines for companies/individuals in violation according to the AI category of their respective products.

For more details, you can refer to https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai.

Implementing ethical standards in your organization

Many companies might be tempted to create an AI ethics role within their companies and make it that person’s problem or make them a scapegoat if and when they fail to meet certain standards, but this is a lazy and unethical way of managing the ethics around your AI programs if that’s all you choose to do. A better way would be to train and empower all of the people who are involved in building your AI/ML products to be aware of the surrounding ethics and potential harm that could be caused to the customers or third parties interacting with your product.

While we must recognize the importance of understanding that recurring model updates are vital to maintaining good ethics with regard to ML and AI, as we’ve discussed previously in this chapter, it’s also important to look into how your product can affect groups of people downstream who don’t even use your product.

We don’t exist in a vacuum. As we saw in the previous sections of this chapter, many factors at play already work against algorithms used in AI/ML products, which you have to keep track of even to stay on top of the natural chaos created by the constant input and output of data. This natural tendency that models have toward various types of drift is what demands a focus on ethics. According to a recent episode from TechTarget’s Today I Learned podcast, FICO, the credit reporting and analytics vendor, conducted a survey of AI users and it showed that 67% of respondents do not monitor their models for accuracy or drift, which is pretty mind-blowing. These were AI users who were directly responsible for building and maintaining AI systems, which shows that the problems that come with unethical AI/data practices are the norm.

Ethical AI practices should be applied throughout every step we’ve outlined in this in-depth chapter on model development and maintenance. If we build AI/ML products that we are sure don’t cause harm, both directly as part of our products’ integrity and indirectly as part of our products’ model maintenance, we can confidently market and promote our products without fear of retribution or punishment from the market that we want to serve. Every entrepreneur and technologist will have their own relationship with ethical business practices but, eventually, if you are a champion, promoter, or leader of a product that has come to market that harms others, you will be asked to explain what measures were put in place to inform your customers of the potential risks.