When generative AI violates your own values

Hyperscale generative AI is ethically unacceptable because it is based on unconsented training data, causes enormous ecological damage and depends entirely on the commercial interests of a few corporations. Small, domain-specific models with transparent training data, on the other hand, offer stable benefits without creating these problems.

Podcast Episode: When generative AI violates your own values

Simply not using generative AI because it contradicts your own values: this is the position that Johannes Link consistently advocates. I talk to him about why he considers hyperscaled Gen AI to be ethically unacceptable and what brought him to this conclusion. We talk about training data that is used without the consent of the creators, about massive energy consumption, about the disintegration of the free internet and about what happens to students who delegate writing and thinking before they have ever really practiced it. Johannes also explains what would have to change for him to reconsider his opinion and whether he thinks this change is realistic.

“A statistical model knows neither right nor wrong, nor truth.” - Johannes Link

Johannes Link has been programming for more than 40 years, 30 of them professionally. Since the end of the last century, extreme programming and other human-centered software development approaches have been at the heart of his work. His professional focus is on the (re)organization of teams towards more personal responsibility and self-direction. The meaningful and ethical design of his private and professional life has been driving him for years. He has been involved with GenAI since the early days of OpenAI’s GPT language models.

Highlights der Episode

• Generative AI outsourced to US hyperscalers is based on training data whose authors in most cases have never consented to their work being used for model training.
• The projected increase in electricity consumption by AI data centers in the US from 4.4 percent to 25 percent of total consumption is roughly equivalent to the total addition of renewable energy over the same period.
• Statistical language models cannot structurally distinguish between truth and falsity, which is why hallucinations are not an implementation error, but a basic feature of the model type.
• Students who delegate learning tasks entirely to generative AI do not acquire the skills that these tasks are intended to build, and the first cohorts of graduates with fully AI-supported studies already exist.
• Small, domain-specific models with transparent training data, such as for language analysis or protein structure prediction, deliver stable benefits but receive a fraction of the investment that goes into large commercial models.

What it’s about: hyperscale generative AI, not AI in general

The ethical criticism is directed against a very specific variant of artificial intelligence, not against the entire field. This refers to generative AI, which is outsourced to a few large providers, predominantly in the USA. This distinction is the basis of any meaningful assessment.

The collective term “AI” obscures this distinction. Anyone who says “AI” can bring up all kinds of good applications and use them as a distraction. However, many successful AI applications are not generative AI at all, or at least not hyperscaled AI.

Johannes Link rates today’s hyperscaled generative AI in its current form as ethically unusable. That is a personal assessment based on personal values. This is exactly how ethics should be conceived: as an individual decision about what you want to do and what you don’t want to do.

Why the models are based on stolen data

The training data comes from works whose authors never consented to their use. The word theft is meant here morally, not legally. At some point, lawyers will decide whether the licenses cover this, but by then it will be too late.

The problem is concrete and affects everyone who has ever published anything. Open source software, lectures, articles: All of this ends up in training data without there ever being any question of it being used to train a model.

Data quality cannot be controlled because the foundation is a statistical model. A statistical model knows neither right nor wrong nor truth. Everything that comes out remains a statistic.

No matter how good the training data is, we will always get hallucinations, because that is the basis of this model: that even where I have no parameter in the training data, I still get something out of it. Johannes Link

The ecological price eats away at progress in renewable energies

The projected electricity consumption of data centers is projected to increase from 4.4 percent of total US electricity consumption to 25 percent within four years. This is roughly equivalent to the amount of renewable energy that will be added during this period. The entire expansion will therefore be swallowed up by the AI hype.

In addition, there is water for cooling, which is not easy in these data centers. And then there is electronic waste. Some of the raw materials for the GPUs could be mined in the Global North, but they are not, so as not to poison our own soils. Mining and disposal take place where the raw materials come from.

The hardware cycle exacerbates the problem. A GPU lasts around one and a half to two years in this mode. The data centers practically always run at full load because they cannot be ramped up and down in small steps, but only switched on or off in large blocks. After around five years, the entire data center infrastructure has to be replaced, something that traditional data centers are not used to.

Generative AI has strong potential to destroy the education system

The education system is crying out for AI because it is underfunded. Teachers are overworked, students are overloaded. This is precisely where the promise that a machine can fill the gap is particularly seductive.

The problem lies in delegated learning. Students hand over tasks that are supposed to help them acquire skills and knowledge. As a result, the acquisition of skills shrinks, if it takes place at all. There are neurological studies on what this does to the brain’s ability to solve problems.

The effect comes quickly. A bachelor’s degree lasts three years. The first students have spent their entire studies using these tools. Within a very short time, there are graduates who no longer have the skills they should have.

The same applies to anyone who has texts generated instead of writing them themselves. The point of writing is to engage with the topic, to rub your own thoughts together and to research. If you can generate a text, anyone else can generate it too. Then nobody needs it anymore.

The free Internet bears the burden that others create

Non-profit projects such as Wikipedia and OpenStreetMap are under heavy strain from AI crawlers. These crawlers grab every available input. However, the infrastructure for this is financed by the very initiatives that supply the crawlers with the material. This is money that they lack elsewhere.

At the same time, the database is poisoning itself. It is estimated that 50 to 60 percent of all new content has already been partially generated. Hardly anything new can be created from this material. AI companies are already reacting by concluding contracts with publishers and editorial offices to secure genuine human input.

Filters do not solve the hallucination problem

Downstream and upstream filters fail at the same point as naive text filters against SQL injection. Filtering before and after using regex does not solve the problem.

The reason lies deeper. In large language models, the distinction between data and statement is blurred. A prompt contains both at the same time. If the two cannot be separated, it is not possible to filter reliably.

There is also a counter-intuitive property of large models. The more data and parameters, the smaller the proportion of targeted pollution that an attacker has to hit in order to steer a model in a certain direction. Anyone who wants to manipulate knows this. And it happens.

What would have to change to make the use justifiable

The models would have to be under public control. Open here means: traceable, what goes in, whether authors have consented, and verifiable by the public. This is the only way to rule out manipulation.

At the moment, almost everything depends on the goodwill of a few American billionaires. At best, they are only looking for profit. At worst, they follow misanthropic ideologies and do not shy away from manipulating the models. We need to get away from that.

The use of resources should also be reasonable, whatever that means in individual cases. The prerequisite for this is an initial tangible, qualitative benefit. So far, convenience, friction reduction and profit have dominated. Faster Java code does not count as a benefit that outweighs the damage.

A technology that helps the climate or benefits the part of humanity living in poverty would make sense. Such application scenarios hardly appear in the current calculation, even if they are sold as such.

Small, open models are the useful part

Real benefits arise from very small models whose training data can actually be viewed. Small here means a different scale than is usual today: a few million parameters instead of a hundred billion. Even a medium-sized company can train such models itself.

These models are not only useful in narrow areas, they are stably useful. Examples:

• Language analysis, often reduced to a single language and therefore significantly smaller
• Information extraction from texts, a problem that could not be solved before these models
• AlphaFold as a protein folder, a generative model, but small and only trained on its domain

However, most small models available today are only open-weight and distilled from large commercial models. A lot could be done in this area. The billions are flowing elsewhere.

You don’t have to learn prompting as a skill

The concern about being left behind without using AI is based on a fallacy. If you don’t use these tools, you won’t suffer a major loss of skills. The specific prompts are constantly changing with each model generation anyway.

What is really needed in prompting is the ability to formulate things clearly. Most people have this ability to a greater or lesser extent, and it can be refreshed at short notice as soon as you need it.

Two scenarios remain. Either software development is not completely replaced, in which case the specific skills can be learned at any time if required. Or software development is completely replaced, in which case something completely different is on the agenda anyway.

A collapse is likely, the extent is open

The financial bubble is big, and so is the interest of governments and very large companies in keeping it going. Some form of collapse will come, if only because of the lack of energy at this point. When and how hard cannot be predicted.

Changes to the overall system need to be made at the political level. Those who are politically active can mix the issue with the many others and sensitize fellow campaigners to it. There is a digital Ludist movement, the Maschinenstürmer in Germany, which is more progressive than its image suggests. You can get involved there.

For the individual, the first step is to check what you can do for yourself. Energy consumption, chips and raw materials seem abstract, but in the end they affect everyone, just like the climate and the social upheavals that are linked to it. The concrete question is: What does this mean for me and what can I do or not do in my environment?