Assessing the environmental impact of large language models

The problem with LLMs

The environmental problems with LLMs spring from the large aspect. The amount of power consumed by the current generation of LLMs is associated with the size of the data sets they are trained on. An LLM's size can be characterized in part by the number of parameters used in its inference operations. More parameters means more data to move around and more computations to make use of that data.

Today's LLMs have orders of magnitude more parameters than earlier models. For example, Google's Bidirectional Encoder Representation from Transformers, or BERT, LLM, which achieved state-of-the-art performance when it was released in 2018, had 340 million parameters. In contrast, GPT-3.5, the LLM behind ChatGPT, has 175 billion.

Paralleling parameter counts, the power necessary to train some LLMs has jumped by four to six orders of magnitude. Power consumption has become a significant consideration when deciding how much training to perform -- along with cost, as some LLMs cost millions of dollars to train.

Training cycles consume the full attention of energy-hungry GPUs and CPUs. Extensive computational loads plus storing and moving massive amounts of data, contribute to large electrical draw and huge heat exhaust.

Heat load, in turn, means that more power goes toward cooling. Some data centers use water-based liquid cooling. But this method raises water temperatures, which can have adverse impacts on local ecosystems. Moreover, some water-based methods pollute the water used.

In comparison to training, the power consumed by an individual inference for a deployed model can seem miniscule. But that comparatively tiny amount must be multiplied by the number of inferences run when using that model in production.

In addition, many deployed models can only be used for a short time -- weeks or months -- before the model needs to be retrained. Addressing the problem of model drift requires repeating steps from the original training process and consuming a similar amount of power.

Reducing the environmental impact of language models

To address these problems, developers can reduce the size of their AI model and training operations.

LLMs are not the only kind of generative AI or natural language processing model. Smaller models, which have lower training costs and less significant environmental impacts, can perform nearly as well in many situations. For example, the Alpaca model from researchers at Stanford University and Meta's Large Language Model Meta AI, or Llama, are small enough to run on a desktop and can be trained for hundreds of dollars rather than millions.

Another way to reduce training costs over the full model lifecycle is with one-shot or few-shot training. Using this technique, trained LLMs can learn to deal with new input from one or more examples and adapt to deal with similar inputs thereafter.

In addition, enterprises can make their hardware more efficient by using different chip architectures or different architecture tools based on that hardware. The SpiNNaker2 chip architecture, for example, emulates biological neural networks by supporting locally dense computation across a sparsely active network. That is, where nothing is currently happening in the neural net, the chips consume nearly no power. Despite being built on larger transistors, the chip architecture consumes much less power compared with most current CPUs and GPUs, while accomplishing a similar amount of computational work.

To deliver sustainable tools and continue to make profits, AI companies need to make quick shifts to more efficient technologies and practices. Customers and prospects should be holding their feet to the fire on their environmental impacts and demand plans for mitigation in the near future.