Reduce project expenses with AI cost optimization strategies

What are the main cost drivers in AI projects?

In a typical AI project, several factors can lead to high costs:

• Data acquisition and management. Training AI models requires vast quantities of high-quality data tailored to specific needs or use cases. Collecting and cleaning that data for model training can be a costly endeavor.
• Model training infrastructure. To train models efficiently, organizations typically turn to specialized AI infrastructure, such as servers equipped with GPUs and NPUs. This infrastructure can be expensive: A high-end GPU alone can cost around $25,000. Servers, storage and additional infrastructure can drive AI project costs even higher.

• Inference infrastructure. After training a model, the business must also pay for the infrastructure used to host it for inference, when the model makes predictions or decisions on new data in production. Inference infrastructure is usually less expensive than training infrastructure, as it requires less specialized hardware, but it can still demand considerable CPU and memory resources.
• AI talent and expertise. Developing, training and managing AI models requires specialized -- and in-demand -- skills, which translates to high salaries for employees capable of supporting AI projects. That's exacerbated by the short supply of data scientists with AI skills, which gives workers leverage to negotiate high compensation even in a sluggish job market.

Not every project will incur all these expenses. For example, businesses with healthy data management practices already in place might not face high data acquisition costs if the data they need for training is already on hand.

Best practices for AI cost optimization

The best approach to reducing AI project costs depends on the specific cost drivers and available resources. But in general, the following strategies can optimize costs without compromising the value that AI brings to the business.

1. Take advantage of open source

There is a burgeoning ecosystem of free and open source AI resources: open source models, data quality and management tools, open data sets that can be used for training purposes, and more.

When appropriate, using open source AI tools or models is likely to lead to lower overall costs. For example, tailoring an open source model for a specific project won't require as many data scientists and developers compared with building and training a model from scratch.

2. Consider outsourcing AI development

In some cases, working with an external vendor to develop and deploy AI models can reduce costs, especially when hiring in-house AI engineers and data scientists would be expensive. Companies that specialize in AI development on an outsourced basis might be able to complete projects at a lower total cost than an internal team could achieve.

3. Automate AI

Automating important workflows in AI projects leads to more efficient use of AI teams' time. In addition, automation reduces the risk of oversights and errors that could lead to wasted time and resources. Approaches such as AutoMLOps, which automates machine learning operations workflows, can help reduce the overall cost of AI.

4. Decrease data volume

In general, processing larger volumes of data -- whether during training or inference -- leads to higher costs. Because more data translates to higher load on infrastructure, removing irrelevant data can help to optimize AI from a cost perspective.

Decreasing data volume can be as simple as stripping model input of unnecessary words so that the model has less information to process. It can also involve extricating redundant or irrelevant data from training data sets -- a process that, although more complex, will both save money and improve data quality. But be sure that decreased data volume doesn't come at the expense of accuracy or completeness.

Generative AI cost optimization strategies

The AI cost optimization strategies described above apply to most types of AI projects. But for generative AI in particular, additional cost optimization practices come into play due to the high computational costs associated with these models.

1. Use IaaS for model training

Investing in specialized AI hardware doesn't make good financial sense if the hardware is only needed to perform periodic tasks, like model training. Owning hardware but using it only sporadically means poor ROI.

A more cost-effective approach is to use IaaS platforms that provide access to AI accelerators. For example, the major public cloud providers offer access to servers with GPUs or other accelerator devices. Businesses can effectively rent this hardware to train models, then stop paying for it once training is complete.

2. Train models on spot instances

Those using IaaS for model training can reduce costs further by taking advantage of "spot" cloud server instances. Spot instances are available at steep discounts -- with the caveat that the cloud provider can turn off the instance without warning.

But for tasks like model training, unpredictable shutdowns are typically manageable. AI teams can pause and resume training operations as needed, although this might somewhat extend training time due to the disruptions. But if the business can afford some delays, using spot instances designed for AI training can significantly reduce cloud costs.

3. Fine-tune models

Fine-tuning a generative AI model enhances its ability to produce results that are relevant for a specific use case. Although reducing costs is not typically the main goal of fine-tuning, it can be an added benefit.

A fine-tuned model is more likely to deliver the results users are seeking on the first try, without requiring multiple prompts or requests to regenerate content. This, in turn, translates to lower inference costs, as the model doesn't consume as much CPU and memory during user interactions.

Chris Tozzi is a freelance writer, research adviser, and professor of IT and society who has previously worked as a journalist and Linux systems administrator.