At Red Hat Summit 2024, generative AI goes open source

This year's Red Hat Summit presented a vision of an open source future for generative AI.

According to Red Hat, businesses will not be limited to running a small number of proprietary models derived from unique data sets. Instead, the future of AI will be open: Models can be trained and tuned by developers and developer communities based on open source concepts.

Red Hat predicts that AI will be integrated into a wide variety of applications, with businesses managing thousands of applications and AI models. However, for this open generative AI future to succeed, the cost and complexity of tuning language models must decrease significantly. Additionally, organizations must simplify and accelerate the modernization and automation of their underlying infrastructures.

Demonstrating its commitment to this open generative AI vision, Red Hat announced that it will open source the Granite family of language and code models under the Apache license, in collaboration with IBM Research. Along with open sourcing the Granite models, Red Hat unveiled several innovations intended to facilitate model training, tuning and augmentation, as well as simplify infrastructure automation and modernization.

Those announcements included the following:

• InstructLab. This is an open source community project designed to simplify the tuning of existing language models. It enables developers or developer communities to collaborate using open source principles. Developers can fine-tune an existing language model in specific knowledge areas by generating synthetic data from developer-generated seed data and then using that synthetic data to train the model.
• Developer preview of Red Hat Enterprise Linux AI (RHEL AI). This foundation model platform helps organizations develop, test and run open source Granite generative AI models.
• Podman AI Lab. This is an extension for Podman Desktop designed to simplify building, testing and running generative AI-based applications in containers.
• Multiple enhancements to Red Hat OpenShift AI. These include the ability to deploy more models at edge locations and support multimodel servers used for predictive and generative AI.
• Image mode for RHEL. This enables IT teams to manage the OS using container-based tools and concepts, such as GitOps and continuous integration/continuous delivery. Through simplifying the process of aligning the OS with the needs of specific applications using container concepts, Red Hat anticipates better support for AI-infused application development projects. 
• Ansible Policy as Code. A new tool to help automate compliance enforcement across on-premises, cloud and edge environments. With the number of AI applications expected to increase, Red Hat expects policy as code to serve as guardrails for AI, ensuring that any updates or changes made manually or by AI adhere to evolving internal and external requirements.

This open source-centric vision for generative AI is no surprise given Red Hat's history and approach to innovation. While not necessarily opposing the visions of major public cloud providers, such as AWS, Microsoft Azure and Google Cloud, Red Hat's vision diverges slightly from the common perception that the ability to train, tune or augment generative AI with unique, proprietary enterprise data will be the best way for organizations to differentiate themselves from their competitors and create an advantage. The jury is out, however, on whether either vision for the future of generative AI will come to fruition.

The deployment and use of generative AI are still at an early stage but are growing quickly. According to research from TechTarget's Enterprise Strategy Group, 54% of organizations said they expect to have a generative AI project in production in the next 12 months. However, the implications of Red Hat's vision of generative AI can help clarify the logic behind their announcements this week. 

As mentioned, to apply open source concepts to generative AI, the cost and complexity of tuning models need to decrease significantly, and tuning must become far more accessible to enable a large community of developers to participate. The introduction of InstructLab, an open source project designed to radically simplify the process of fine-tuning models using synthetic data generation, addresses this need. A single developer can identify a gap in a language model's knowledge and then instruct the model with that specific knowledge.

This approach differs from retrieval-augmented generation (RAG), where an existing model can be augmented with organization-specific data for inference. While RAG and InstructLab offer different approaches, they can be complementary.

If generative AI models become easier and less costly to tune and train, the number of available models will likely increase, and developers will require tools such as Podman AI Lab to integrate AI into their applications. The infrastructure and automation platforms will need to evolve to keep pace with the rapid advancement of AI. This need partly drives the introduction of image mode for RHEL and Ansible Policy as Code.

In other words, Red Hat is not just enabling the open sourcing of generative AI models but is also developing the tools to help organizations use models resulting from an open generative AI future. The launch of InstructLab aims to democratize data science and empower developers with data science capabilities and responsibilities.

The generative AI space is still very new, and questions remain about whether an open source language model can deliver a competitive advantage for users or whether developer communities can improve upon the work of the existing data science space. A likely outcome is that organizations will use multiple models -- some open, some proprietary -- or internally augment open source models with their own unique data. Time will tell how the future of generative AI unfolds.

Scott Sinclair is practice director with TechTarget's Enterprise Strategy Group, covering the storage industry.

Enterprise Strategy Group is a division of TechTarget. Its analysts have business relationships with technology vendors.