Running Watsonx on ROSA with an integrated application pipeline for Generative AI driven app modernization

This post was written with the help of Sanjay Doiphode, Senior Technology Executive with focus on AI and Hybrid Cloud Computing, IBM.

At AWS re:Invent 2023, IBM announced support for IBM Watsonx on AWS with support for self-managed Red Hat OpenShift and  Red Hat OpenShift Service on AWS (ROSA) in Q1 of 2024. We are excited to announce that Watsonx is now supported on ROSA, a fully-managed OpenShift cloud service. This blog discusses why running Watsonx on ROSA is a great combination for using Generative AI to drive App Modernization and deliver those Cloud Native applications on AWS with ROSA.

Combining Generative AI with application modernization can reduce the developer tasks by creating up to 60% of content in a typical automation use case (IBM CIO ). Forrester Consulting also found that ROSA service on AWS can improve developer productivity by 65% faster development time and improve operational efficiency by 50% (Forrester Total Economic Impact study ) for your developer tasks. The combination can drive real business value by mixing Generative AI and an integrated application platform to deliver a complete App Modernization experience.

To start, let's ask Watsonx why you should run Watsonx on ROSA.

Figure 1, Watsonx.ai prompt studio answering why Watsonx on ROSA

That’s a great start but let's provide a bit more clarity on Watsonx so you can get the bigger picture.

IBM Watsonx is a cognitive computing platform that uses artificial intelligence to help organizations solve complex problems. Watsonx consists of several modules that enable you to use Generative AI LLMs (Large Language Models) to help businesses gain productivity improvements with support for chatbots, customer management and app modernization code generation as 3 example use cases.

Watsonx is IBM's commercial generative AI and scientific data platform based on cloud. It offers a studio, data store, and governance toolkit, all 3 available on OpenShift and ROSA. We support multiple large language models (LLMs) including the new OpenSource IBM Granite models that IBM offers indemnification if you use. It also supports a select set of OpenSource models such as Flan, Llama and mt5 but these change dynamically or you can always import your own models.

Watsonx consists of six modules and two of the most used are  Watsonx.ai and Watsonx.governance, both available on AWS. Watsonx.data has been available on ROSA for some time.

watsonx.ai™

A pre-integrated GUI based training and prompt studio to enable you to Train, validate, tune and deploy foundation and machine learning models with ease.

watsonx.governance™

A set of work flows and tools to track LLM training, model drift and document the use of models for business use cases. Watsonx.governance will accelerate responsibility, transparency and explainability in your data and AI workflows.

watsonx.data™

A framework of OpenSource data management tools that enable you to build a hybrid Data Lakehouse across your hybrid cloud data assets. With Watsonx.data you can scale analytics and AI workloads for all your data, anywhere with a data store that is open, hybrid and governed.

Figure 2 Watsonx major components

Why Red Hat OpenShift

Red Hat OpenShift Service on AWS (ROSA) is a managed OpenShift offering that automates the deployment and management of OpenShift clusters, so you can build, deploy and scale applications quickly and focus your business, not managing infrastructure.

Figure 3 Asking Watsonx about Watsonx on the Chat tab

What makes ROSA different? ROSA is a first party solution that is jointly engineered and supported by both Red Hat and AWS that is a comprehensive pre-integrated application platform That means you can:

• Access ROSA directly within the AWS console with a self-service experience
• Contact either Red Hat or AWS for support tickets for an integrated support experience
• Integrate with a wide variety of AWS applications and services including AWS App Runner, Elastic Load Balancing, AWS Directory Service, AWS CloudWatch, AWS X-Ray, and more
• Get a single bill from AWS for both the OpenShift service and AWS infrastructure consumption for streamlined billing and procurement
• Use your AWS committed spend, negotiated discounts and entitlements to procure ROSAHave a consistent OpenShift experience across the hybrid cloud for increased flexibility

Additionally, ROSA provides a declarative managed framework for building, packaging, and deploying containerized applications. Developers have access to comprehensive tooling for consistent application delivery, which natively include CI/CD tools like OpenShift Pipelines and GitOps to enable a consistent approach to software development and delivery so you can move code to production fast.

Click here to learn watch this re:Invent 2023 session about why you should modernize applications on ROSA.

Generative AI and Application LifeCycle

Why do Watsonx and OpenShift or ROSA go great together? The workflow and pipeline for an MLOPs model preparation, model development, deployment and iterative updates is very very similar to the DevOps process followed by application development. In fact in many cases a “model” trained or tuned by Watsonx will be published into a cloud native application that will have the same characteristics of life cycle, scale needs and governance of what is developed and delivered. Combining Watsonx with an application platform like ROSA is a great combination to compound the benefits of running a Generative AI platform alongside the ROSA application platform to integrate your MLOps and DevOps workflows.

Figure 4 The combined DevOps and MLOps Pipelines

Here are some of the key benefits of using Watsonx.ai for app modernization:

1. Using LLM’s for Code Translation, Code Generation, Code Transformation and Code documentation tasks can dramatically increase the productivity of developers from 30-60%  (IBM CIO link).
2. With the ability to utilize the IBM/Red Hat Granite LLM model series, the built in models from OpenSource or bring your own models, you can rapidly trial, tune and build out your Generative AI capabilities to accelerate your application modernization.

Here is the Output from a model code generation and code translation task that is common for a developer working with legacy code looking to modernize an application.

Figure 5 Watsonx.ai for Code Generation using the codellama-34b-instruct-hf model

Now we can try the new Llama-3 with just changing the model identified in seconds.

You can quickly move between models and test out the various options and then progress directly into model tuning with your data samples.

Figure 6 Watsonx.ai for Code Generation using the llama-3-8b-instruct

As of this sampling, here are the available models but there are many other options, or just bring your own models.

Figure 7 Built in Watsonx LLM Library

The future of AI is OpenSource

Watsonx currently utilizes the CP4D architecture with enhancements from OpenSource projects called Ray and CloudFlare, which are built into the new Watsonx workflows. In a future release, IBM Watsonx will fully integrate with Red Hat OpenShift AI, which will not only expand the capabilities to work with Watsonx.ai, but also bring in other integrations with ROSA as well as support for a broader set of features for the App Platform and Developer integrations. IBM and Red Hat are both fully committed to the OpenSource paradigm for Watsonx, OpenShift, ROSA and Red Hat OpenShift AI.

Figure 8 Watsonx, OpenShift and OpenSource integrations.

Other AI OpenSource announcements

On May 8th, IBM and Red Hat partnered to release the IBM Granite family of LLM’s into the OpenSoruce community that will continue the model of IBM Watsonx commitment to OpenSource and provide OpenSource access to the IBM Granite family. (IBM Granite link). Red Hat made a number of announcements at Red Hat Summit 2024 for AI integration across all of our OpenSource technologies across Podman, RHEL, Anisble, OpenShift and OpenShift AI so watch this space for more.

In Summary

This blog covered the reasons why Watsonx and Watsonx.ai with the built in foundation models are a great addition to ROSA  when combining an MLOps platform with DevOps capabilities. The combination can accelerate your developer journey and bring business benefits to your organization.

Follow the Red Hat blog for more announcements about new capabilities as they roll out. The power of Generative AI combined with a fully integrated App Platform powered by OpenSource is going to be a powerful tool for retiring technical debt.

For more information:

• Red Hat ROSA short video

• Red Hat OpenShift AI

• Watsonx on ROSA Resources

• Watsonx and OpenSource Resources

  • https://www.youtube.com/watch?v=Cgiqx0pJuLo&t=301s

• Why use OpenSource LLM’s

• Watsonx.ai demo