Master Your AI: Explore Local AI with OVH for Just 1 Dollar

Today, generative AIs meet many uses, and it would be a shame not to take advantage of them. However, in the age of the GDPR, it’s legitimate to question the exposure of customer or partner data when relying solely on AI providers whose data processing remains opaque.

Having myself participated in the construction of one of these new virtual assistant services, I offer here a simple guide to starting up a machine and evaluating the performance of your AI application, or to offering your employees access to sovereign AI, with the aim of easily comparing the different models available.

This tutorial will show you how to migrate your existing application from an external provider to your own AI service in the cloud.

Requirements

To carry out this test, we’ll need :

• an OVHcloud account,
• a domain name (hosted by OVH or not),
• a sub-domain dedicated to this experiment,
• an application to be migrated,
• 1 $.

Domain Name

In the rest of this tutorial, I’ll be using the example.com domain. Within this domain, I’ve reserved the sub-domain ia for experimentation. When you see ia.example.com., feel free to replace it with your own domain.

Our Example Application

Here’s a very basic example of an application using OpenAI’s ChatGPT:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Suggest a travel itinerary to visit Norway",
        }
    ],
    model="gpt-4",
)

print(chat_completion.choices[0].message.content)

It’s a simple script that asks a virtual assistant a question and displays its answer in return, on the standard output.

In the case of a more complex application, we’ll see that the changes required are minimal and realistic.

Costs

It costs €1 to rent a machine for an hour, with a graphics card capable of handling a wide range of scenarios.

If you have a more complex application, this may require more than an hour’s testing, but we can see that cost is not a decisive factor in not testing.

Note also that OVHcloud is offering a €200 cloud credit for all new Public Cloud projects, giving you the opportunity to evaluate all their GPU-enabled machines, free of charge.

Ready?

Creating Our Test Machine

OVHcloud offers to rent machines with graphics cards by the hour, in their Public Cloud offer. This is the basis for our evaluation: prices range from €1 to €3 per hour.

1. Public Cloud Project

To get started, we’ll need to create a Public Cloud project.

Go to the OVH manager console, in the Public Cloud section, to create a new project.

Enter a name for the project, which will only be useful for finding your way around the OVH console later on.

A second screen will ask you for the payment method to be debited. As this is a cloud offer, the invoice arrives at the end of the month, depending on what you’ve used during the month.

If this is your first OVHcloud project, don’t forget to enter the promo code to benefit from the €200 in credits.

2. Start a New Instance

Once you’ve created your project, all the cloud offerings are just a click away.

The one we’re interested in here is “Compute”. We’re going to “Create an instance”:

Instance Type

We want a machine with a graphics card, so go to the GPU tab.

There’s a wide choice of machines, with different characteristics in terms of hardware, amount of RAM and graphics card(s). The best choice for evaluating models is to use those with NVIDIA L4 or NVIDIA L40S graphics cards: L4-90 or L40S-90 respectively. The latter type of instance is 2 times more expensive per hour, and is only useful if you intend to evaluate larger models such as FLUX.1-dev.

Geographical Location

Next, choose a datacenter in which to start the instance. There may not be one in every datacenter, but that doesn’t matter anyway, as this is a simple evaluation.

Operating System Selection and Configuration

Next comes the question of the operating system.

I recommend you choose the latest version of Ubuntu to use the cloud-init script which will deploy everything you need automatically.

Add an SSH key, even though you probably won’t need it today:

Instance Configuration

Now comes the instance configuration. We’re going to pass a post-installation script so that the machine is configured as it should be, without having to connect to it:

Place the contents of the cloud-init script I shared with you in this article, in the small dedicated text field:

The post-installation script will automatically install the LocalAI service. This is a service that exposes an API fully compatible with OpenAI, but using AI models directly on the machine. No data is sent outside the machine.

Network Mode

Last question: how is the machine connected?

To keep things simple, we choose Public mode, which assigns a public IP to the machine.

This will enable us to contact our AI service without having to go through another service, such as a load balancer. In a production setup, we’d obviously want to use a load-balancer, but it’s superfluous here.

We’re off!

We’re off!

All that’s left to do is choose the hourly rate, then validate the form to start creating the instance.

The billing method to choose](select-price.png)

Allow 10 minutes before the machine is fully operational:

• 1 minute for OVH to allocate a machine and start it up,
• 8 minutes to run the cloud-init script: configure and download containers, then download templates.

3. Domain Name Configuration

While the instance is starting up, we need to configure our domain name to point our sub-domain ia.example.com. to our freshly instantiated machine.

Once the creation has been validated, the OVH robots get to work, allocating a public IP to our machine. The operation takes a few seconds, so click on the button to refresh the list:

If the instance creation is successful, you should have an IP assigned to you:

Now we need to create a new sub-domain pointing to this IP. In the case of OVH, this takes place in the “Web Cloud” section:

We create the sub-domain ia, pointing to the IP given by OVH:

If your DNS provider isn’t OVH (and even if it is), you can use happyDomain to easily modify your domain.

AI Evaluation

All that remains now is to modify our application so that it uses our AI service rather than OpenAI :

 client = OpenAI(
-    api_key=os.environ.get("OPENAI_API_KEY"),
+    base_url="https://ia.example.com/v1",
 )

That’s it! As the LocalAI API is compatible, all the services used will work with the local templates.

We should add, however, that the model names are different. For example, if we have gpt-4 in LocalAI, it’s actually an alias to phi-2 or hermes-2-pro-mistral models, depending on the hardware configuration. The LocalAI interface allows you to install other models. It may be necessary to test several of them, as each has its advantages and disadvantages.

Take a look at these in particular:

• flux.1-dev for image generation,
• hermes-3-llama-3.1-8b to generate text with interesting reflections.

The rapid evolution of generative AI offers extraordinary opportunities, but it also poses challenges, particularly in terms of data protection.

Adopting a more sovereign approach, like the one proposed in this tutorial, allows you to take advantage of technological advances while retaining full control over sensitive data, cost control and flexibility. This will enable you not only to guarantee the confidentiality of your information, but also to tailor AI performance to your specific needs, without depending entirely on external service providers. Remember that the effectiveness of AI depends not only on its technical capabilities, but also on how it is integrated and managed within your infrastructure. By taking charge of these aspects, you can ensure that you remain competitive while complying with current regulations.