Installing HuggingFace GPT Models on Ubuntu

The image was generated Created with DALL·E, an AI system by OpenAI

I am starting to dabble with GPT models. I thought I would start by installing the open source versions on my own servers to play around . I chose DigitalOcean droplets , you can choose Amazon EC2 or Azure , or Google .

1. In my case I had to first set up a new Droplet:

For DigitalOcean the Droplet specification is — 8 GB Memory / 4 Intel vCPUs / 160 GB Disk — Ubuntu 22.10 x64

You can choose an EC2 instance of an equivalent size. — I would suggest either a t3.medium( 2 vCPUs, 4 GB of memory) or a m5a.large( 2 vCPUs, 8 GB of memory) or a c5.large

For Azure go with Standard_B2ms (2 vCPUs, 8 GB of memory) or a Standard_D2s_v4: This VM has 2 vCPUs, 8 GB of memory

• Choose an image: For deploying the Hugging Face model, select the “Ubuntu” image (preferably the latest LTS version, e.g., Ubuntu 20.04).
• Choose a plan: Select the appropriate plan based on your needs. GPT models can be resource-intensive, so you might want to choose a plan with sufficient RAM and CPU cores.
• I did not add any storage yet, more on this in future posts.
• Choose a datacenter region: Select the region that is closest to your users for lower latency.
• Add your SSH keys: To securely access your droplet, add your SSH public key. If you don’t have one, follow this guide to generate an SSH key pair: https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys-on-ubuntu-20-04

After you have created the Droplet , Use an SSH client to connect to your droplet using the IP address provided by DigitalOcean and the private key corresponding to the public key you added earlier.

I just used the web console from the DigitalOcean site , but you can use terminal on Mac or Command Prompt on Windows .

ssh root@your_droplet_ip -i path/to/your/private_key

Next , update the server

sudo apt update
sudo apt upgrade

Python 3 was already installed for me . Your case may be different.

Use this command if its not.

sudo apt install python3

Also install pip

sudo apt install pip

Next follow — https://huggingface.co/transformers/v4.7.0/installation.html.

I used PyTorch over tensorflow. This may be a personal preference.

 pip install transformers[torch]

Next I created my first python file to execute the model. I used vi(i know , its old school!). You can use nano if you are cool.

vi first.py

from transformers import GPT2Tokenizer, GPT2LMHeadModel

# Instantiate tokenizer and model
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2LMHeadModel.from_pretrained('gpt2')

# Define input prompt
prompt = "what is the capital of France?"

# Encode input prompt
inputs = tokenizer(prompt, return_tensors='pt')

# Generate output text
outputs = model.generate(
    inputs['input_ids'], 
    max_length=50, 
    temperature=0.7, 
    num_beams=5,
    no_repeat_ngram_size=2,
    early_stopping=True
)

# Decode output text
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)

# Print generated text
print(generated_text)

Try this out !

I know . I know the output is not what you expected

Here is what I get

The French capital is France. It is not a capital, it is a country. The French people are not French. They are French, and they want to live in France, but they don't want

Well the model is not trained to perfection as is the OpenAI ChatGPT model. But hey , it works !

I tried several other models with minimum code changes , maybe I will upload them to github, put comments below if interested.

So much for now , till next post.