DEV Community

Cover image for Easiest Way to Deploy HuggingFace Transformers
wjiuhe
wjiuhe

Posted on

Easiest Way to Deploy HuggingFace Transformers

You must know transformers. Well, I don’t mean autobots, Bumble Bee, but the famous machine learning structure.
You probably have used the huggingface transformers models. But have you ever deployed them?

With Pinferencia, just add three more lines and your model goes online!

Never heard of Pinferencia? It’s not late. Go to its GitHub to take a look. Don’t forget to give it a star if you like it.


codes

HuggingFace transformer pipeline

How do you use HuggingFace transformer pipeline?

from transformers import pipeline

vision_classifier = pipeline(task="image-classification")


def predict(data):
    return vision_classifier(images=data)
Enter fullscreen mode Exit fullscreen mode

hfimg

And you can predict a image with its url:

predict("https://cdn.pixabay.com/photo/2018/08/12/16/59/parrot-3601194_1280.jpg")
Enter fullscreen mode Exit fullscreen mode

Result:

[[{'score': 0.9489120244979858, 'label': 'macaw'},
  {'score': 0.014800671488046646, 'label': 'broom'},
  {'score': 0.009150494821369648, 'label': 'swab, swob, mop'},
  {'score': 0.0018255198374390602, 'label': "plunger, plumber's helper"},
  {'score': 0.0017631321679800749,
   'label': 'African grey, African gray, Psittacus erithacus'}]]
Enter fullscreen mode Exit fullscreen mode

Deploy

Now deploy it with Pinferencia, just add three lines and save as app.py

from transformers import pipeline
from pinferencia import Server

vision_classifier = pipeline(task="image-classification")


def predict(data):
    return vision_classifier(images=data)


service = Server()
service.register(model_name="vision", model=predict)
Enter fullscreen mode Exit fullscreen mode

Now go to the terminal and run

uvicorn app:service --reload
Enter fullscreen mode Exit fullscreen mode

Your service is online! Go to http://127.0.0.1:8000 and check out the API.
Now you can send a request:

curl --location --request POST 'http://127.0.0.1:8000/v1/models/vision/predict' \
    --header 'Content-Type: application/json' \
    --data-raw '{
        "data": "https://cdn.pixabay.com/photo/2018/08/12/16/59/parrot-3601194_1280.jpg"
    }'
Enter fullscreen mode Exit fullscreen mode

The result:

[[{'score': 0.9489120244979858, 'label': 'macaw'},
  {'score': 0.014800671488046646, 'label': 'broom'},
  {'score': 0.009150494821369648, 'label': 'swab, swob, mop'},
  {'score': 0.0018255198374390602, 'label': "plunger, plumber's helper"},
  {'score': 0.0017631321679800749,
   'label': 'African grey, African gray, Psittacus erithacus'}]]
Enter fullscreen mode Exit fullscreen mode

Or just use the interactive UI Pinferencia provides:

Predict

Results

Simple enough, huh?


If you like Pinferencia go https://github.com/underneathall/pinferencia/star and give it a star.

Top comments (0)