Using Cloudflare AI Gateway Proxy

The Cloudflare AI Gateway proxy is a great way to get more usage insight. It is also plug and play.

You can use it in python by creating a AI Gateway in your Cloudflare Dashboard.

Here is an example code.

import os
account_id = os.getenv('ACCOUNT_ID')
gateway_id = os.getenv('GATEWAY_ID')
base_url = f"https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai"

from openai import OpenAI
client = OpenAI(
    base_url=base_url,
)

It is as simple as that.

You can see how the requests look in the example below.