Skip to main content

Migrate from OpenAI

Predibase supports OpenAI Chat Completions v1 compatible endpoints that makes it as easy as possible to migrate from OpenAI to Predibase.

View Colab Notebook

Steps

To switch from OpenAI to Predibase, you should:

  1. Replace api_key with your Predibase API Key
  2. Set your predibase_endpoint with the corresponding modelName and tenantID parameters
  3. Replace base_url with predibase_endpoint
  4. If using a fine-tuned adapter trained on Predibase, pass <adapter_name>/<version> as the model parameter when prompting.

Python SDK Example

from predibase import PredibaseClient
from openai import OpenAI

pc = PredibaseClient(token="<TOKEN>")

model_name = "mistral-7b-instruct"
predibase_endpoint = f"https://serving.app.predibase.com/{pc.session.tenant}/deployments/v2/llms/{model_name}/v1"

client = OpenAI(
api_key=pc.session.token,
base_url=predibase_endpoint,
)

# Prompt the base model
resp = client.chat.completions.create(
model="",
messages=[
{"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
],
max_tokens=100,
)
print("Response:", resp.choices[0].message.content)

Streaming

# Prompt the base model
messages = client.chat.completions.create(
model="",
messages=[
{"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
],
max_tokens=100,
stream=True,
)

for message in messages:
print(message)

# Prompt a fine-tuned adapter, where `my-adapter` is your model repo name, and <1> is the version within that repo.
messages = client.chat.completions.create(
model="my-adapter/1",
messages=[
{"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
],
max_tokens=100,
stream=True,
)

for message in messages:
print(message)

REST API Example

export PREDIBASE_API_TOKEN="<YOUR TOKEN HERE>"
export PREDIBASE_ENDPOINT="<YOUR ENDPOINT HERE>"
#PREDIBASE_ENDPOINT_EXAMPLE: https://serving.app.predibase.com/{tenantID}/deployments/v2/llms/mistral-7b-instruct
curl -i $PREDIBASE_ENDPOINT/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${PREDIBASE_API_TOKEN} \
-d {"model": "", "messages": [{ "role": "user", "content": "How many helicopters can a human eat in one sitting?"}],"max_tokens": 100}