Skip to main content

Migrate from OpenAI

Predibase supports OpenAI Chat Completions v1 compatible endpoints that makes it as easy as possible to migrate from OpenAI to Predibase.

View Colab Notebook

Steps​

  1. Obtain your Predibase API token
  2. For the base_url, obtain your tenant ID (found on the Settings > My Profile page) and deployment name (found on the Deployments page)
  3. If using a fine-tuned adapter trained on Predibase, you'll need the adapter ID ("adapter repo name"/"version number"), which is included as the model parameter when prompting.

Python SDK Example​

from predibase import PredibaseClient
from openai import OpenAI

api_token = "<PREDIBASE API TOKEN>"
tenant_id = "<PREDIBASE TENANT ID>"
model_name = "<DEPLOYMENT NAME>" # Ex. "mistral-7b"
base_url = f"https://serving.staging.predibase.com/{tenant_id}/deployments/v2/llms/{model_name}/v1"

client = OpenAI(
    api_key=api_token,
    base_url=base_url,
)

completion = client.completions.create(
    model="",
    prompt="How many helicopters can a human eat in one sitting?",
    max_tokens=100,
)
print("Completion result:", completion.choices[0].text)

Streaming​

from openai import OpenAI

api_token = "<PREDIBASE API TOKEN>"
tenant_id = "<PREDIBASE TENANT ID>"
model_name = "<DEPLOYMENT NAME>" # Ex. "mistral-7b"
base_url = f"https://serving.staging.predibase.com/{tenant_id}/deployments/v2/llms/{model_name}/v1"

client = OpenAI(
    api_key=api_token,
    base_url=base_url,
)

# Prompt the base model
completion_stream = client.completions.create(
    model="",
    prompt="How many helicopters can a human eat in one sitting?",
    max_tokens=100,
    stream=True
)
text = []
for message in completion_stream:
    print(message)
    text.append(message.choices[0].text)
print("".join(text))

REST API Example​

export PREDIBASE_API_TOKEN="<YOUR TOKEN HERE>"
export PREDIBASE_ENDPOINT="<YOUR ENDPOINT HERE>"
#PREDIBASE_ENDPOINT_EXAMPLE: https://serving.app.predibase.com/{tenantID}/deployments/v2/llms/mistral-7b-instruct
curl -i $PREDIBASE_ENDPOINT/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${PREDIBASE_API_TOKEN} \
  -d {"model": "", "messages": [{ "role": "user", "content": "How many helicopters can a human eat in one sitting?"}],"max_tokens": 100}