Quickstart

Predibase provides the fastest way to fine-tune and serve open-source LLMs. It's built on top of open-source LoRAX.

• Inference: Try the Python SDK / REST or the Web Playground to prompt serverless endpoints
• Fine-Tuning: Fine-tune and serve a model in just a few steps using the SDK or UI

Run inference using the SDK or REST

1. Create an account here.
2. Navigate to the Settings page and click Generate API Token.
3. Install the Python SDK with pip install -U predibase
4. See available serverless deployments. (Note: VPC customers will need to first deploy a dedicated deployment.)

Python SDK

import os
from predibase import Predibase, FinetuningConfig, DeploymentConfig

pb = Predibase(api_token="<PREDIBASE API TOKEN>")

lorax_client = pb.deployments.client("llama-3-8b-instruct") # Insert deployment name here
print(lorax_client.generate("What are some popular tourist spots in San Francisco?", max_new_tokens=100).generated_text)

REST

# Export environment variables
export PREDIBASE_API_TOKEN="<YOUR TOKEN HERE>" # Settings > My Profile > Generate API Token
export PREDIBASE_TENANT_ID="<YOUR TENANT ID>" # Settings > My Profile > Overview > Tenant ID
export PREDIBASE_DEPLOYMENT="llama-3-8b-instruct"

# query the LLM deployment
curl -d '{"inputs": "What are some popular tourist spots in San Francisco?", "parameters": {"max_new_tokens": 512}}' \
    -H "Content-Type: application/json" \
    -X POST https://serving.app.predibase.com/$PREDIBASE_TENANT_ID/deployments/v2/llms/$PREDIBASE_DEPLOYMENT/generate \
    -H "Authorization: Bearer ${PREDIBASE_API_TOKEN}"

Next steps

• Try out the full example to fine-tune and prompt an adapter in Predibase using the SDK
• Don't want to code at all? Use the UI to connect a dataset and start fine-tuning an adapter.
• Coming from OpenAI? Check out our migration guides for fine-tuning and serving
• Explore additional complete examples
• See how you Predibase integrates with other frameworks in the ecosystem

Get in touch

Reach out to us at support@predibase.com or join us on Discord for any questions, comments, or feedback!