Skip to main content

Structured Output (JSON)

Predibase endpoints allow you to enforce that responses contain only of valid JSON and adhere to a provided schema.

The schema can be provided either using JSON schema (REST, Python) or Pydantic (Python).


from pydantic import BaseModel, constr
from predibase import Predibase

# Initialize Predibase client
pb = Predibase(api_token="<PREDIBASE API TOKEN>")

# Define a schema for the response
class Character(BaseModel):
name: constr(max_length=10)
age: int
strength: int

# Get a handle to the base LLM deployment
lorax_client = pb.deployments.client("mistral-7b-instruct")

# Generate a response that adheres to the schema
response = lorax_client.generate(
"[INST] Generate a new character for my awesome game. Strength 1-10. [/INST]",
"type": "json_object",
"schema": Character.schema(),

# Load the response as JSON and init an object of the desired schema
response_json = json.loads(response.generated_text)
my_character = Character(**response_json)