Prompt LLM (REST)

This /prompt endpoint is similar to the prompt command in the SDK.

Example:

POST https://api.app.predibase.com/v1/prompt

{
    "model_name": ["flan-t5-xxl"],
    "templates": ["What is your name?"],
    "options": {
        "temperature": 0.17
    },
    "limit": 10
}

You must provide a bearer authorization token as a header in the request.

Parameters:

templates: array
The prompt to be passed to the specified LLM. Unlike with the SDK, this must be an array of
raw strings which will be combined into a single prompt to send to the LLM. This is helpful
for structuring few-shot learning examples by passing them in as a list of examples.

model_name: array
The name of the LLM deployment to query. Unlike with the SDK, this must be an array
of strings indicating one or more models to query with the same prompt.

connection: Optional, integer
The connection to use as the source for the batch and indexing datasets that can be provided.
If not provided, will default to using the user's file uploads connection. Note this property
is not present in the SDK's version of this feature.

index_name: Optional, string
The dataset to use for dataset indexing via RAG with the LLM query. Unlike with the SDK
this must be a string mapping to the name of one of your Predibase Datasets.

dataset_name: Optional, string
The dataset for the LLM to generate responses for via batch inference. By passing in a dataset
the LLM will provide a result for every row of the provided dataset. Unlike in the SDK
this must be a string mapping to the name of one of your Predibase Datasets.

limit: Optional, integer
The maximum number of rows for the LLM to read for batch inference.

options: Optional, object
The additional query options passed along with the query. These are things like temperature
(the level of randomness in the LLMs response) or max sequence length (maximum number of
tokens the LLM can produce) of the LLM output.

Parameters:​

Parameters: