Process inference requests in bulk at a lower cost
Warning: This is a beta feature. APIs and behavior may change.Use the batch inference API to process inference requests in bulk at a lower cost.
model
field. The base model
will be specified when creating the batch job.
Once your JSONL file is ready,
upload it to Predibase.
For information about preparing your dataset, see
Dataset Preparation.
Warning: Currently only base models up to 16B parameters are supported.
completed
, download the results:
custom_id
field to match results with inputs.