Skip to main content

SDK Troubleshooting

My dataset is too large

If you train a model that fails with an error message that looks like this:

RuntimeError: LLM fine-tuning is supported for datasets of max size 1GB. The selected dataset is 1.070GB. To fine-tune with this dataset, please set sample_ratio to 0.934 in the preprocessing config. (type: ErrorResponse, retryable: true).

You can use the SDK to set the sample ratio like this:

my_tmpl = tmpls["qlora_4bit"]
cfg = my_tmpl.to_config(prompt_template={PROMPT_TEMPLATE}, target={TARGET})
cfg["preprocessing"]["sample_ratio"] = {SAMPLE_RATIO}
llm.finetune(config=cfg, dataset={DATASET})

For more details about templates, see customizing fine-tuning with additional templates.

Prompting my deployment isn't working

If you call llm_deployment.prompt() and run into errors, here are a few ways to solve the issue:

  1. Check the state of your deployment with llm_deployment.get_status(). If this doesn't return 'active', try redeploying the LLM. You can see the statuses of all your deployments with pc.list_llm_deployments() (See list_llm_deployments docs for more details).

  2. If you get an error that looks like this:

    ERROR: HTTPSConnectionPool(host='api.app.predibase.com', port=443): Max retries exceeded with url:

    Try rerunning llm_deployment = pc.LLM({NAME_OF_DEPLOYMENT})