Skip to main content

LLM Status

LLM deployments do not remain completely active all of the time and they can be in various operational statuses over the course of your workstream. These are a few methods available to check on your deployments or trigger them to get ready for prompting.

llm_deployment.get_status

llm_deployment.get_status()

This method returns the state of your deployment job in Predibase.

Parameters:

None

Returns:

A short description of your deployment status, such as "queued", "updating", "active", etc.

Example Usage:

Get the status of an LLM deployment (e.g. llm_deployment from the previous page)

llm_deployment = pc.LLM("pb://deployments/deployment-name")
llm_deployment.get_status() # queued

llm_deployment.wait_for_ready

llm_deployment.wait_for_ready()

This method blocks until your deployment is ready for complete usage (i.e. both active and scaled up compute-wise).

Parameters:

None

Returns:

None

Example Usage:

Get the status of an LLM deployment (e.g. llm_deployment from the previous page)

llm_deployment = pc.LLM("pb://deployments/deployment-name")
llm_deployment.wait_for_ready()
# waits.
# waits..
# waits...
# return when ready!

For users looking for finer-grained control and insight into their deployments, there are two more specialized methods for assessing deployment state:

llm_deployment.is_ready

llm_deployment.is_ready()

This method returns whether your deployment is active and its endpoint is responsive

Parameters:

None

Returns:

True or False

Example Usage:

llm_deployment = pc.LLM("pb://deployments/deployment-name")
llm_deployment.is_ready() # True or False