pb.deployments.update

pb.deployments.update

     Update an existing serverless deployment

Parameters:

       deployment_ref: str
       Name or UUID of the private serverless deployment

       config: Update Deployment Config

Returns:

       Deployment

Usage:

To update a deployment, fetch the configuration of an existing deployment, for example by calling pb.deployments.get, or pb.deployments.list, and modify the fields you want to change. Provide the updated configuration to pb.deployments.update as the config parameter.

NOTE

All field values in UpdateDeploymentConfig must be set to the desired value, even if you are not changing them. Setting a field to None does not mean the parameter will be ignored, instead the parameter will be reset to the default value.

Example:

Get the configuration of an existing deployment

pb.deployments.get(name="my-mistral-7b")

which might return:

Deployment(
    name="my-mistral-7b",
    # <...>
    config=UpdateDeploymentConfig(
        custom_args=[], cooldown_time=600, hf_token=None, min_replicas=1, max_replicas=2, scale_up_threshold=1
    ),
)

Update the deployment configuration and provide it to pb.deployments.update

pb.deployments.update(
    deployment_ref="my-mistral-7b",
    config=UpdateDeploymentConfig(
        custom_args=[],
        cooldown_time=1200,  # Changed from 600
        hf_token="your-hf-token",  # Now providing a hugging face token
        min_replicas=1,  # All other fields unchanged
        max_replicas=2,
        scale_up_threshold=1
    )
)

NOTES

• Updating a deployment will not cause any downtime. The existing deployment will continue to serve requests while the new configuration is applied.
• If you used a HuggingFace token to deploy the model, that token will not be returned by methods like pb.deployments.get. You will need to store it separately and provide it when updating the deployment.
• Not all lorax CLI arguments are supported. Passing a non-supported argument will result in an error.
• The SDK and backend do not validate that the values of custom_args are valid lorax parameters. Passing an invalid value will result in Lorax failing to start the deployment. (However the existing deployment will continue to serve.) custom_args is intended as a break-glass feature for advanced users who need to pass additional parameters to Lorax.