Skip to main content

UpdateDeploymentConfig

Below is the class definition for the Update Deployment Config. It inherits from the BaseModel defined in Pydantic.

class UpdateDeploymentConfig(BaseModel):
    cooldown_time: PositiveInt | None = Field(default=None)
    custom_args: list[str] | None = Field(default=None)
    hf_token: str | None = Field(default=None)
    min_replicas: NonNegativeInt | None = Field(default=None)
    max_replicas: PositiveInt | None = Field(default=None)
    scale_up_threshold: PositiveInt | None = Field(default=None)
    max_total_tokens: int | None = Field(default=None)
    lorax_image_tag: str | None = Field(default=None)
    request_logging_enabled: bool | None = Field(default=None)
    direct_ingress: bool | None = Field(default=None)
    preloaded_adapters: list[str] | None = Field(default=None)
    speculator: str | None = Field(default=None)
    prefix_caching: bool | None = Field(default=None)
    disable_adapters: bool | None = Field(default=None)

The meaning of these fields is identical to those in DeploymentConfig.

Note that there are a very large number of (advanced) configuration parameters that are configured by the custom_args field. See the custom_args section of DeploymentConfig for more information.

The default value of all of these fields is None. When updating a deployment, any fields set to None will be left at their currently-deployed value. Non-None fields will be updated to the value specified in the field. See Update Deployment for more information.

NOTES
  • To update the lorax version to the latest supported version, set lorax_image_tag to <current>.
  • When updating to a new custom_args value, the full value of the new custom_args field must be provided. E.g. if the existing value is ['foo'] and you want it to be ['foo', 'bar'], you must provide ['foo', 'bar'] as the new value.