Model Export

A trained model can be downloaded to your local machine through the Predibase UI. This is useful when you have your own serving stack that you'd like to plug Predibase models into.

Downloading a Model

On the details page for a trained model, click the button with a Download icon and select from one of the supported output frameworks:

The download will start automatically once the model has been successfully packaged for export.

Note: Exporting a model for download requires an active engine. If your selected engine isn't active, it may take some time for the engine to spin up.

Supported Frameworks

The model can be exported to the following formats:

• ludwig a zip file containing the native ludwig model files.
• torchscript a single file that is an intermediate representation of a PyTorch model that can then be run in a high-performance environment.
• triton a tarfile containing a compiled torchscript model and configuration for deployment with the open source Triton Inference Server.

Caveats

Currently, certain models cannot be exported as Torchscript due to limitations with preprocessing support. Specifically, models with the following feature types may not successfully export to Torchscript:

• Vector
• Audio
• Date
• H3

Support for exporting models with these feature types is a work in progress.

Exporting to External Datastores

Predibase supports exporting trained models to external datastores (e.g., S3) via the EXPORT MODEL PQL statement.

Using Downloaded Model

Install dependencies

Ludwig

pip install ludwig

TorchScript

pip install torch

Alternatively, you can run a TorchScript model in C++, removing the need for a Python runtime. See the PyTorch documentation for more details.

Load (and preprocess) data

Ludwig

You can pass the raw data that you used to create the model to model.predict().

Ludwig will take care of the necessary preprocessing steps.

import pandas as pd

input_data = {
    "Pclass": ["3"],
    "Sex": ["male"],
    "Age": [34.5],
    "SibSp": [0.0],
    "Parch": [0.0],
    "Fare": [7.8292],
    "Embarked": ["Q"],
    "Cabin": ["C85"],
}
df = pd.DataFrame(input_data)

TorchScript

The TorchScript model expects a dictionary of feature values as input.

Numeric features need to be converted to tensors before being passed to the model.

See below for an example using the Titanic data:

input_data = {
    "Pclass": ["3"],
    "Sex": ["male"],
    "Age": torch.tensor([34.5], dtype=torch.float32),
    "SibSp": torch.tensor([0.0], dtype=torch.float32),
    "Parch": torch.tensor([0.0], dtype=torch.float32),
    "Fare": torch.tensor([7.8292], dtype=torch.float32),
    "Embarked": ["Q"],
    "Cabin": ["C85"],
}

Load and use the model

Ludwig

Assuming the model was downloaded in Ludwig format and unzipped to a local directory /path/to/model (replace this with the actual path).

from ludwig.api import LudwigModel

model = LudwigModel.load("/path/to/model")

preds, _ = model.predict(df)

print(preds)

This will output something like:

                      Survived_probabilities  Survived_predictions  Survived_probabilities_False  Survived_probabilities_True  Survived_probability
0  [0.8492070138454437, 0.15079298615455627]                 False                      0.849207                     0.150793              0.849207

TorchScript

Assuming the model was downloaded in TorchScript format to a local file /path/to/model.pth (replace this with the actual path).

import torch

model = torch.jit.load("/path/to/model.pth")

preds = model(input_data)

print(preds)

This will output something like:

{
    "Survived": {
        "predictions": tensor([False]),
        "probabilities": tensor([[[0.8492, 0.1508]]]),
    }
}