Skip to main content

Overview

Engines are the abstraction layer that Predibase uses to take care of all computation requirements for your operations including PQL queries, model training, etc. Engines come in different flavors and sizes. Engines created through the UI are available via the SDK and vice versa.

Engine Types​

There are three types of engines offered within Predibase: Training, General, and Serving.

  • Training: (Recommended) a training engine allocates the right amount of resources for the training job. This type of engine is especially useful for users who do not want to think about compute. The compute resource selection is shown under the compute tab in the model builder page. Specifically, a training engine has a few desirable properties:
    • Right-sizing: Based on the dataset and model you plan to train, it will allocate the right amount of compute resources to use for a given training job.
    • Batch: It is automatically spun up when the training run starts and spun down as soon as training finishes, minimizing the amount of compute overhead consumed. There is no need to manuall start/stop it.
    • Retry Logic: It offers more robust error handling for temporary issues ensuring engines are more reliable.
  • General: General engines are the manual version of training engines. They support model training and PQL querying, but do not support model deployments. They offer more control of which resources to allocate for the training/PQL job.
    • They come in two types (CPU, GPU) and sizes ranging from small to large. Examples of general engines include Medium CPU engine, Large GPU engine, etc.
    • General Engines are particularly useful for PQL. The t-shirt size selection allows for a tradeoff between cost and the ability to handle large queries faster.
  • Serving: Serving engines are used for real-time model serving (i.e. 1-click REST deployments)

Note:

  • The default engine for model training is a training engine. However, the user can override this default selection by choosing another engine.
  • Training engines are currently only supported for model training. In the near future, they will be expanded to cover additional functionality including Dataset Upload / Model export that's currently handled by General Engines.
  • As Training engines are not used for PQL queries, they will not show up in the engine dropdown on the Query page.

Engine selection recommendations​

The size of the dataset, the feature types it contains, as well as the model configuration specified greatly influence which engine is best for the training job. Sometimes, an engine could be under-resourced to handle a large dataset/model (e.g. training a large text encoder on a small CPU engine) and will fail to train. Some other times, an engine is too large for the job and the user ends up wasting resources (e.g. training a gradient boosted model on a large GPU engine).

  • We highly recommend Training engines for training models in Predibase. The recommendations come from extensive experimentation on a wide range of datasets and models. The allocated compute resources ensure a lot more robustness than general engines.
  • If you would like to use general training engines, here are a few guidelines:
    • If your dataset contains any of the features text, image, audio, sequence, timeseries, use a GPU engine. These features use encoders that are larger than those used for other features and using a GPU greatly speeds up training and reduces the risk of the engine crashing. A CPU engine is still suitable for all other features, yet GPU engines could result in faster training.
    • If you're training a Gradient Boosted Model (GBM), use a CPU engine. We are planning to add GPU support for GBM in the future.
    • Scale the size of the engine with the size of the dataset.
      • Use a small engine if your dataset size is on the order of thousands of rows.
      • Use a medium engine if your dataset size is on the order of tens of thousands of rows.
      • Use a large engine if your dataset size is on the order of hundreds of thousands of rows.

Engine States​

Engines can be in the following states:

  • Inactive: This is the initial state of an engine.
    • General & Training Engines will never go back to Inactive after they're used at least once. Serving Engines will be Inactive when there are no deployments on them.
  • Initializing: This engine is currently spinning up. It can't be used in this state.
  • Active: This is an engine that is ready and available for immediate use.
  • Suspending: This is an engine that is in transition to Suspended.
  • Suspended: This is a paused state. The engine will automatically transition to Initializing and then turn Active if requested by a model training or a query.
  • Errored: This is an engine where the last operation errored.

Serving Engines, in particular, support two additional statuses:

  • Terminating: This is an engine where a deployment is being deleted.
  • Updating: This is an engine where a deployment is being updated.

Current Engine​

The current engine is your default engine. Currently, it is used as the default for:

  • Dataset upload
  • Model export
  • PQL queries

Note: Your current engine can only be selected from the pool of General engines.