EVALUATE
EVALUATE
is used to evaluate one or more target features from a trained model given a query input.
Syntax
Following is the grammar for making predictions:
EVALUATE target [, target ]* GIVEN query_input
target:
feature_name [ WITH METADATA ]
[ USING model_name [ VERSION model_version] ]
query_input:
SELECT feature_name [, feature_name]*
FROM dataset_name
[ WHERE where_condition ]
[ ORDER BY sort_col { ASC | DESC } [ LIMIT row_count ] ]
The *
syntax in the above grammar indicates that an element can repeat zero or more times.
Target features
Evaluates begins with a series of target features for metrics to be calculated on.
With property
The WITH
clause can specify additional properties for evaluation that are included as additional columns in the result set.
Following is the set of supported properties:
METADATA
: returns themodel_name
andmodel_version
loaded for the target.
Using clause
You can evaluate a model with a specific model and version if desired.
EVALUATE feature_name USING model_name [ VERSION model_version]
By default, PQL will select the default model for all targets, but a specific model name and version can also be provided.
Query input
The test dataset is provide using a SQL select statements eg:
GIVEN SELECT * FROM dataset_name WHERE split=0 LIMIT 10;
Examples
Following are a list of example predict queries, of which could be composed into more complex SELECT statements.
Basic evaluation
The simplest form of the evaluation query.
EVALUATE Survived GIVEN SELECT * from titanic
Compare evaluation
The WITH METADATA
property can be used in conjunction with a UNION
of two evaluation results to compare the default model for a given target vs version 1 of the titanic_small
model as show in the following example which returns the model with the highest accuracy:
with results as (
SELECT * FROM (
EVALUATE Survived WITH METADATA
GIVEN SELECT * from titanic
)
UNION ALL SELECT * FROM (
EVALUATE Survived WITH METADATA
USING titanic_small VERSION 1 -- specific model
GIVEN SELECT * from titanic
)
)
SELECT * FROM results
WHERE metric_name = 'accuracy'
ORDER BY metric_value DESC LIMIT 1