aito.client.v2.responses.V2EvaluationResponse

class aito.client.v2.responses.V2EvaluationResponse(json: Dict)

Bases: V2Response

An evaluation response — the metrics of a held-out evaluation

Parameters:

json (Dict) – the parsed JSON response

Methods

Attributes

accuracy	the share of test rows whose top prediction was correct
base_accuracy	the accuracy of always predicting the most common value
cases	the per-case results, when cases was selected
data	the metrics
engine	which engine answered, if the client requested meta
json	the raw JSON response
kind	the result kind this response class represents
meta	the response metadata, empty unless the client was built with meta=True
test_sample_count	the number of rows held out and predicted
train_sample_count	the number of rows Aito learned from during the evaluation
warnings	the non-fatal notes the engine attached to this response

property accuracy: float

the share of test rows whose top prediction was correct

Return type:

float

property base_accuracy: float

the accuracy of always predicting the most common value

The number accuracy has to beat to mean anything.

Return type:

float

property cases: List[Dict]

the per-case results, when cases was selected

Return type:

List[Dict]

property data: Dict

the metrics

Return type:

Dict

property engine: str | None

which engine answered, if the client requested meta

Return type:

Optional[str]

property json: Dict

the raw JSON response

Return type:

Dict

kind: str = 'evaluation'

the result kind this response class represents

property meta: Dict

the response metadata, empty unless the client was built with meta=True

Currently carries engine — v2 for a collection served by the rep2 engine, v1 for a legacy table served by the compatibility shim.

Return type:

Dict

property test_sample_count: int

the number of rows held out and predicted

Return type:

int

property train_sample_count: int

the number of rows Aito learned from during the evaluation

Return type:

int

property warnings: List[V2Warning]

the non-fatal notes the engine attached to this response

Empty for the great majority of responses. Worth checking when a result is unexpectedly broad or unexpectedly empty — the engine reports a where it could not apply here rather than failing the request.

Return type:

List[V2Warning]