epyt_control.evaluation
epyt_control.evaluation.evaluation
This module provides functions for evaluating policies/agents/control strategies on environments.
- epyt_control.evaluation.evaluation.evaluate_policy(env: RlEnv | Wrapper, policy: Callable[[ndarray], ndarray], n_max_iter: int = 10) tuple[list[float], ScadaData][source]
Evaluates a given policy/agent/control strategy for a given environment – i.e. the policy/agent is applied to the environment and the rewards and ScadaData observations over time are recorded.
- Parameters:
env –
The environment.
Note that in the case of a gymnasium.Wrapper instance, the underlying environment must be an instance of
RlEnv.policy (Callable[[numpy.ndarray], numpy.ndarray]) – Policy/Agent/Control strategy to be evaluated.
n_max_iter (int, optional) –
Upper bound on the number of iterations that is used for evaluating the given policy/agent.
The default is 1.
- Returns:
Tuple of rewards over time and a epyt_flow.simulation.ScadaData instance containing the WDN states over time.
- Return type:
tuple[list[float], epyt_flow.simulation.ScadaData]
epyt_control.evaluation.metrics
This module provides different metrics for evaluation.
- epyt_control.evaluation.metrics.accuracy_score(y_pred: ndarray, y: ndarray) float[source]
Computes the accuracy of a classification.
- Parameters:
y_pred (numpy.ndarray) – Predicted labels.
y – Ground truth labels.
- Returns:
Accuracy score.
- Return type:
float
- epyt_control.evaluation.metrics.f1_micro_score(y_pred: ndarray, y: ndarray) float[source]
Computes the F1 score using for a multi-class classification by counting the total true positives, false negatives and false positives.
- Parameters:
y_pred – Predicted labels.
y – Ground truth labels.
- Returns:
F1 score.
- Return type:
float
- epyt_control.evaluation.metrics.f1_score(y_pred: ndarray, y: ndarray) float[source]
Computes the F1-score for a binary classification.
- Parameters:
y_pred – Predicted labels.
y – Ground truth labels.
- Returns:
F1-score.
- Return type:
float
- epyt_control.evaluation.metrics.mape(y_pred: ndarray, y: ndarray, epsilon: float = 0.05) float[source]
Computes the Mean Absolute Percentage Error (MAPE).
- Parameters:
y_pred – Predicted outputs.
y – Ground truth outputs.
epsilon (float, optional) –
Small number added to predictions and ground truth to avoid division-by-zero.
The default is 0.05
- Returns:
MAPE score.
- Return type:
float
- epyt_control.evaluation.metrics.mase(y_pred: ndarray, y: ndarray, epsilon: float = 0.05) float[source]
Computes the Mean Absolute Scaled Error (MASE).
- Parameters:
y_pred – Predicted outputs.
y – Ground truth outputs.
epsilon (float, optional) –
Small number added to predictions and ground truth to avoid division-by-zero.
The default is 0.05
- Returns:
MASE score.
- Return type:
float
- epyt_control.evaluation.metrics.mean_squared_error(y_pred: ndarray, y: ndarray) float[source]
Computes the Mean Squared Error (MSE).
- Parameters:
y_pred – Predicted outputs.
y – Ground truth outputs.
- Returns:
MSE.
- Return type:
float
- epyt_control.evaluation.metrics.precision_score(y_pred: ndarray, y: ndarray) float[source]
Computes the precision of a classification.
- Parameters:
y_pred – Predicted labels.
y – Ground truth labels.
- Returns:
Precision score.
- Return type:
float
- epyt_control.evaluation.metrics.r2_score(y_pred: ndarray, y: ndarray) float[source]
Computes the R^2 score (also called the coefficient of determination).
- Parameters:
y_pred – Predicted outputs.
y – Ground truth outputs.
- Returns:
R^2 score.
- Return type:
float
- epyt_control.evaluation.metrics.roc_auc_score(y_pred: ndarray, y: ndarray) float[source]
Computes the Area Under the Curve (AUC) of a classification.
- Parameters:
y_pred – Predicted labels.
y – Ground truth labels.
- Returns:
ROC AUC score.
- Return type:
float
- epyt_control.evaluation.metrics.running_mse(y_pred: ndarray, y: ndarray) list[float][source]
Computes the running Mean Squared Error (MSE) – i.e. the MSE for every point in time.
- Parameters:
y_pred – Predicted outputs.
y – Ground truth outputs.
- Returns:
Running MSE.
- Return type:
float
- epyt_control.evaluation.metrics.running_r2_score(y_pred: ndarray, y: ndarray) list[float][source]
Computes and returns the running R^2 score – i.e. the R^2 score for every point in time.
- Parameters:
y_pred – Predicted outputs.
y – Ground truth outputs.
- Returns:
The running R^2 score.
- Return type:
list[float]
- epyt_control.evaluation.metrics.smape(y_pred: ndarray, y: ndarray, epsilon: float = 0.05) float[source]
Computes the Symmetric Mean Absolute Percentage Error (SMAPE).
- Parameters:
y_pred – Predicted outputs.
y – Ground truth outputs.
epsilon (float, optional) –
Small number added to predictions and ground truth to avoid division-by-zero.
The default is 0.05
- Returns:
SMAPE score.
- Return type:
float