Basic Usage

This example demonstrates the basic usage and interface of a control environment in EPyT-Control.

from IPython.display import display, HTML
display(HTML('<a target=\"_blank\" href=\"https://colab.research.google.com/github/WaterFutures/EPyT-Control/blob/main/docs/examples/basic_usage.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>'))

%pip install epyt-control --quiet

Note: you may need to restart the kernel to use updated packages.

# Import a pre-defined control environment where the chlorine injection is to be controlled
from my_env import SimpleChlorineInjectionEnv

# Create new instance of the control environment
env = SimpleChlorineInjectionEnv()

/tmp/Hanoi.inp: 100%|##########| 9.63k/9.63k [00:00<00:00, 24.1MB/s]
/tmp/weekPat_30min.mat: 100%|##########| 622/622 [00:00<00:00, 2.35MB/s]
/tmp/yearOffset_30min.mat: 100%|##########| 281/281 [00:00<00:00, 1.03MB/s]

Inspect the observation space (i.e. input to the agent/controller) by accessing the observation_space property:

print(env.observation_space)  # Observations: 34 dimension real-valued vector

Box(-inf, inf, (34,), float32)

Inspect the observation space (i.e. input to the agent/controller) by accessing the action_space property:

print(env.action_space)      # Action: A single scalar between 0 and 10000

Box(0.0, 10000.0, (1,), float32)

Reset the environment by calling the reset() function:

obs, _ = env.reset()
print(obs)

[ 1.1227845e+04  1.1101812e+04  4.6092886e+03  4.5289849e+03
  3.8234822e+03  3.3415984e+03  2.6894299e+03  2.2702065e+03
  1.9046804e+03  9.9389673e+02  6.8829651e+02  3.8027267e+02
  5.7851605e+02  9.3580475e+00 -8.3752792e+01  4.3202118e+02
 -6.0583356e+02 -1.1959899e+03 -1.2280574e+03  4.7628770e+03
  8.7186487e+02  3.0258121e+02  3.2158037e+03  1.7553257e+03
  1.4964138e+03 -6.9883972e+02 -3.2474880e+01  1.7242969e+02
  4.0417154e+02  2.7232983e+02  4.6422138e+01 -1.2628763e+02
  1.9799408e+02  6.2584277e+02]

Execute some random actions by calling the step() function – note that a random action can be generated by calling the sample() function of the environment’s action space:

# Run some iterations -- note that autorest=True
for _ in range(20):
    # Pick a random action
    act = env.action_space.sample()

    # Apply the action, get a reward (to be maximized) and
    # new observations (i.e. sensor readings)
    obs, reward, terminated, _, _ = env.step(act)
    print(f"Action: {act}, Reward: {reward}")

Action: [5543.831], Reward: -9.396614074707031
Action: [8796.011], Reward: -9.109367370605469
Action: [7485.9565], Reward: -8.962181091308594
Action: [2464.8687], Reward: -8.97867202758789
Action: [2005.1232], Reward: -8.974919319152832
Action: [5263.681], Reward: -8.839492797851562
Action: [3301.5642], Reward: -8.720865249633789
Action: [4057.037], Reward: -8.720376968383789
Action: [2398.902], Reward: -8.80842399597168
Action: [1029.4977], Reward: -8.938753128051758
Action: [5309.22], Reward: -8.812617301940918
Action: [8388.496], Reward: -8.622573852539062
Action: [3040.5603], Reward: -8.682611465454102
Action: [1211.669], Reward: -8.740642547607422
Action: [931.327], Reward: -8.938386917114258
Action: [6742.1387], Reward: -8.73513412475586
Action: [3965.8188], Reward: -8.728279113769531
Action: [5934.339], Reward: -8.669855117797852
Action: [1179.0647], Reward: -8.868907928466797
Action: [4414.4487], Reward: -8.938741683959961

Do not forget to close the environment by calling the close() function:

env.close()