Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: run prototype with command line tool #36

Merged
merged 36 commits into from
Aug 28, 2024
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
144616d
InputComponent added work in progress
vargastat Mar 27, 2024
1a66ac9
wip components parsing corrected
vargastat Mar 28, 2024
4e156f0
port_connections to be defined
vargastat Mar 28, 2024
27e8377
wip
vargastat Apr 2, 2024
c7ccaa6
resolve_components.py completed
vargastat Apr 2, 2024
0153969
git commit -m 'connections from list to Dict'
vargastat Apr 3, 2024
c888296
consistency check between components and models added
vargastat Apr 4, 2024
8b1872e
wip
vargastat Apr 4, 2024
23c31d8
wip
vargastat Apr 5, 2024
c778b1f
build_network and build_database added
vargastat Apr 5, 2024
551d4e6
build network and read data from yaml
vargastat Apr 6, 2024
c25a2d0
timeseries read with pandas
vargastat Apr 8, 2024
c4579d0
timeseries read with pandas
vargastat Apr 8, 2024
22a62d2
test_evaluate_time_series corrected
vargastat Apr 8, 2024
2e77a30
rebase from main
vargastat Apr 15, 2024
c3413e8
mypy.ini modified to ignore missing imports pandas
vargastat Apr 15, 2024
f038183
Use pandas-stubs
sylvlecl Apr 15, 2024
f537fd1
TimeScenarioSeriesData refacto
vargastat Apr 16, 2024
1eb5c2e
test corrected to use dataframe for TimeScenarioSeriesData
vargastat Apr 17, 2024
b1c7a1e
TimeScenarioSeriesData modified for dataframe
vargastat Apr 17, 2024
f0a9bcf
pandas-stub <=2.0.3 for dependency issue
vargastat Apr 18, 2024
aff0de8
test test_short_term_storage_base_with_yaml added for components with…
vargastat Apr 23, 2024
336336d
timeseries provided as parameter without file extension
vargastat Apr 25, 2024
24c0bfc
add executable main file
ChouaneLouis May 6, 2024
b78bfea
add the --scenario parameter
ChouaneLouis May 7, 2024
29dd5e8
Merge branch 'main' into feature/executable
ChouaneLouis May 16, 2024
941d5f5
moved main code into a main() function
ChouaneLouis May 17, 2024
f45d135
project file structure implemented
ChouaneLouis May 21, 2024
db6eda1
add the possibility to make library import another library
ChouaneLouis May 23, 2024
8541707
Merge remote-tracking branch 'origin/main' into feature/executable
sylvlecl Jun 3, 2024
0e0b8bc
moved library resolving in the resolve_library module
ChouaneLouis Jun 5, 2024
4249f6b
add tests for library resolving
ChouaneLouis Jun 5, 2024
5a21ccd
add yaml libraries used in tests
ChouaneLouis Jun 5, 2024
532f7a4
Better organized command parsing
ianmnz Aug 28, 2024
2a1d478
Merge branch 'main' into feature/executable
ianmnz Aug 28, 2024
2d77cca
Added CLI
ianmnz Aug 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 138 additions & 0 deletions src/andromede/main/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Copyright (c) 2024, RTE (https://www.rte-france.com)
#
# See AUTHORS.txt
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
# SPDX-License-Identifier: MPL-2.0
#
# This file is part of the Antares project.
import argparse
from os import listdir
from pathlib import Path
from typing import List, Optional

from andromede.model.library import Library
from andromede.model.parsing import parse_yaml_library
from andromede.model.resolve_library import resolve_library
from andromede.simulation import OutputValues, TimeBlock, build_problem
from andromede.study import DataBase
from andromede.study.parsing import parse_yaml_components
from andromede.study.resolve_components import (
NetworkComponents,
build_data_base,
build_network,
consistency_check,
resolve_components_and_cnx,
)


class AntaresTimeSeriesImportError(Exception):
pass


def input_models(model_path: List[Path]) -> Library:
with model_path[0].open("r") as file:
lib = resolve_library(parse_yaml_library(file))
for path in model_path[1:]:
with path.open("r") as file:
lib = resolve_library(parse_yaml_library(file), [lib])
return lib


def input_database(study_path: Path, timeseries_path: Optional[Path]) -> DataBase:
with study_path.open() as comp:
return build_data_base(parse_yaml_components(comp), timeseries_path)


def input_components(study_path: Path, model: Library) -> NetworkComponents:
with study_path.open() as comp:
return resolve_components_and_cnx(parse_yaml_components(comp), model)


def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument(
"--study", type=Path, help="path to the root dirertory of the study"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo directory

)
parser.add_argument(
"--models", nargs="+", type=Path, help="list of path to model file, *.yml"
)
parser.add_argument(
"--component", type=Path, help="path to the component file, *.yml"
)
parser.add_argument(
"--timeseries", type=Path, help="path to the timeseries dirertory"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo directory

)
parser.add_argument(
"--duration", type=int, help="duration of the simulation", default=1
)
parser.add_argument(
"--scenario", type=int, help="number of scenario of the simulation", default=1
)

args = parser.parse_args()

components = None
database = None

if args.study:
if args.models or args.component or args.timeseries:
Copy link
Contributor

@ianmnz ianmnz May 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a way in ParseArgs to create exclusive groups, i.e., sets of arguments that can't be passed together so you would automatically check for this, but maybe it won't be enough for your needs here :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is but you can't make an argument excludes with other arguments which are not mutually exclusive

parser.error(
"--study flag can't be use with --models, --component and --timeseries"
)
components_path = args.study / "input" / "components" / "components.yml"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this logic (of parsing all data from one directory) to andromede.study.parsing

timeseries_dir = args.study / "input" / "components" / "series"
model_paths = [
args.study / "input" / "models" / file
for file in listdir(args.study / "input" / "models")
]

models = input_models(model_paths)
components = input_components(components_path, models)
consistency_check(components.components, models.models)
try:
database = input_database(components_path, timeseries_dir)
except UnboundLocalError:
raise AntaresTimeSeriesImportError(
f"An error occurred while importing time series. Are all timeseries files in {timeseries_dir} ?"
)

else:
if not args.models or not args.component:
parser.error("--models and --component must be entered")

models = input_models(args.models)
components = input_components(args.component, models)
consistency_check(components.components, models.models)

try:
database = input_database(args.component, args.timeseries)
except UnboundLocalError:
raise AntaresTimeSeriesImportError(
"An error occurred while importing time series. Did you correctly use the '--timeseries' parameter ?"
)

network = build_network(components)

for scenario in range(1, args.scenario + 1):
timeblock = TimeBlock(1, list(range(args.duration)))
try:
problem = build_problem(network, database, timeblock, scenario)
except IndexError as e:
raise IndexError(
str(e)
+ ". Did you correctly use the '--duration' and '--scenario' parameters ?"
)

status = problem.solver.Solve()
print("scenario ", scenario)
print("status : ", status)

print("avarage final cost : ", problem.solver.Objective().Value())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo "average"



if __name__ == "__main__":
main()
2 changes: 2 additions & 0 deletions src/andromede/model/resolve_library.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ def resolve_library(
port_types.extend(lib.port_types.values())
port_types_dict = dict((p.id, p) for p in port_types)
models = [_resolve_model(m, port_types_dict) for m in input_lib.models]
for lib in preloaded_libraries:
models.extend(lib.models.values())
return library(port_types, models)


Expand Down
2 changes: 2 additions & 0 deletions src/andromede/study/data.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,8 @@ def load_ts_from_txt(
ts_path = path_to_file / timeseries_with_extension
try:
return pd.read_csv(ts_path, header=None, sep="\s+")
except FileNotFoundError:
raise FileNotFoundError(f"File '{timeseries_name}' does not exist")
except Exception:
raise Exception(f"An error has arrived when processing '{ts_path}'")

Expand Down
83 changes: 83 additions & 0 deletions tests/functional/input/components/components.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Copyright (c) 2024, RTE (https://www.rte-france.com)
#
# See AUTHORS.txt
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
# SPDX-License-Identifier: MPL-2.0
#
# This file is part of the Antares project.


#test file for command line execution
#run :
#$ python src/andromede/main/main.py --models tests/functional/input/models/lib_1.yml tests/functional/input/models/lib_2.yml --component tests/functional/input/components/components.yml --duration 3 --timeseries tests/functional/input/components/series/
ianmnz marked this conversation as resolved.
Show resolved Hide resolved
# or
#$ python src/andromede/main/main.py --study tests/functional/ --duration 3
#expected value :
#> 0
#> 72000

study:
nodes:
- id: N
model: node

components:
- id: G
model: thermal-cluster-dhd
parameters:
- name: cost
type: constant
value: 100
- name: p_min
type: constant
value: 100
- name: p_max
type: constant
value: 500
- name: d_min_up
type: constant
value: 3
- name: d_min_down
type: constant
value: 3
- name: nb_units_max
type: constant
value: 1
- name: nb_failures
type: constant
value: 0
- id: D
model: demand
parameters:
- name: demand
type: timeseries
timeseries: demand-ts
- id: S
model: spillage
parameters:
- name: cost
type: constant
value: 10

connections:
- component1: N
port_1: injection_port
component2: D
port_2: injection_port

- component1: N
port_1: injection_port
component2: G
port_2: injection_port

- component1: N
port_1: injection_port
component2: S
port_2: injection_port



3 changes: 3 additions & 0 deletions tests/functional/input/components/series/demand-ts.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
500 100
0 100
0 100
67 changes: 67 additions & 0 deletions tests/functional/input/models/lib_1.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Copyright (c) 2024, RTE (https://www.rte-france.com)
#
# See AUTHORS.txt
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
# SPDX-License-Identifier: MPL-2.0
#
# This file is part of the Antares project.
library:
id: basic
description: Basic library

port-types:
- id: flow
description: A port which transfers power flow
fields:
- name: flow

models:

- id: thermal-cluster-dhd
parameters:
- name: p_max
- name: p_min
- name: cost
- name: d_min_up
- name: d_min_down
- name: nb_units_max
- name: nb_failures
variables:
- name: nb_units_on
lower-bound: 0
upper-bound: nb_units_max
variable-type: integer
- name: nb_starting
lower-bound: 0
upper-bound: nb_units_max
variable-type: integer
- name: nb_stoping
lower-bound: 0
upper-bound: nb_units_max
variable-type: integer
- name: production
lower-bound: 0
upper-bound: nb_units_max * p_max
ports:
- name: injection_port
type: flow
port-field-definitions:
- port: injection_port
field: flow
definition: production
constraints:
- name: max production
expression: production <= nb_units_on * p_max
- name: min production
expression: production >= nb_units_on * p_min
- name: on units variation
expression: nb_units_on = nb_units_on[-1] + nb_starting - nb_stoping
- name: starting time
expression: sum(nb_starting[-d_min_up + 1 .. 0]) <= nb_units_on
- name: stoping time
expression: sum(nb_stoping[-d_min_down + 1 .. 0]) <= nb_units_max - nb_units_on
objective: expec(sum(cost * production))
64 changes: 64 additions & 0 deletions tests/functional/input/models/lib_2.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Copyright (c) 2024, RTE (https://www.rte-france.com)
#
# See AUTHORS.txt
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
# SPDX-License-Identifier: MPL-2.0
#
# This file is part of the Antares project.
library:
id: basic
description: Basic library

port-types:
- id: flow
description: A port which transfers power flow
fields:
- name: flow

models:


- id: node
description: A basic balancing node model
ports:
- name: injection_port
type: flow
binding-constraints:
- name: balance
expression: sum_connections(injection_port.flow) = 0

- id: spillage
description: A basic spillage model
parameters:
- name: cost
time-dependent: false
scenario-dependent: false
variables:
- name: spillage
lower-bound: 0
ports:
- name: injection_port
type: flow
port-field-definitions:
- port: injection_port
field: flow
definition: -spillage
objective: expec(sum(cost * spillage))

- id: demand
description: A basic fixed demand model
parameters:
- name: demand
time-dependent: true
scenario-dependent: true
ports:
- name: injection_port
type: flow
port-field-definitions:
- port: injection_port
field: flow
definition: -demand
Loading