unnötige variablen rausgelöscht
This commit is contained in:
Nicolai
parent
16633f25b1
commit
4bf7d47007
217
AGENTS.md
Normal file
217
AGENTS.md
Normal file
@ -0,0 +1,217 @@
|
|||||||
|
in Agents.md steht alles drin, was du brauchst# AGENTS.md
|
||||||
|
|
||||||
|
## Projektkontext
|
||||||
|
|
||||||
|
LEAG-COALLOG ist ein Proof of Concept zur Optimierung der Braunkohle-Logistik.
|
||||||
|
Das System verarbeitet eine Excel-Parameterdatei, erzeugt daraus normalisierte
|
||||||
|
Parquet-Tabellen und loest anschliessend ein Pyomo-Optimierungsmodell fuer
|
||||||
|
Lieferungen von Tagebauen zu Kraftwerken und Veredlung.
|
||||||
|
|
||||||
|
Ziel ist eine planbare Rohkohleverteilung unter Beruecksichtigung von Nachfrage,
|
||||||
|
Toleranzen, Mischungsverhaeltnissen, Foerder- und Verladungskapazitaeten,
|
||||||
|
Verfuegbarkeiten, Schichtmustern, Bunkerlogik und Solver-Limits. Die Webapp dient
|
||||||
|
als Bedienoberflaeche fuer Upload, Parametrisierung, Laufstatus, Logs,
|
||||||
|
Visualisierung und Ergebnisdownload.
|
||||||
|
|
||||||
|
## Architektur
|
||||||
|
|
||||||
|
Die Anwendung besteht aus drei Hauptteilen:
|
||||||
|
|
||||||
|
1. **Preprocessing**
|
||||||
|
- Datei: `src/preprocessing/exploration_preprocess.py`
|
||||||
|
- Liest die Excel-Datei aus `POC1_INPUT_XLSX`.
|
||||||
|
- Schreibt vorbereitete Tabellen als Parquet nach `POC1_OUTPUT_DIR`.
|
||||||
|
- Die Logik stammt aus einem Notebook-Export und ist stark an die Struktur der
|
||||||
|
Eingabe-Excel gebunden.
|
||||||
|
|
||||||
|
2. **Optimierung**
|
||||||
|
- Einstieg: `src/optimization/run_optimization.py`
|
||||||
|
- Modell: `src/optimization/model_builder.py`
|
||||||
|
- Laedt alle Parquet-Dateien aus einem Datenverzeichnis.
|
||||||
|
- Baut ein Pyomo `ConcreteModel`.
|
||||||
|
- Loest mit `highs`, `gurobi` oder `scip`.
|
||||||
|
- Exportiert `output.xlsx` und optional `warmstart.json`.
|
||||||
|
|
||||||
|
3. **Webapp**
|
||||||
|
- Backend: `webapp/backend/main.py`
|
||||||
|
- Frontend: `webapp/frontend/src/App.jsx`
|
||||||
|
- FastAPI nimmt Uploads entgegen, legt Job-Verzeichnisse unter `var/jobs` an
|
||||||
|
und startet Preprocessing und Optimierung als Subprozesse.
|
||||||
|
- React/Vite stellt Upload, Solver-Parameter, Job-Status, Logs, Plots,
|
||||||
|
Warmstart und Downloads bereit.
|
||||||
|
|
||||||
|
## Datenfluss
|
||||||
|
|
||||||
|
Standardablauf eines Webapp-Laufs:
|
||||||
|
|
||||||
|
1. Browser sendet Excel-Datei, Solver und Parameter an `POST /api/run`.
|
||||||
|
2. FastAPI erstellt ein Job-Verzeichnis unter `var/jobs/<job_id>/`.
|
||||||
|
3. Die Upload-Datei wird als `input.xlsx` gespeichert.
|
||||||
|
4. `exploration_preprocess.py` schreibt `processed/*.parquet`.
|
||||||
|
5. `run_optimization.py` liest `processed/*.parquet`, baut und loest das Modell.
|
||||||
|
6. Ergebnisse landen in `output/output.xlsx` und `output/warmstart.json`.
|
||||||
|
7. Logs liegen unter `logs/preprocess.log` und `logs/optimization.log`.
|
||||||
|
8. Das Frontend pollt Status, Logs und abgeleitete Visualisierungsdaten.
|
||||||
|
|
||||||
|
Wichtige API-Endpunkte:
|
||||||
|
|
||||||
|
- `GET /api/health`
|
||||||
|
- `POST /api/run`
|
||||||
|
- `GET /api/jobs/{job_id}`
|
||||||
|
- `POST /api/jobs/{job_id}/cancel`
|
||||||
|
- `GET /api/jobs/{job_id}/output`
|
||||||
|
- `GET /api/jobs/{job_id}/warmstart`
|
||||||
|
- `GET /api/jobs/{job_id}/monthly-flows`
|
||||||
|
- `GET /api/jobs/{job_id}/capacity-timeseries`
|
||||||
|
- `GET /api/jobs/{job_id}/logs/{log_name}`
|
||||||
|
- `GET /api/jobs/{job_id}/logs/{log_name}/stream`
|
||||||
|
|
||||||
|
## Optimierungsmodell
|
||||||
|
|
||||||
|
Das Modell bildet Liefermengen als ganzzahlige Schritte ab:
|
||||||
|
|
||||||
|
- Quellen `I`: `Reichwalde`, `Nochten`, `Welzow`
|
||||||
|
- Ziele `J`: `J`, `SP`, `B3`, `B4`, `V`
|
||||||
|
- Tage `D`: Bergbauwoche Samstag bis Freitag
|
||||||
|
- Schichten `S`: `F`, `S`, `N`
|
||||||
|
- Entscheidung `k[i,j,w,d,s]`: ganzzahlige Anzahl Lieferschritte
|
||||||
|
- Fluss `x[i,j,w,d,s] = step_size_tonnes * k[i,j,w,d,s]`
|
||||||
|
|
||||||
|
Wichtige Nebenbedingungen und Konzepte:
|
||||||
|
|
||||||
|
- Tages-, Wochen- und Monatstoleranzen fuer Kraftwerke
|
||||||
|
- separate Veredlungsbedarfe fuer Nochten- und Welzower-Kohle
|
||||||
|
- harte Mix-Min/Max-Grenzen und weiche Zielabweichungen
|
||||||
|
- Foerderkapazitaeten pro Monat
|
||||||
|
- Verladungskapazitaeten pro Schicht und Tag
|
||||||
|
- Verfuegbarkeiten aus `Verfuegbarkeiten.parquet`
|
||||||
|
- feste Routenverbote und kombinierte Flussgrenzen
|
||||||
|
- Schichtglaettung und Schichtmuster
|
||||||
|
- optionale Bunkerlogik fuer Ziele mit Bunkerparametern
|
||||||
|
- Warmstart Import/Export fuer Folgelaeufe
|
||||||
|
|
||||||
|
Weitere fachliche Details stehen in:
|
||||||
|
|
||||||
|
- `README.md`
|
||||||
|
- `docs/app-kommunikation.md`
|
||||||
|
- `notebooks/constraints_overview.md`
|
||||||
|
- `latex/modellierung.tex`
|
||||||
|
|
||||||
|
## Wichtige Verzeichnisse
|
||||||
|
|
||||||
|
- `src/preprocessing/`: Excel-zu-Parquet-Datenaufbereitung
|
||||||
|
- `src/optimization/`: Pyomo-Modell, Solver-Lauf, Ergebnisexport
|
||||||
|
- `webapp/backend/`: FastAPI-App und Job-Orchestrierung
|
||||||
|
- `webapp/frontend/`: React/Vite-Frontend
|
||||||
|
- `data/input/`: lokale Eingabedateien
|
||||||
|
- `data/processed/`: lokale Preprocessing-Ausgaben
|
||||||
|
- `data/out/`: lokale Ergebnisartefakte
|
||||||
|
- `var/jobs/`: persistente Webapp-Jobdaten
|
||||||
|
- `docs/`: Architektur- und Kommunikationsdokumentation
|
||||||
|
- `notebooks/`: Analyse- und Modellnotizen
|
||||||
|
- `results/`: Solver-Debugartefakte wie `iis.ilp`
|
||||||
|
|
||||||
|
## Lokale Kommandos
|
||||||
|
|
||||||
|
Backend starten:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python -m uvicorn webapp.backend.main:app --reload
|
||||||
|
```
|
||||||
|
|
||||||
|
Frontend starten:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd webapp/frontend
|
||||||
|
npm install
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
Preprocessing direkt ausfuehren:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python src/preprocessing/exploration_preprocess.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Optimierung direkt ausfuehren:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python src/optimization/run_optimization.py --data-dir data/processed --solver highs
|
||||||
|
```
|
||||||
|
|
||||||
|
Docker-All-in-One:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
## Laufzeit und Solver
|
||||||
|
|
||||||
|
Unterstuetzte Solver sind aktuell `highs`, `gurobi` und `scip`, sofern lokal
|
||||||
|
installiert und fuer Pyomo verfuegbar. Im Docker-Image wird `gurobipy`
|
||||||
|
installiert. Fuer Gurobi wird entweder eine Lizenzdatei ueber
|
||||||
|
`GRB_LICENSE_FILE` oder WLS-Umgebungsvariablen verwendet:
|
||||||
|
|
||||||
|
- `GRB_LICENSE_FILE`
|
||||||
|
- `GRB_WLSACCESSID`
|
||||||
|
- `GRB_WLSSECRET`
|
||||||
|
- `GRB_LICENSEID`
|
||||||
|
|
||||||
|
Die Webapp prueft Solver-Verfuegbarkeit ueber `/api/health`.
|
||||||
|
|
||||||
|
## Entwicklungsregeln fuer Agenten
|
||||||
|
|
||||||
|
- Aendere keine Eingabe-, Ergebnis- oder Jobdaten ohne expliziten Grund.
|
||||||
|
- Behandle `var/jobs`, `data/out`, `data/processed` und `results` als erzeugte
|
||||||
|
Artefakte, sofern die Aufgabe nicht genau diese Dateien betrifft.
|
||||||
|
- Respektiere bestehende fachliche Logik im Optimierungsmodell. Viele Grenzen
|
||||||
|
sind hart kodiert und fachlich motiviert.
|
||||||
|
- Bei Aenderungen an `model_builder.py` immer pruefen, ob Export,
|
||||||
|
Visualisierung und Warmstart noch konsistent sind.
|
||||||
|
- Bei Aenderungen am Excel-Output die Backend-Parser in `webapp/backend/main.py`
|
||||||
|
mitdenken. Sie lesen konkrete Sheet- und Spaltenstrukturen.
|
||||||
|
- Bei Aenderungen an API-Antworten das Frontend in `webapp/frontend/src/App.jsx`
|
||||||
|
synchron halten.
|
||||||
|
- Die Preprocessing-Datei ist notebookartig und teilweise redundant. Kleine,
|
||||||
|
lokale Korrekturen sind besser als grosse Refactors ohne Tests.
|
||||||
|
- Keine Gurobi-Lizenzdateien, Credentials oder lokale Pfade committen.
|
||||||
|
- UI und Backend muessen auch ohne Gurobi sinnvoll starten koennen, wenn HiGHS
|
||||||
|
verfuegbar ist.
|
||||||
|
|
||||||
|
## Validierung
|
||||||
|
|
||||||
|
Es gibt derzeit keine klar etablierte automatisierte Testsuite. Je nach Aenderung
|
||||||
|
sind diese Checks sinnvoll:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python src/preprocessing/exploration_preprocess.py
|
||||||
|
uv run python src/optimization/run_optimization.py --data-dir data/processed --solver highs --time-limit 60
|
||||||
|
cd webapp/frontend && npm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
Bei Backend- oder Webapp-Aenderungen zusaetzlich:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python -m uvicorn webapp.backend.main:app --reload
|
||||||
|
```
|
||||||
|
|
||||||
|
Dann im Browser oder per API pruefen:
|
||||||
|
|
||||||
|
- `/api/health`
|
||||||
|
- Upload ueber Frontend
|
||||||
|
- Statuspolling
|
||||||
|
- Logs
|
||||||
|
- Download von `output.xlsx`
|
||||||
|
- Download von `warmstart.json`
|
||||||
|
- Plots fuer Monatsfluesse und Kapazitaetszeitreihen
|
||||||
|
|
||||||
|
## Bekannte Besonderheiten
|
||||||
|
|
||||||
|
- Die Bergbauwoche beginnt fachlich am Samstag. Im Modell wird dafuer ein
|
||||||
|
Datum-plus-zwei-Tage-Shift zur ISO-Woche verwendet.
|
||||||
|
- Lieferabweichungen und Mischungsverhaeltnisse beziehen sich auf Lieferungen,
|
||||||
|
nicht auf Bunkerabfluss.
|
||||||
|
- `no_three_in_a_row` ist laut README aktuell auf `<= 3` gelockert.
|
||||||
|
- Bei Infeasibility kann Gurobi ein IIS nach `results/iis.ilp` schreiben.
|
||||||
|
- Der Projektname in Metadaten und README ist teilweise noch `POC1`; fachlich
|
||||||
|
entspricht das dem aktuellen LEAG-COALLOG-Prototyp.
|
||||||
252
COALLOG.drawio
Normal file
252
COALLOG.drawio
Normal file
@ -0,0 +1,252 @@
|
|||||||
|
<mxfile host="app.diagrams.net">
|
||||||
|
<diagram name="Seite-1" id="P-KqJ_aeJxb4NgLkL2IL">
|
||||||
|
<mxGraphModel dx="1132" dy="605" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="0" />
|
||||||
|
<mxCell id="1" parent="0" />
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-1" parent="1" style="whiteSpace=wrap;strokeWidth=2;" value="Benutzer" vertex="1">
|
||||||
|
<mxGeometry height="54" width="124" x="20" y="275" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-2" parent="1" style="whiteSpace=wrap;strokeWidth=2;fillColor=#f8cecc;strokeColor=#b85450;" value="Excel Input
Planungsdaten" vertex="1">
|
||||||
|
<mxGeometry height="78" width="164" x="322" y="222" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-3" parent="1" style="whiteSpace=wrap;strokeWidth=2;fillColor=#ffe6cc;strokeColor=#d79b00;" value="React Frontend
Vite Build / App.jsx" vertex="1">
|
||||||
|
<mxGeometry height="78" width="200" x="672" y="263" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-4" parent="1" style="whiteSpace=wrap;strokeWidth=2;fillColor=#ffe6cc;strokeColor=#d79b00;" value="FastAPI Backend
webapp/backend/main.py" vertex="1">
|
||||||
|
<mxGeometry height="78" width="250" x="1128" y="257" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-5" parent="1" style="shape=cylinder3;boundedLbl=1;backgroundOutline=1;size=10;strokeWidth=2;whiteSpace=wrap;fillColor=#f8cecc;strokeColor=#b85450;" value="var/jobs
Job-Status, Uploads, Logs, Outputs" vertex="1">
|
||||||
|
<mxGeometry height="134" width="215" x="1628" y="217" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-6" parent="1" style="whiteSpace=wrap;strokeWidth=2;fillColor=#dae8fc;strokeColor=#6c8ebf;" value="Preprocessing
src/preprocessing/exploration_preprocess.py" vertex="1">
|
||||||
|
<mxGeometry height="78" width="386" x="2060" y="88" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-7" parent="1" style="whiteSpace=wrap;strokeWidth=2;fillColor=#dae8fc;strokeColor=#6c8ebf;" value="Optimierung
src/optimization/run_optimization.py" vertex="1">
|
||||||
|
<mxGeometry height="78" width="333" x="3082" y="249" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-8" parent="1" style="whiteSpace=wrap;strokeWidth=2;fillColor=#e1d5e7;strokeColor=#9673a6;" value="Pyomo Modell
src/optimization/model_builder.py" vertex="1">
|
||||||
|
<mxGeometry height="78" width="312" x="3665" y="221" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-9" parent="1" style="whiteSpace=wrap;strokeWidth=2;fillColor=light-dark(#eeeeee,#1f2020);strokeColor=light-dark(#999999,#cccccc);fontColor=light-dark(#333333,#cccccc);" value="Solver
HiGHS / Gurobi / SCIP" vertex="1">
|
||||||
|
<mxGeometry height="78" width="219" x="4227" y="221" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-10" parent="1" style="shape=cylinder3;boundedLbl=1;backgroundOutline=1;size=10;strokeWidth=2;whiteSpace=wrap;fillColor=#f8cecc;strokeColor=#b85450;" value="processed Parquet
Job-Verzeichnis" vertex="1">
|
||||||
|
<mxGeometry height="103" width="146" x="2696" y="236" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-11" parent="1" style="shape=cylinder3;boundedLbl=1;backgroundOutline=1;size=10;strokeWidth=2;whiteSpace=wrap;fillColor=#f8cecc;strokeColor=#b85450;" value="output.xlsx
warmstart.json" vertex="1">
|
||||||
|
<mxGeometry height="100" width="124" x="3759" y="465" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-12" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-1" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.17;entryX=0;entryY=0.5;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-2" value="lädt Excel hoch">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="224" y="261" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-13" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-2" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.5;entryX=0;entryY=0.22;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-3" value="Datei auswählen">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="588" y="261" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-14" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-3" style="curved=1;startArrow=none;endArrow=block;exitX=0.66;exitY=0.01;entryX=0.36;entryY=0.01;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="POST /api/run
Excel + Parameter + optional Warmstart">
|
||||||
|
<mxGeometry relative="1" x="0.2387" y="-60" as="geometry">
|
||||||
|
<mxPoint as="offset" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1000" y="20" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-15" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-4" style="curved=1;startArrow=none;endArrow=block;exitX=0.37;exitY=1.01;entryX=0.65;entryY=1.01;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-3" value="job_id / Status / Daten / Downloads">
|
||||||
|
<mxGeometry relative="1" x="-0.3341" y="-88" as="geometry">
|
||||||
|
<mxPoint as="offset" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1000" y="600" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-16" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-4" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.07;entryX=0;entryY=0.31;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-5" value="Job-Verzeichnis, input.xlsx, job.json schreiben">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1503" y="228" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-17" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-5" style="curved=1;startArrow=none;endArrow=block;exitX=0.87;exitY=0;entryX=0;entryY=0.28;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-6" value="input.xlsx bereitstellen">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1951" y="100" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-18" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-4" style="curved=1;startArrow=none;endArrow=block;exitX=0.75;exitY=0.01;entryX=0;entryY=0.64;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-6" value="Subprozess starten">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1503" y="143" />
|
||||||
|
<mxPoint x="1951" y="143" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-19" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-6" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.5;entryX=0.06;entryY=0;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-10" value="processed/*.parquet schreiben">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="2571" y="127" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-20" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-4" style="curved=1;startArrow=none;endArrow=block;exitX=0.65;exitY=0.01;entryX=0.36;entryY=0;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-7" value="Subprozess starten">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1503" y="41" />
|
||||||
|
<mxPoint x="2962" y="41" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-21" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-10" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.5;entryX=0;entryY=0.5;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-7" value="processed/*.parquet lesen">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-22" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-7" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.13;entryX=0;entryY=0.34;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-8" value="Modell bauen">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="3540" y="238" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-23" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-8" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.26;entryX=0;entryY=0.3;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-9" value="Optimierungsproblem senden">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="4102" y="226" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-24" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-9" style="curved=1;startArrow=none;endArrow=block;exitX=0;exitY=0.7;entryX=1;entryY=0.74;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-8" value="Lösung / Status zurückgeben">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="4102" y="294" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-25" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-8" style="curved=1;startArrow=none;endArrow=block;exitX=0.18;exitY=1;entryX=0.92;entryY=1;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-7" value="Lösung bereitstellen">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="3540" y="369" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-26" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-7" style="curved=1;startArrow=none;endArrow=block;exitX=0.66;exitY=1;entryX=0;entryY=0.47;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-11" value="output.xlsx + warmstart.json schreiben">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="3540" y="501" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-27" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-7" style="curved=1;startArrow=none;endArrow=block;exitX=0.31;exitY=1;entryX=0.87;entryY=1;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-5" value="optimization.log schreiben">
|
||||||
|
<mxGeometry relative="1" x="-0.0008" as="geometry">
|
||||||
|
<mxPoint as="offset" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="2962" y="467" />
|
||||||
|
<mxPoint x="1951" y="467" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-28" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-4" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.51;entryX=0;entryY=0.54;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-5" value="job.json aktualisieren">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1503" y="296" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-29" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-3" style="curved=1;startArrow=none;endArrow=block;exitX=0.71;exitY=0.01;entryX=0.31;entryY=0.01;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="GET /api/jobs/job_id">
|
||||||
|
<mxGeometry relative="1" x="0.1857" y="-49" as="geometry">
|
||||||
|
<mxPoint as="offset" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1000" y="88" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-30" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-3" style="curved=1;startArrow=none;endArrow=block;exitX=0.78;exitY=0.01;entryX=0.24;entryY=0.01;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="GET /api/jobs/job_id/logs/name">
|
||||||
|
<mxGeometry relative="1" x="-0.1467" y="-38" as="geometry">
|
||||||
|
<mxPoint as="offset" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1000" y="144" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-31" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-3" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.06;entryX=0;entryY=0.05;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="GET /api/jobs/job_id/monthly-flows">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1000" y="224" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-32" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-3" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.58;entryX=0;entryY=0.63;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="GET /api/jobs/job_id/capacity-timeseries">
|
||||||
|
<mxGeometry relative="1" x="0.015" y="26" as="geometry">
|
||||||
|
<mxPoint as="offset" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1000" y="316" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-33" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-3" style="curved=1;startArrow=none;endArrow=block;exitX=0.97;exitY=1.01;entryX=0.1;entryY=1.01;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="GET /api/jobs/job_id/output">
|
||||||
|
<mxGeometry relative="1" x="-0.1741" y="43" as="geometry">
|
||||||
|
<mxPoint as="offset" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1000" y="396" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-34" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-3" style="curved=1;startArrow=none;endArrow=block;exitX=0.77;exitY=1.01;entryX=0.26;entryY=1.01;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="GET /api/jobs/job_id/warmstart">
|
||||||
|
<mxGeometry relative="1" x="-0.2589" y="69" as="geometry">
|
||||||
|
<mxPoint as="offset" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1000" y="464" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-35" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-3" style="curved=1;startArrow=none;endArrow=block;exitX=0.69;exitY=1.01;entryX=0.33;entryY=1.01;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="POST /api/jobs/job_id/cancel">
|
||||||
|
<mxGeometry relative="1" x="-0.3176" y="79" as="geometry">
|
||||||
|
<mxPoint as="offset" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1000" y="532" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-36" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-4" style="curved=1;startArrow=none;endArrow=block;exitX=1;exitY=0.79;entryX=0;entryY=0.7;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-5" value="job.json / Logs lesen">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="1503" y="340" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-37" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-11" style="curved=1;startArrow=none;endArrow=block;exitX=0;exitY=0.6;entryX=0.65;entryY=1.01;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="output.xlsx / warmstart.json lesen">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="3540" y="559" />
|
||||||
|
<mxPoint x="1503" y="559" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-38" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-10" style="curved=1;startArrow=none;endArrow=block;exitX=0;exitY=0.93;entryX=0.85;entryY=1.01;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-4" value="Kapazitaetsdaten lesen">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="2571" y="408" />
|
||||||
|
<mxPoint x="1503" y="408" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="LSBXT6lm1qTOuGE88BYh-39" edge="1" parent="1" source="LSBXT6lm1qTOuGE88BYh-3" style="curved=1;startArrow=none;endArrow=block;exitX=0;exitY=0.9;entryX=1;entryY=0.97;rounded=0;" target="LSBXT6lm1qTOuGE88BYh-1" value="zeigt Status, Logs, Diagramme, Downloadlinks">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="588" y="359" />
|
||||||
|
<mxPoint x="224" y="359" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
</mxfile>
|
||||||
BIN
COALLOG_v0.1.png
Normal file
BIN
COALLOG_v0.1.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 258 KiB |
102
docs/app-kommunikation.md
Normal file
102
docs/app-kommunikation.md
Normal file
@ -0,0 +1,102 @@
|
|||||||
|
# App-Kommunikation
|
||||||
|
|
||||||
|
Dieses Diagramm zeigt die Kommunikation zwischen Browser, FastAPI-Backend,
|
||||||
|
Job-Ablage und der Python-Optimierungspipeline.
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
user[Benutzer im Browser]
|
||||||
|
frontend[React Frontend<br/>Vite Build / App.jsx]
|
||||||
|
backend[FastAPI Backend<br/>webapp/backend/main.py]
|
||||||
|
jobs[(var/jobs<br/>Job-Status, Uploads, Logs, Outputs)]
|
||||||
|
preprocess[Preprocessing<br/>src/preprocessing/exploration_preprocess.py]
|
||||||
|
optimization[Optimierung<br/>src/optimization/run_optimization.py]
|
||||||
|
model[Pyomo Modell<br/>src/optimization/model_builder.py]
|
||||||
|
solver[Solver<br/>HiGHS / Gurobi / SCIP]
|
||||||
|
processed[(processed Parquet<br/>Job-Verzeichnis)]
|
||||||
|
output[(output.xlsx<br/>warmstart.json)]
|
||||||
|
|
||||||
|
user --> frontend
|
||||||
|
frontend -->|POST /api/run<br/>Excel + Parameter + optional Warmstart| backend
|
||||||
|
backend -->|Job-Verzeichnis anlegen| jobs
|
||||||
|
backend -->|Subprozess starten| preprocess
|
||||||
|
preprocess -->|liest Excel<br/>schreibt vorbereitete Tabellen| processed
|
||||||
|
backend -->|Subprozess starten| optimization
|
||||||
|
optimization -->|lädt Tabellen| processed
|
||||||
|
optimization --> model
|
||||||
|
model --> solver
|
||||||
|
solver -->|Lösung| model
|
||||||
|
optimization -->|Excel + Warmstart exportieren| output
|
||||||
|
backend -->|Status completed / failed / cancelled| jobs
|
||||||
|
|
||||||
|
frontend -->|GET /api/jobs/{job_id}| backend
|
||||||
|
frontend -->|GET /api/jobs/{job_id}/logs/{name}| backend
|
||||||
|
frontend -->|GET /api/jobs/{job_id}/monthly-flows| backend
|
||||||
|
frontend -->|GET /api/jobs/{job_id}/capacity-timeseries| backend
|
||||||
|
frontend -->|GET /api/jobs/{job_id}/output| backend
|
||||||
|
frontend -->|GET /api/jobs/{job_id}/warmstart| backend
|
||||||
|
frontend -->|POST /api/jobs/{job_id}/cancel| backend
|
||||||
|
|
||||||
|
backend -->|liest job.json / Logs| jobs
|
||||||
|
backend -->|liest output.xlsx| output
|
||||||
|
backend -->|liest processed Parquet| processed
|
||||||
|
```
|
||||||
|
|
||||||
|
## Ablauf eines Optimierungslaufs
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant U as Benutzer
|
||||||
|
participant F as React Frontend
|
||||||
|
participant B as FastAPI Backend
|
||||||
|
participant P as Preprocessing
|
||||||
|
participant O as Optimierung
|
||||||
|
participant S as Solver
|
||||||
|
participant V as var/jobs
|
||||||
|
|
||||||
|
U->>F: Excel-Datei und Solver-Parameter auswaehlen
|
||||||
|
F->>B: POST /api/run
|
||||||
|
B->>V: Job-Verzeichnis, input.xlsx, job.json anlegen
|
||||||
|
B-->>F: job_id zurueckgeben
|
||||||
|
B->>P: exploration_preprocess.py als Subprozess
|
||||||
|
P->>V: processed/*.parquet und preprocess.log schreiben
|
||||||
|
B->>O: run_optimization.py als Subprozess
|
||||||
|
O->>S: Pyomo-Modell loesen
|
||||||
|
S-->>O: Loesung / Status
|
||||||
|
O->>V: output.xlsx, warmstart.json, optimization.log schreiben
|
||||||
|
B->>V: job.json auf completed/failed/cancelled setzen
|
||||||
|
|
||||||
|
loop Polling waehrend der Laufzeit
|
||||||
|
F->>B: GET /api/jobs/{job_id}
|
||||||
|
B-->>F: Status
|
||||||
|
F->>B: GET /api/jobs/{job_id}/logs/optimization
|
||||||
|
B-->>F: Logtext
|
||||||
|
end
|
||||||
|
|
||||||
|
F->>B: GET /api/jobs/{job_id}/monthly-flows
|
||||||
|
B-->>F: Monatsfluesse fuer Plotly
|
||||||
|
F->>B: GET /api/jobs/{job_id}/capacity-timeseries
|
||||||
|
B-->>F: Kapazitaets-Zeitreihen fuer Plotly
|
||||||
|
F->>B: GET /api/jobs/{job_id}/output
|
||||||
|
B-->>F: output.xlsx Download
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deployment-Sicht
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TB
|
||||||
|
subgraph docker[Docker Container leag-coallog]
|
||||||
|
nginxless[FastAPI / Uvicorn<br/>Port 8080]
|
||||||
|
static[React Static Build<br/>webapp/frontend/dist]
|
||||||
|
python[Python Runtime<br/>Preprocessing + Optimierung]
|
||||||
|
gurobi[Gurobi / HiGHS / SCIP]
|
||||||
|
end
|
||||||
|
|
||||||
|
browser[Browser] -->|HTTP :8080| nginxless
|
||||||
|
nginxless -->|liefert statische Dateien| static
|
||||||
|
nginxless -->|API-Aufrufe /api/*| python
|
||||||
|
python --> gurobi
|
||||||
|
python --> volume[(Host Volume ./var:/app/var)]
|
||||||
|
license[(Host ./licenses:/app/licenses:ro<br/>oder GRB_WLS*)] --> gurobi
|
||||||
|
```
|
||||||
|
|
||||||
@ -906,29 +906,6 @@ def build_model(
|
|||||||
model.W, model.D, rule=lambda m, w, d: m.devV_W_pos[w, d] - m.devV_W_neg[w, d] == m.devV_W[w, d]
|
model.W, model.D, rule=lambda m, w, d: m.devV_W_pos[w, d] - m.devV_W_neg[w, d] == m.devV_W[w, d]
|
||||||
)
|
)
|
||||||
|
|
||||||
model.lambda_glatt = pyo.Param(initialize=1e5, mutable=True, within=pyo.NonNegativeReals)
|
|
||||||
|
|
||||||
model.m_ijwd = pyo.Expression(
|
|
||||||
model.I,
|
|
||||||
model.J,
|
|
||||||
model.W,
|
|
||||||
model.D,
|
|
||||||
rule=lambda m, i, j, w, d: (1 / len(m.S))
|
|
||||||
* sum(m.x[i, j, w, d, s] for s in m.S if (i, j, w, d, s) in m.x),
|
|
||||||
)
|
|
||||||
|
|
||||||
model.t_glatt = pyo.Var(model.I, model.J, model.W, model.D, model.S, domain=pyo.NonNegativeReals)
|
|
||||||
|
|
||||||
def glatt_hi(m, i, j, w, d, s):
|
|
||||||
return m.t_glatt[i, j, w, d, s] >= m.x[i, j, w, d, s] - m.m_ijwd[i, j, w, d]
|
|
||||||
|
|
||||||
def glatt_lo(m, i, j, w, d, s):
|
|
||||||
return m.t_glatt[i, j, w, d, s] >= -(m.x[i, j, w, d, s] - m.m_ijwd[i, j, w, d])
|
|
||||||
|
|
||||||
# Constraint: Upper absolute deviation from average shift flow (smoothness).
|
|
||||||
model.glatt_hi = pyo.Constraint(model.I, model.J, model.W, model.D, model.S, rule=glatt_hi)
|
|
||||||
# Constraint: Lower absolute deviation from average shift flow (smoothness).
|
|
||||||
model.glatt_lo = pyo.Constraint(model.I, model.J, model.W, model.D, model.S, rule=glatt_lo)
|
|
||||||
|
|
||||||
lambda_dev = 100_000
|
lambda_dev = 100_000
|
||||||
lambda_v = 100_000
|
lambda_v = 100_000
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user