Soft-linking

Note

In brief — This page is the operational how-to for running OPEN-PROM’s external couplings: the iterative soft-link with the MAgPIE land-use model and the one-way/iterative soft-link with the METEOR climate emulator. It covers the relevant configuration switches, task IDs, paths, how to drive an iteration, and how convergence and checkpointing work in practice. For why these couplings exist and what they exchange, see the interlinkages chapter.

Soft-linking with MAgPIE

See also

For the rationale and the exchanged quantities (carbon-price signal, advanced-bioenergy demand, bioenergy price, AFOLU emissions), see Link with MAgPIE.

The OPEN-PROM ⇄ MAgPIE soft-link is driven by task 7 of the start.R harness. It runs the two models as independently solved jobs and exchanges data between them as a round-based state machine: a cold round 0 solve, followed by zero or more hot rounds that each run a forward / MAgPIE / backward / hot sequence, looping until the shared boundary signals converge or a round cap is reached.

Prerequisites

• A working MAgPIE installation (the magpie/ sibling repository), with config.json:paths.magpie_path pointing at its root (trailing slash included).

• A MAgPIE project folder under <magpie_path>/e3m_projects/<project>/ containing a scenarios.csv. The project name is set in config.json:scenario.soft_link_magpie.project (the template default is uptake).

• A scenario_name, which is used as the label on both sides: it names the OPEN-PROM run folder and selects the MAgPIE subscenario (matched against the title column of the project’s scenarios.csv by exact or prefix match).

Running an iteration

Task 7 is batchable (one scenario per CSV row), so either form works:

Rscript start.R task_id=7        # single scenario from config.json:scenario
Rscript start.R scenarios.csv    # batch; only task_id 2 and 7 are batchable

The harness runs a cold round 0 and then iterates hot rounds k ≥ 1:

1. Round 0 — OPEN-PROM (cold). A first energy-system solve with the link disabled (--softLinkMAgPIE=off), producing blabla_round0.gdx. This is the baseline the forward coupling reads from.

2. Hot round k — four sequential phases, each checkpointed to coupling_state.json before the next begins:

   • forward — postprom::couplePromToMagpie() converts the previous round’s GDX into openprom_coupling.mif (the carbon-price and advanced-bioenergy-demand signals MAgPIE consumes).

   • magpie — MAgPIE is executed inside its project folder via Rscript e3m_start.R, reading the forwarded boundary; the harness diffs the output/ listing to capture the one new run directory.

   • backward — postprom::coupleMagpieToProm() reads MAgPIE’s report.mif and writes iPrices_magpie.csv (the bioenergy price fed back into GAMS) plus iEmissions_magpie.mif (AFOLU emissions for reporting only).

   • openprom_hot — a solve with the link enabled (--softLinkMAgPIE=on), so the biomass-and-waste price is fixed from the just-written iPrices_magpie.csv rather than from OPEN-PROM’s internal price dynamics. The result is saved as blabla_round{k}.gdx.

3. The hot-round sequence repeats. From round k ≥ 2 a convergence test runs after openprom_hot (see below); the loop exits at the first converged round, or when k reaches max_iter.

4. Finalisation. The last round’s GDX is copied blabla_round{final}.gdx → blabla.gdx, coupling_summary.json is written, and (per the behavior flags) reportOutput.R and run synchronisation run on the final solution.

Important

Only the bioenergy price (iPrices_magpie.csv) feeds back into the GAMS solve. The MAgPIE AFOLU emissions (iEmissions_magpie.mif) do not influence the energy-system solution; the softmif land-emission mode merely tags blabla.gdx so postprom sources AFOLU emissions from the MAgPIE .mif during reporting.

Configuration

The task-7 knobs live under config.json:scenario.soft_link_magpie (and the matching soft_link_magpie.* columns of scenarios.csv):

Key	Default	Meaning
project	uptake	MAgPIE project folder under <magpie_path>/e3m_projects/
existing_prom_run	null	path to an interrupted run folder to resume from (see below)
max_iter	1	maximum number of hot rounds
price_tol	0.05	relative tolerance on the bioenergy price (5%)
quant_tol	0.05	relative tolerance on the bioenergy quantity (5%)

Warning

The template default is max_iter = 1, which runs round 0 plus a single hot round and stops without ever reaching the convergence test (that test starts at round k ≥ 2). For a genuinely converged soft-link set max_iter to a larger value (e.g. 5) together with price_tol/quant_tol.

Convergence controls and state files

From the second hot round onward (k ≥ 2) the harness compares the exchanged bioenergy price and bioenergy quantity pathways against the previous round, on the natural shared boundary of the two models (the H12 land-use regions × years), restricted to years from the model start year (2024) onward. For each series it computes a per-cell relative change against a denominator floor (1.0 US$2017/GJ for price, 0.01 EJ/yr for quantity) and reduces it to a maximum, delta_price_max / delta_quant_max (the judges) and an L2 norm (diagnostic only). The coupling is declared converged only when both maxima fall below their tolerances:

converged  iff  delta_price_max < price_tol  AND  delta_quant_max < quant_tol

Otherwise another hot round runs until convergence or max_iter. A missing value on either side counts as not converged, so the loop never declares victory on an empty (region, year) grid.

State and diagnostics written into the run folder:

File	Purpose
blabla_round0.gdx	the cold round-0 OPEN-PROM solution
blabla_round{k}.gdx	the hot-round OPEN-PROM solution for round k
blabla.gdx	a copy of the final round, used by reportOutput
openprom_coupling.mif	the last forward boundary sent to MAgPIE (overwritten each round)
iPrices_magpie.csv	the last bioenergy price returned to GAMS (overwritten each round)
iEmissions_magpie.mif	the last AFOLU emissions returned to postprom (overwritten each round)
coupling_state.json	per-phase checkpoint (status, rounds, captured H12 series) enabling resume
convergence_log.csv	the per-round deviation trajectory (one row per round k ≥ 2)
coupling_summary.json	finalisation summary: status, thresholds, final deltas, timestamps

coupling_state.json is rewritten atomically after every phase, recording one of the statuses iterating, failed, converged, or max_iter. To resume an interrupted run, point config.json:scenario.soft_link_magpie.existing_prom_run at its run folder; the harness skips round 0 and continues from the highest round’s last completed phase. A resume is only allowed when the run is still iterating and its pinned config snapshot (max_iter, price_tol, quant_tol, scenario_name) matches the current configuration exactly; runs that already failed, converged, or hit max_iter cannot be resumed.

Emulator alternative (without running MAgPIE)

For cases where running the full MAgPIE model on every round is too costly, the land-use response can instead be represented by pre-fitted emulator curves, selected with the user switches at the top of main.gms. These are configured for ordinary single-solve tasks (0–6) through config.json:scenario.land_use_emulator, which start.R maps onto the GAMS flags --landUseEmulator and --emulatorGHGScen:

• softLinkMAgPIE (on/off) — the iterative task-7 soft-link above. When on it wins: the biomass price is fixed from MAgPIE and AFOLU emissions are read from MAgPIE’s .mif. The emulator switches below only take effect when softLinkMAgPIE is off.

• land_use_emulator.source → landUseEmulator (legacy/globiom/magpie) — selects the emulator source. legacy means no emulator (static biomass price, exogenous AFOLU emissions); globiom/magpie drive the biomass price from a fitted supply curve and AFOLU emissions from fitted land-use-emission curves.

• land_use_emulator.carbon_price → emulatorGHGScen (GHG000, GHG010, GHG020, GHG050, GHG100) — picks the active land-use carbon-price row in the emulator coefficient tables.

Tip

The three approaches form a hierarchy of increasing fidelity and cost: exogenous land-use assumptions (land_use_emulator.source = legacy) → emulator-based coupling (globiom/magpie) → full iterative OPEN-PROM ⇄ MAgPIE soft-linking (softLinkMAgPIE=on, task 7). The emulator route never launches an external land-use model; it only evaluates the fitted curves during the solve.

Soft-linking with climate emulators (METEOR)

See also

For an overview of METEOR and what the coupling adds (spatially explicit climate feedbacks, HDD/CDD impact indicators), see Link with METEOR.

The OPEN-PROM ⇄ METEOR link is an external post-processing pipeline rather than a built-in task: OPEN-PROM emissions trajectories are translated into spatial climate outcomes and impact indicators that are then fed back into selected model parameters.

Workflow

1. Run an OPEN-PROM scenario and export its emissions trajectories — CO₂, CH₄, N₂O and sulphur-related emissions — in an IAMC-compatible format.

2. Compute radiative forcing by passing the IAMC emissions to the C-SCM emissions-to-forcing module, which converts the trajectories into radiative-forcing time series.

3. Emulate the spatial climate response with METEOR, producing gridded projections of climate variables (temperature, precipitation) for the selected horizons, e.g. mid-century and end-of-century.

4. Post-process to OPEN-PROM regions — apply regional masks, aggregate the gridded fields to OPEN-PROM regions, compute regional annual or monthly averages, and derive impact indicators such as Heating Degree Days (HDD) and Cooling Degree Days (CDD).

5. Feed back into OPEN-PROM — use the regional HDD/CDD indicators to adjust final-energy-demand assumptions, most directly in the buildings (heating and cooling) sector.

Summarised as a chain:

OPEN-PROM emissions scenario → IAMC-formatted emissions → C-SCM forcing calculation
  → METEOR spatial climate emulation → regional aggregation and impact indicators → feedback to OPEN-PROM

One-way vs iterative use

The pipeline can be run as a one-way soft link, where METEOR outputs simply inform selected OPEN-PROM parameters for a single pass, or as an iterative coupling, where OPEN-PROM is rerun after the climate-impact indicators have been updated, letting climate impacts influence energy demand, technology deployment, emissions and the resulting mitigation pathway across successive rounds.

Warning

The most direct operational feedback currently implemented is the HDD/CDD adjustment to heating and cooling demand. Wider feedbacks (for example on water-system energy requirements or electricity peak loads) are part of the coupling’s intended scope but are not yet wired into the standard run.