European High-Voltage Network — PyPSA-Eur Data

This notebook demonstrates two complete NPAP workflows on the real European high-voltage grid (220 kV – 750 kV, 35 countries) derived from OpenStreetMap data, as described in:

Xiong, B., Fioriti, D., Neumann, F., Riepin, I., & Brown, T. (2025). Modelling the high-voltage grid using open data for Europe and beyond. Scientific Data, 12(1). https://doi.org/10.1038/s41597-025-04550-7

The dataset is hosted on Zenodo: Xiong, B., Fioriti, D., Neumann, F., Riepin, I., & Brown, T. (2026). Prebuilt electricity network for PyPSA-Eur based on OpenStreetMap data. Zenodo. 10.5281/zenodo.18619025

We illustrate the full NPAP workflow for a simple voltage-unaware (Section 2) and voltage-aware (Section 3) treatment of the electricity network graph data. The following table provides an overview of the corresponding different features.

	Simple	Voltage-aware
Data loader	csv_files	va_loader
Edge types	all uniform	line / trafo / DC link
AC islands	✗	✓
Partition strategy	geographical_kmedoids_haversine	va_geographical_kmedoids_haversine
Per-type aggregation	✗	✓

1. Setup & Data Download

import networkx as nx
import pandas as pd
from npap.datasets import fetch_pypsa_eur

import npap
from npap import AggregationProfile

First, we download the network data from Zenodo and perform some minor preprocessing steps to follow NPAP’s naming conventions that can be checked in the fetch_pypsa_eur function.

data_files = fetch_pypsa_eur()

Dataset cached in /Users/taaarmar/.cache/npap (2.7 MB).
These files are kept for future runs and are never removed automatically. To delete them, run:
    from npap.datasets import clear_data_home; clear_data_home()

2. Simple (Voltage-Unaware) Workflow

The csv_files strategy only requires a node file and an edge file. Every edge is treated uniformly — no type differentiation, no AC-island detection. This makes it a fast starting point when the full grid topology is not needed.

2.1 Load

manager = npap.PartitionAggregatorManager()

graph = manager.load_data(
    strategy="csv_files",
    node_file=str(data_files["buses.csv"]),
    edge_file=str(data_files["lines.csv"]),
)

print(f"Graph type : {type(graph).__name__}")
print(f"Nodes      : {graph.number_of_nodes():,}")
print(f"Edges      : {graph.number_of_edges():,}")

Graph type : MultiDiGraph
Nodes      : 6,863
Edges      : 9,162

/Users/taaarmar/git/NPAP/npap/input/csv_loader.py:197: UserWarning: Parallel edges detected in CSV edge file. A MultiDiGraph will be created. Call manager.aggregate_parallel_edges() to collapse parallel edges before partitioning.
  log_warning(

Note: parallel edges detected The cell above will emit a warning: “Parallel edges detected. A MultiDiGraph will be created.”

This is expected: the PyPSA-Eur dataset contains many double-circuit lines, i.e., two transmission lines sharing the same pair of buses. The csv_files strategy preserves them as a MultiDiGraph. At the moment, NPAP can only partition DiGraph without parallel edges. Therefore, the parallel edges of the MultiDiGraph must be aggregated into a single equivalent before partitioning. For that, we need to provide edge strategies that define how the attributes are aggregated.

if isinstance(graph, nx.MultiDiGraph):
    edges_before = graph.number_of_edges()
    graph = manager.aggregate_parallel_edges(
        edge_properties={
            "x": "equivalent_reactance",
            "r": "equivalent_reactance",
            "s_nom": "sum",
            "length": "average",
        },
        default_strategy="sum",
        warn_on_defaults=False,
    )
    print(f"Edges before aggregation: {edges_before:,}")
    print(f"Edges after aggregation: {graph.number_of_edges():,}")
    print(f"Aggregated   : {edges_before - graph.number_of_edges():,} parallel edges")

Edges before aggregation: 9,162
Edges after aggregation: 8,065
Aggregated   : 1,097 parallel edges

Now we can illustrate the network. Note that the graph will not be fully connected, e.g., the UK and continental Europe, as the edges between them (DC links), are missing.

manager.plot_network(style="simple", title="European HV Grid — Simple")

2.2 Partitioning

We use geographical k-medoids with Haversine distance to create geographically compact clusters.

N_CLUSTERS = 50

partition = manager.partition(
    strategy="geographical_kmedoids_haversine",
    n_clusters=N_CLUSTERS,
)


cluster_sizes = pd.Series({k: len(v) for k, v in partition.mapping.items()})
print(f"Clusters created : {partition.n_clusters}")
print(
    f"Cluster size — min: {cluster_sizes.min()}, "
    f"max: {cluster_sizes.max()}, mean: {cluster_sizes.mean():.1f}"
)

Clusters created : 50
Cluster size — min: 34, max: 310, mean: 137.3

manager.plot_network(style="clustered", title=f"Simple — Partitioned ({N_CLUSTERS} clusters)")

2.3 Aggregation

Each cluster is collapsed into a single bus. Because the simple graph contains only AC lines (all edges have x and r), we apply the equivalent_reactance formula uniformly.

profile = AggregationProfile(
    topology_strategy="simple",
    node_properties={
        "lat": "average",
        "lon": "average",
        "voltage": "first",
    },
    edge_properties={
        "x": "equivalent_reactance",
        "r": "equivalent_reactance",
        "s_nom": "sum",
        "length": "average",
    },
    default_node_strategy="average",
    default_edge_strategy="sum",
    warn_on_defaults=False,
)

nodes_before = manager.get_current_graph().number_of_nodes()
edges_before = manager.get_current_graph().number_of_edges()

agg_graph = manager.aggregate(profile=profile)

print(
    f"Nodes: {nodes_before:,} -> {agg_graph.number_of_nodes():,} "
    f"({agg_graph.number_of_nodes() / nodes_before:.1%} of original)"
)
print(
    f"Edges: {edges_before:,} -> {agg_graph.number_of_edges():,} "
    f"({agg_graph.number_of_edges() / edges_before:.1%} of original)"
)

Nodes: 6,863 -> 50 (0.7% of original)
Edges: 8,065 -> 153 (1.9% of original)

/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'tags'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'geometry'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'symbol'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'under_construction'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'country'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'dc'. Falling back to first available value.
  log_warning(

Note: We receive a couple of warnings here, as we are aggregating over various node types, e.g., AC and DC nodes, with different attributes. While this is not a problem for most applications, if it should be avoided, one should account for it during partitioning.

manager.plot_network(style="simple", title=f"Simple — Aggregated ({N_CLUSTERS} nodes)")

3. Voltage-Aware Workflow

The va_loader ingests all five PyPSA-Eur files and builds a physically detailed grid: every edge is classified as an AC line, transformer, or DC link, and each bus is assigned to an AC island — the set of buses reachable without crossing a DC link. This richer representation enables voltage-aware partitioning that never merges buses from different islands or different voltage levels.

3.1 Load

va_manager = npap.PartitionAggregatorManager()

va_graph = va_manager.load_data(
    strategy="va_loader",
    node_file=str(data_files["buses.csv"]),
    line_file=str(data_files["lines.csv"]),
    transformer_file=str(data_files["transformers.csv"]),
    converter_file=str(data_files["converters.csv"]),
    link_file=str(data_files["links.csv"]),
)

print(f"Graph type : {type(va_graph).__name__}")
print(f"Nodes      : {va_graph.number_of_nodes():,}")
print(f"Edges      : {va_graph.number_of_edges():,}")

/Users/taaarmar/git/NPAP/npap/input/va_loader.py:900: UserWarning: 6 DC link(s) skipped due to missing references
  log_warning(
/Users/taaarmar/git/NPAP/npap/input/va_loader.py:213: UserWarning: Parallel edges detected in voltage-aware loader. A MultiDiGraph will be created. Call manager.aggregate_parallel_edges() to collapse parallel edges before partitioning.
  log_warning(
/Users/taaarmar/git/NPAP/npap/input/va_loader.py:373: UserWarning: Found 71 isolated node(s) with no connections. These will be removed.
  log_warning(

Graph type : MultiDiGraph
Nodes      : 6,792
Edges      : 10,073

Note — parallel edges detected Like with the simple loader, the va_loader returns a MultiDiGraph because the PyPSA-Eur dataset contains many double-circuit lines — two transmission lines sharing the same tower between the same pair of buses.

Unlike the simple case, the VA graph contains mixed edge types: AC lines and transformers both carry an x (reactance) attribute, but DC links do not. Because aggregate_parallel_edges processes every edge — not only parallel ones — the equivalent_reactance strategy would raise an error whenever it encounters a DC link group that has no x. We therefore use "average" for x and r: for single (non-parallel) AC edges, the average equals the original value; for parallel AC lines, it is a conservative approximation of the parallel-combination formula; and for DC links it falls back safely to 0.0.

if isinstance(va_graph, nx.MultiDiGraph):
    edges_before = va_graph.number_of_edges()
    va_graph = va_manager.aggregate_parallel_edges(
        edge_properties={
            "x": "average",
            "r": "average",
            "s_nom": "sum",
            "length": "average",
        },
        default_strategy="first",
        warn_on_defaults=False,
    )
    print(f"Edges before: {edges_before:,}")
    print(f"Edges after : {va_graph.number_of_edges():,}")
    print(f"Collapsed   : {edges_before - va_graph.number_of_edges():,} parallel edges")
else:
    print("No parallel edges found — va_graph is already a DiGraph.")

Edges before: 10,073
Edges after : 8,974
Collapsed   : 1,099 parallel edges

Note — voltage level harmonisation The raw dataset contains lines on different voltage levels of similar magnitude (e.g. 380 kV and 400 kV). The voltage-aware partitioning strategy treats each distinct voltage level independently, so those deviations would create spurious small or singleton clusters. Therefore, we first group all buses to the two main European transmission levels, i.e., 220 kV and 380 kV, before partitioning.

summary = va_manager.group_by_voltage_levels([220, 380])

print("Voltage distribution after grouping:")
for v, count in sorted(summary["voltage_distribution"].items()):
    print(f"  {v:>6.0f} kV: {count:>5,} buses")
print(f"\nNodes with inferred/missing voltage: {summary['missing_handled']}")

Voltage distribution after grouping:
     220 kV: 3,993 buses
     380 kV: 2,799 buses

Nodes with inferred/missing voltage: 0

va_manager.plot_network(style="voltage_aware", title="European HV Grid — Voltage-Aware Load")

3.2 Partitioning

We use the voltage-aware geographical k-medoids strategy with Haversine distance. This guarantees that:

• Buses in different AC islands are never merged.

• Buses at different voltage levels are never merged.

• Clusters are geographically compact.

va_partition = va_manager.partition(
    strategy="va_geographical_kmedoids_haversine",
    n_clusters=N_CLUSTERS,
)

cluster_sizes = pd.Series({k: len(v) for k, v in va_partition.mapping.items()})
print(f"Clusters created : {va_partition.n_clusters}")
print(
    f"Cluster size — min: {cluster_sizes.min()}, "
    f"max: {cluster_sizes.max()}, mean: {cluster_sizes.mean():.1f}"
)

Clusters created : 50
Cluster size — min: 4, max: 370, mean: 135.8

va_manager.plot_network(
    style="clustered", title=f"Voltage-Aware — Partitioned ({N_CLUSTERS} clusters)"
)

3.3 Aggregation

Each cluster of buses is collapsed into a single representative bus. Because the VA graph contains three distinct edge types, we use edge_type_properties to apply the physically correct strategy per type:

Edge type	Property	Strategy
line	x, r	equivalent_reactance
line	s_nom, b, circuits	sum
line	length	average
trafo	x	equivalent_reactance
trafo	s_nom	sum
dc_link	p_nom	sum
dc_link	length	average

Node coordinates and country are preserved via average / first.

va_profile = AggregationProfile(
    topology_strategy="simple",
    node_properties={
        "lat": "average",
        "lon": "average",
        "voltage": "first",
        "country": "first",
    },
    edge_type_properties={
        "line": {
            "x": "equivalent_reactance",
            "r": "equivalent_reactance",
            "s_nom": "sum",
            "b": "sum",
            "circuits": "sum",
            "length": "average",
        },
        "trafo": {
            "x": "equivalent_reactance",
            "s_nom": "sum",
        },
        "dc_link": {
            "p_nom": "sum",
            "length": "average",
        },
    },
    default_node_strategy="average",
    default_edge_strategy="sum",
    warn_on_defaults=False,
)

nodes_before = va_manager.get_current_graph().number_of_nodes()
edges_before = va_manager.get_current_graph().number_of_edges()

va_agg_graph = va_manager.aggregate(profile=va_profile)

print(
    f"Nodes: {nodes_before:,} -> {va_agg_graph.number_of_nodes():,} "
    f"({va_agg_graph.number_of_nodes() / nodes_before:.1%} of original)"
)
print(
    f"Edges: {edges_before:,} -> {va_agg_graph.number_of_edges():,} "
    f"({va_agg_graph.number_of_edges() / edges_before:.1%} of original)"
)

Nodes: 6,792 -> 50 (0.7% of original)
Edges: 8,974 -> 166 (1.8% of original)

/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'tags'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'geometry'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'symbol'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'under_construction'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'dc'. Falling back to first available value.
  log_warning(

va_manager.plot_network(
    style="voltage_aware", title=f"Voltage-Aware — Aggregated ({N_CLUSTERS} nodes)"
)