Getting Started with NPAP#

NPAP (Network Partitioning and Aggregation Package) is a standalone, pip-installable open-source Python library for reducing the spatial complexity of network graphs. Built on top of NetworkX, it is decoupled from any specific modelling framework and can therefore be readily integrated into existing tools or used in isolation.

A key design decision in NPAP is the explicit separation of the network reduction process into two distinct steps:

  1. Partitioning — maps each vertex (node) in the original network to a group (cluster). This step only determines group membership; it does not modify the graph itself.

  2. Aggregation — reduces the network topology based on a given partitioning result by aggregating nodes, edges, and their associated properties.

This separation gives users fine-grained control: partitioning algorithms can be swapped independently of the aggregation method, different aggregation strategies can be applied to the same partitioning result, or either step can be used in isolation. NPAP follows a strategy pattern architecture so that users can register custom data loaders, partitioning algorithms, and aggregation rules without modifying the core code. The library currently provides 13 partitioning strategies organised in four families and two pre-defined aggregation profiles.

Although NPAP was initially developed with a focus on power systems, its architecture is general-purpose and applicable to any network graph. The target audience therefore includes power systems researchers, energy modelers, network analysts, and more broadly, anyone interested in reducing the spatial complexity of a graph.

This notebook is a 10-minute tour that shows you how to:

  1. Install the library and initialise the PartitionAggregatorManager facade (main manager).

  2. Load data into NPAP.

  3. Pre-process the graph (e.g., aggregate parallel edges).

  4. Partition the network into clusters.

  5. Aggregate each cluster into a representative node via an AggregationProfile.

  6. Visualise the network throughout the pipeline.

Section 2 runs the full pipeline on a general-purpose focus. Section 3 closes with a short power-systems example illustrating NPAP’s voltage-aware and AC-island-aware partitioning on real European high-voltage grid data.

📚 Full documentation: https://npap.readthedocs.io

Pipeline Architecture#

The full NPAP pipeline is shown below. The whole workflow is orchestrated and accessed through a single facade (main manager) — the PartitionAggregatorManager — which coordinates three specialised manager classes (InputDataManager, PartitioningManager, and AggregationManager). Each stage of the pipeline is backed by a pluggable strategy and can be illustrated at any point through the visualization component.

NPAP pipeline architecture

NPAP first prepares the network graph in two stages: data is loaded and a NetworkX graph is created and validated, then optional pre-processing steps (e.g., parallel-edge aggregation, voltage-level grouping) prepare the graph for the reduction process. The reduction itself is then split into the partitioning and aggregation steps introduced above.

See the Available Strategies guide for the complete catalogue of loaders, partitioners and aggregators.

1. Installation#

NPAP is released under the MIT license and published on PyPI as a standalone, pip-installable package. Uncomment the cell below if you do not have it yet.

See the installation guide for details.

# %pip install npap

2. General-Purpose Workflow — A Real Geo-Referenced Network#

To highlight that NPAP’s core workflow is domain-agnostic, this section runs the full pipeline on the European high-voltage grid (~6,800 buses, 35 countries) derived from OpenStreetMap data and hosted on Zenodo by Xiong et al. (2025). At this stage we deliberately ignore the electrical semantics of the dataset and treat it as a generic geo-referenced graph: vertices have coordinates, edges have a length and a capacity. Section 3 then re-loads the same network with NPAP’s power-systems features enabled, so you can compare the two treatments side-by-side.

2.1 Initialise the Facade Manager#

The whole NPAP workflow is orchestrated and accessed through the PartitionAggregatorManager facade, which internally coordinates the data-loading, partitioning, and aggregation managers.

import networkx as nx
import pandas as pd
from npap.datasets import fetch_pypsa_eur

import npap
from npap import AggregationProfile

manager = npap.PartitionAggregatorManager()

data_files = fetch_pypsa_eur()

Dataset cached in /Users/taaarmar/.cache/npap (2.7 MB).
These files are kept for future runs and are never removed automatically. To delete them, run:
    from npap.datasets import clear_data_home; clear_data_home()

2.2 Data Loading#

NPAP graphs can be supplied in three ways: directly as a NetworkX graph (networkx_direct), through a generic node/edge CSV pair (csv_files), or through the power-systems–oriented va_loader (buses, lines, transformers, converters, DC links). Custom data-loading strategies can be registered seamlessly thanks to the strategy pattern.

For a generic treatment of the network we use csv_files: it only needs a node file and an edge file, and treats every edge uniformly with no type differentiation. This is the right choice whenever the full physical topology is not required.

See the data-loading guide for the full list of strategies.

graph = manager.load_data(
    strategy="csv_files",
    node_file=str(data_files["buses.csv"]),
    edge_file=str(data_files["lines.csv"]),
)

print(f"Graph type : {type(graph).__name__}")
print(f"Vertices   : {graph.number_of_nodes():,}")
print(f"Edges      : {graph.number_of_edges():,}")

Graph type : MultiDiGraph
Vertices   : 6,863
Edges      : 9,162

/Users/taaarmar/git/NPAP/npap/input/csv_loader.py:197: UserWarning: Parallel edges detected in CSV edge file. A MultiDiGraph will be created. Call manager.aggregate_parallel_edges() to collapse parallel edges before partitioning.
  log_warning(

manager.plot_network(style="simple", title="European HV Grid — Raw")

2.3 Pre-processing: Aggregate Parallel Edges#

Partitioning strategies in NPAP operate on simple DiGraphs. The PyPSA-Eur dataset contains many double-circuit lines — two edges sharing the same pair of vertices — so the csv_files loader returns a MultiDiGraph and emits a warning. We collapse the parallel edges through the facade manager and explicitly declare how each edge property should be combined.

if isinstance(graph, nx.MultiDiGraph):
    edges_before = graph.number_of_edges()
    graph = manager.aggregate_parallel_edges(
        edge_properties={
            "length": "average",  # mean edge length
            "s_nom": "sum",  # total capacity of parallel edges
        },
        default_strategy="average",
        warn_on_defaults=False,
    )
    print(f"Edges before: {edges_before:,}")
    print(f"Edges after : {graph.number_of_edges():,}")

Edges before: 9,162
Edges after : 8,065

2.4 Partitioning#

The partitioning step assigns each vertex of the network to a cluster — without yet modifying the graph itself. NPAP currently provides four families of partitioning strategies combining geographical and electrical node distance with the option of treating voltage levels independently (voltage-awareness). Each family supports up to 13 algorithms, including k-means, k-medoids, DBSCAN, HDBSCAN, and hierarchical clustering.

For a generic geo-referenced graph like ours, geographical distance (latitudes and longitudes) is the natural choice. We use k-medoids with the Haversine metric to obtain spatially compact clusters. The outcome is returned as a PartitionResult that stores the mapping together with metadata such as the number of clusters and the strategy name.

See the partitioning guide for details on each strategy.

N_CLUSTERS = 200

partition = manager.partition(
    strategy="geographical_kmedoids_haversine",
    n_clusters=N_CLUSTERS,
)

sizes = pd.Series({k: len(v) for k, v in partition.mapping.items()})
print(f"Clusters created : {partition.n_clusters}")
print(f"Cluster size — min: {sizes.min()}, max: {sizes.max()}, mean: {sizes.mean():.1f}")

manager.plot_network(style="clustered", title=f"Partitioned ({N_CLUSTERS} clusters)")

Clusters created : 200
Cluster size — min: 5, max: 190, mean: 34.3

2.5 Aggregation with an AggregationProfile#

The aggregation step reduces the network topology based on the partitioning result. NPAP decomposes this process into a three-tier pipeline:

  1. Topology tier — builds a new graph by creating one representative vertex per cluster and adjusting the edges between them accordingly.

  2. Domain-specific tier (optional) — modifies physical properties of the resulting graph (e.g., adding edges that did not exist in the original graph or overriding specific quantities such as line reactances).

  3. Property aggregation tier — applies user-specified functions (sum, average, equivalent_reactance, …) to the remaining node and edge properties.

These tiers are configured through an AggregationProfile. Below we average vertex coordinates, take the first voltage per cluster, sum the edge capacities, and average the edge lengths.

profile = AggregationProfile(
    topology_strategy="simple",
    node_properties={
        "lat": "average",
        "lon": "average",
        "voltage": "first",
    },
    edge_properties={
        "s_nom": "sum",
        "length": "average",
    },
    default_node_strategy="average",
    default_edge_strategy="sum",
    warn_on_defaults=False,
)

nodes_before = manager.get_current_graph().number_of_nodes()
edges_before = manager.get_current_graph().number_of_edges()

agg_graph = manager.aggregate(profile=profile)

print(
    f"Nodes: {nodes_before:,} -> {agg_graph.number_of_nodes():,} "
    f"({agg_graph.number_of_nodes() / nodes_before:.1%} of original)"
)
print(
    f"Edges: {edges_before:,} -> {agg_graph.number_of_edges():,} "
    f"({agg_graph.number_of_edges() / edges_before:.1%} of original)"
)

Nodes: 6,863 -> 200 (2.9% of original)
Edges: 8,065 -> 567 (7.0% of original)

/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'country'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'symbol'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'dc'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'tags'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'under_construction'. Falling back to first available value.
  log_warning(
/Users/taaarmar/git/NPAP/npap/aggregation/basic_strategies.py:327: UserWarning: No numeric values found for node property 'geometry'. Falling back to first available value.
  log_warning(

2.6 Visualise the Aggregated Network#

The network can be illustrated throughout the pipeline using NPAP’s built-in visualization component, powered by Plotly. Available styles are simple, clustered, and voltage_aware. See the visualization guide for more.

manager.plot_network(style="simple", title=f"Aggregated ({N_CLUSTERS} nodes)")

3. Power-Systems Workflow — Voltage-Aware Partitioning on the Same Network#

We now re-load the same European HV grid with NPAP’s power-systems features enabled, so the contrast with Section 2 is immediate. The va_loader ingests all five PyPSA-Eur files (buses, lines, transformers, converters, DC links) and builds a physically detailed graph in which every edge is typed (line / trafo / dc_link) and every bus is assigned to an AC island — the set of buses reachable without crossing a DC link.

NPAP automatically detects AC islands linked solely by DC interconnections and partitions them independently. AC-island and voltage-level constraints are enforced by setting the distance-matrix entries between nodes in different islands or voltage levels to infinity, which makes both mechanisms algorithm-agnostic: they work with any distance-based partitioning method without modifying the algorithm itself.

Section 2 (simple)

Section 3 (voltage-aware)

Data loader

csv_files

va_loader

Edge types

all uniform

line / trafo / DC link

AC islands

✗

✓

Voltage-level awareness

✗

✓

Partitioning strategy

geographical_kmedoids_haversine

va_geographical_kmedoids_haversine

Per-edge-type aggregation

✗

✓

 
va_manager = npap.PartitionAggregatorManager()

va_graph = va_manager.load_data(
    strategy="va_loader",
    node_file=str(data_files["buses.csv"]),
    line_file=str(data_files["lines.csv"]),
    transformer_file=str(data_files["transformers.csv"]),
    converter_file=str(data_files["converters.csv"]),
    link_file=str(data_files["links.csv"]),
)

print(f"Nodes: {va_graph.number_of_nodes():,}  |  Edges: {va_graph.number_of_edges():,}")

/Users/taaarmar/git/NPAP/npap/input/va_loader.py:900: UserWarning: 6 DC link(s) skipped due to missing references
  log_warning(
/Users/taaarmar/git/NPAP/npap/input/va_loader.py:213: UserWarning: Parallel edges detected in voltage-aware loader. A MultiDiGraph will be created. Call manager.aggregate_parallel_edges() to collapse parallel edges before partitioning.
  log_warning(
/Users/taaarmar/git/NPAP/npap/input/va_loader.py:373: UserWarning: Found 71 isolated node(s) with no connections. These will be removed.
  log_warning(

Nodes: 6,792  |  Edges: 10,073



# Collapse parallel double-circuit lines and harmonise voltage levels.
if isinstance(va_graph, nx.MultiDiGraph):
    va_graph = va_manager.aggregate_parallel_edges(
        edge_properties={
            "x": "average",
            "r": "average",
            "s_nom": "sum",
            "length": "average",
        },
        default_strategy="first",
        warn_on_defaults=False,
    )

va_manager.group_by_voltage_levels([220, 380])
va_manager.plot_network(style="voltage_aware", title="European HV Grid — VA Load")

va_partition = va_manager.partition(
    strategy="va_geographical_kmedoids_haversine",
    n_clusters=N_CLUSTERS,
)

va_manager.plot_network(style="clustered", title=f"Voltage-Aware — {N_CLUSTERS} clusters")