P2P installation¶
The P2P component is the federation service. It announces the datasets held by this site to other sites, receives their announcements in turn, and maintains the shared catalogue that makes federated search possible.
Responsible team: Simavi.
When this component is required¶
Install the P2P component where the site participates in the federation, which is the usual case for a central deployment.
A standalone installation operates fully without it: datasets are catalogued and searched locally. Install it only when the site is to exchange metadata with other sites.
How federation works¶
Each site runs one P2P node. When a dataset is announced, the node publishes a description of it to the other nodes and records which of them have acknowledged receipt. Peers fetch the description, store it in their own catalogue, and acknowledge. A search at any site therefore returns records describing datasets held elsewhere, while the data itself remains at the originating site.
Nodes reach one another in two ways. A gossip protocol over TCP carries announcements and acknowledgements between known neighbours, and a distributed hash table over UDP allows a node to locate the site holding a given dataset. Where the gossip path is unavailable, announcements fall back to direct HTTP requests.
Before you begin¶
- The Database component installed and verified, or the shared database confirmed reachable
- Ports 8060, 4447, and 8468 free
- The addresses of the peer sites this node will connect to
- Agreement with the other sites on the node identifier for this site
Configuration¶
This component is configured by a Python settings file mounted into the container, not by environment variables. The file is supplied per node and mounted read-only.
Create a settings file for this site based on the following. Substitute the database address from Hosts and ports.
# Database
DB_HOST = "<data host address>"
DB_PORT = 5432
DB_NAME = "cvdlink"
DB_USER = "postgres"
DB_PASSWORD = "<database password>"
# Initialise the federation schema at startup: 1 for the first node only, otherwise 0
INIT_DB = 0
# Node identity, unique across the federation
NODE_ID = "<site identifier>"
# Network
P2P_HOST = "0.0.0.0"
P2P_PORT = 4444
DHT_PORT = 8468
API_HOST = "0.0.0.0"
API_PORT = 8060
# Peer discovery
SEEDS = [("<peer host>", 8468)]
NEIGHBORS = [("<peer host>", 4444)]
# Sites expected to acknowledge this node's announcements
EXPECTED_PEERS = ["<peer identifier>"]
# Direct HTTP announcement where gossip is unavailable
HTTP_FANOUT_ENABLED = True
The settings files supplied with the component contain example addresses. Replace these with the values for your own environment before starting the node.
Configuration reference¶
| Setting | Purpose |
|---|---|
DB_HOST, DB_PORT |
Database location |
INIT_DB |
Create the federation schema at startup; set to 1 on one node only |
NODE_ID |
Identifier for this site, unique across the federation |
P2P_PORT |
Gossip listener, container port 4444 |
DHT_PORT |
Discovery listener, 8468 |
API_PORT |
HTTP API, 8060 |
SEEDS |
Nodes used to join the discovery overlay |
NEIGHBORS |
Nodes contacted directly for gossip |
EXPECTED_PEERS |
Sites whose acknowledgement is awaited before an announcement is considered synchronised |
HTTP_FANOUT_ENABLED |
Announce over HTTP where gossip fails |
Peer host names must be resolvable and reachable from this container. Where sites are on separate networks, use addresses rather than names, and confirm that the gossip and discovery ports are open in both directions.
Set INIT_DB = 1 on exactly one node when establishing a new federation, so that the
schema is created once. Leave it at 0 everywhere else, including on that node after the
first successful start.
1. Place the settings file¶
The compose file mounts the settings file into the container. Confirm the mapping matches the file name used:
volumes:
- ./conf/<site>.settings.py:/app/src/settings.py:ro
2. Build the image¶
docker compose build
For installations without build capability, load a supplied archive instead:
docker load -i cvdlink-p2p.tar
3. Start the node¶
docker compose up -d
4. Verify the installation¶
Confirm the container is running¶
docker compose ps
The container must show a status beginning Up.
Confirm the node responds¶
curl http://localhost:8060/health
Confirm the node has located its peers¶
curl http://localhost:8060/peers
The response lists the peers currently known to this node. An empty list immediately after start is normal; peers appear once discovery completes. If the list remains empty, see Troubleshooting below.
Confirm the catalogue is readable¶
curl http://localhost:8060/catalog/all
A new node returns an empty catalogue until the first announcement is received.
Confirm the database connection¶
docker compose logs | grep -iE "error|refused|could not connect"
No output is expected.
API¶
The node exposes the following operations. Those prefixed _debug are diagnostic and
should not be used in normal operation.
| Method | Path | Purpose |
|---|---|---|
| GET | /health |
Service health check |
| GET | /peers |
Peers currently known |
| GET | /peers/db |
Peers recorded in the database |
| POST | /services |
Register a service |
| POST | /catalog/sd |
Store a dataset description |
| GET | /catalog/sd/{id} |
Retrieve a dataset description |
| POST | /catalog/announce |
Announce a dataset to peers |
| POST | /catalog/create_and_announce |
Register and announce in one operation |
| GET | /catalog/search |
Search the federated catalogue |
| GET | /catalog/all |
Retrieve the full catalogue |
| GET | /catalog/sync_status |
Acknowledgement state of an announcement |
| GET | /catalog/dht_lookup |
Locate the site holding a dataset |
| POST | /sync/pull |
Request records from a peer |
Running several nodes on one machine¶
For testing, several nodes may run on one machine. Each requires its own settings file, its own container, and distinct published ports, since only the container ports are fixed.
The project's test environment runs three nodes with the following assignments.
| Node | API | Gossip | Discovery |
|---|---|---|---|
| First | 8060 | 4447 | 8468 |
| Second | 8061 | 4445 | 8469 |
| Third | 8062 | 4446 | 8470 |
Within the Docker network the nodes address one another by container name on the
container ports, so NEIGHBORS uses port 4444 regardless of the published port.
Routine operations¶
| Operation | Command |
|---|---|
| Restart | docker compose restart |
| Stop | docker compose down |
| View logs | docker compose logs -f |
| Apply a settings change | docker compose restart |
The settings file is read at startup, so a restart is required after any change.
Troubleshooting¶
The peer list remains empty¶
Confirm that the addresses in SEEDS and NEIGHBORS are reachable from inside the
container:
docker compose exec <container> getent hosts <peer host>
Confirm that the gossip port is open between the sites in both directions. Discovery over UDP and gossip over TCP are separate paths, and both must be permitted.
Announcements are not acknowledged¶
Inspect the synchronisation state:
curl "http://localhost:8060/catalog/sync_status?service_id=<identifier>"
The response lists the peers that have acknowledged. Where a peer is missing, confirm
that its identifier appears in EXPECTED_PEERS exactly as that site configured its own
NODE_ID, and that the peer node is running.
The federation schema is missing¶
The schema is created only when a node starts with INIT_DB = 1. If no node was
started with that setting, set it on one node, restart it, then return the setting to 0.
Two sites report the same identifier¶
NODE_ID must be unique across the federation. Where two sites share an identifier,
acknowledgements are attributed incorrectly and synchronisation state becomes
unreliable. Agree identifiers with the other sites before installation.
Next steps¶
Return to the documentation home for the remaining components, or continue to Starting and stopping services.