Architecture overview¶
This page describes how CVDLink components interact. It is intended to be read before carrying out an installation, so that the purpose of each configuration value is clear.
Design principles¶
CVDLink is designed around three constraints that shape the whole architecture.
Data remains at the originating institution. Clinical data is not centralised. Each site holds its own data in its own database and exposes only metadata to other sites. This allows participation without transferring patient data across institutional or national boundaries.
Sites operate independently. An installation remains functional when the network is unavailable or when other sites are offline. Federation adds capability but is not required for local operation.
Components are installed separately. Each component is packaged as a Docker image with its own configuration, so a site can install only the components it needs and upgrade them independently.
Component responsibilities¶
Database¶
A PostgreSQL instance holding the metadata schemas, dataset records, reference vocabularies, and the embeddings used by the assistant features. All other components read from and write to this database; none of them hold persistent state of their own.
The database requires the vector extension. The assistant features answer questions
using a language model, and to find the material relevant to a question they compare
embeddings, which are numerical representations of meaning. The extension provides the
column type that stores them and the index that searches them by similarity. This is
why the database is built from an image that includes the extension, as described in
Database installation.
ETL¶
A service that reads clinical and imaging files from a configured directory, extracts structured metadata, and writes it to the database. It handles DICOM imaging, FHIR resources, HL7 messages, genomic variant files, HDF5 and EDF signal recordings, waveform records, and tabular data. It also transforms data into the OMOP Common Data Model where a mapping is defined.
The ETL service is reached over an HTTP API and may additionally be run as a command-line tool for bulk loading.
Metadata¶
The web application that staff interact with. It provides dataset browsing and search, metadata viewing and editing, annotation, dataset upload, a federation status view, and administrative functions including user management and audit review. It also hosts the assistant features where these are enabled.
This component holds no data itself; it presents what the database contains.
Models¶
Analytical and predictive model services, maintained by a separate team. See Models installation.
UI¶
A clinical user interface maintained by a partner team. See UI installation.
P2P¶
The federation service. Each site runs a P2P node that announces its presence to other nodes, discovers peers, and exchanges metadata records so that a search carried out at one site can return results describing datasets held at another. Nodes communicate over a gossip protocol and locate one another through a distributed hash table.
How a request flows through the system¶
The following sequence describes the most common operation, a user searching for datasets.
- The user opens the Metadata application in a browser and submits a search.
- The Metadata application queries the local database for matching records.
- Where federated results are requested, the Metadata application asks the local P2P node for records previously received from peer sites.
- Results are returned to the browser, each identified by the site that holds the underlying data.
- Where the user requests data held at another site, the request is directed to that site rather than served locally.
Data arrives in the system through a separate path. Files are placed in the configured data directory, the ETL service extracts their metadata, and the records it writes become visible to the Metadata application and, once announced, to peer sites.
Communication between components¶
| From | To | Protocol | Purpose |
|---|---|---|---|
| Metadata | Database | PostgreSQL | Read and write metadata records |
| ETL | Database | PostgreSQL | Write extracted metadata |
| Metadata | ETL | HTTP | Trigger extraction jobs |
| Metadata | P2P node | HTTP | Retrieve federated records |
| P2P node | Peer P2P nodes | Gossip over TCP, discovery over UDP | Exchange metadata between sites |
| Metadata | Message broker | MQTT | Publish and receive node status events |
| Metadata | Keycloak | OpenID Connect | Authenticate users, central deployments only |
Port numbers for each of these are listed in Hosts and ports.
Authentication¶
In central deployments, authentication is delegated to Keycloak using OpenID Connect with the authorization code flow and PKCE. User roles are read from the token issued by Keycloak and determine access to administrative and data management functions.
In standalone deployments, Keycloak is not installed. The Metadata application uses local accounts instead, with self-registration, and role restrictions are not applied. This difference is significant for access control and is described in Deployment profiles.
Pending content¶
A detailed architecture description, including the system diagram, is held in a separate document and has not yet been incorporated into this page. The sections above describe the component relationships as verified against the running deployment.