Metadata installation¶
The Metadata component is the web application that staff use to browse, search, annotate, and manage metadata. It is the primary user interface of a CVDLink installation.
Responsible team: Simavi.
What the component provides¶
- Dataset browsing, search, and metadata editing
- Dataset upload and file annotation
- Extraction pipelines for each supported data format, including transformation to the OMOP Common Data Model
- Metadata APIs for imaging, clinical, signal, and genomic records
- Federation status views showing the state of peer sites
- Administration pages for users, monitoring, topology, and audit review
- Assistant features, where enabled and supported by the database
Architecture¶
The application runs under Gunicorn inside a container, listening on container port 8000, published on host port 48000. Static files are served by the application itself. Where the application is published on the internet, a reverse proxy terminates TLS and forwards requests to port 48000.
A second container runs the federation listener, which maintains the connection to the message broker and processes node status events. Both containers use the same image and the same configuration file, and are started together.
Before you begin¶
- The Database component installed and verified, or the shared database confirmed reachable
- The ETL component installed, if extraction will be triggered from the application
- Port 48000 free
- A data directory on the host for uploaded and processed files
- For central deployments, Keycloak client configuration for the realm
1. Prepare the data directory¶
The application reads and writes files in a directory on the host, mounted into the container. Create it before the first start:
sudo mkdir -p /home/simavi/data
sudo chown "$USER":"$USER" /home/simavi/data
If a different location is used, update the volume mapping in docker-compose.yml and
set DATA_DIR to the path inside the container.
2. Generate a secret key¶
The application requires a secret key, which must be unique to each installation and must not be shared between environments.
python3 -c "import secrets; print(secrets.token_urlsafe(64))"
Record the output for the next step.
3. Prepare the configuration¶
Create a file named .env.docker in the installation directory.
For a central deployment, substituting the addresses from Hosts and ports:
ENV=production
DEBUG=false
OFFLINE_MODE=false
TIME_ZONE=Europe/Bucharest
SECRET_KEY=<generated secret key>
DJANGO_SETTINGS_MODULE=config.settings
ALLOWED_HOSTS=<application host address>,<public hostname>
CSRF_TRUSTED_ORIGINS=https://<public hostname>
SESSION_COOKIE_SECURE=true
CSRF_COOKIE_SECURE=true
DB_HOST=<data host address>
DB_PORT=5432
DB_NAME=cvdlink
DB_USER=postgres
DB_PASSWORD=<database password>
OIDC_OP_DISCOVERY_ENDPOINT=https://cvdlink.simavi.ro/auth/realms/cvdlink/.well-known/openid-configuration
OIDC_OP_LOGOUT_ENDPOINT=https://cvdlink.simavi.ro/auth/realms/cvdlink/protocol/openid-connect/logout
OIDC_OP_USER_ENDPOINT=https://cvdlink.simavi.ro/auth/realms/cvdlink/protocol/openid-connect/userinfo
OIDC_RP_CLIENT_ID=metadata_federation
OIDC_RP_CLIENT_SECRET=
OIDC_RP_SCOPES=openid profile email roles
OIDC_USE_PKCE=true
OIDC_VERIFY_SSL=true
MESH_ENABLED=true
MESH_MQTT_ENABLED=true
MESH_MQTT_BROKER_URL=<data host address>
MESH_MQTT_BROKER_PORT=1883
MESH_NODE_SEED=<unique node identifier>
MESH_NODE_NAME=<readable node name>
MESH_NODE_ROLE=LEAF
MESH_NODE_PORT=48000
DATA_DIR=/home/simavi/data
For a standalone deployment:
ENV=offline
DEBUG=false
OFFLINE_MODE=true
TIME_ZONE=Europe/Bucharest
SECRET_KEY=<generated secret key>
DJANGO_SETTINGS_MODULE=config.settings
ALLOWED_HOSTS=*
CSRF_TRUSTED_ORIGINS=http://localhost:48000
SESSION_COOKIE_SECURE=false
CSRF_COOKIE_SECURE=false
DB_HOST=host.docker.internal
DB_PORT=5444
DB_NAME=cvdlink
DB_USER=postgres
DB_PASSWORD=<database password>
MESH_ENABLED=false
MESH_MQTT_ENABLED=false
DATA_DIR=/home/simavi/data
Restrict access to the file, which contains the database password and the secret key:
chmod 600 .env.docker
The file must not be committed to version control.
Node identity¶
MESH_NODE_SEED identifies the installation within the federation and must be unique
across all participating sites. Set it explicitly rather than relying on the default,
which is derived from the machine hostname and will change if the machine is renamed or
replaced.
Key configuration values¶
| Variable | Purpose |
|---|---|
SECRET_KEY |
Cryptographic signing key, unique per installation |
DEBUG |
Must be false outside development |
OFFLINE_MODE |
Selects local authentication instead of Keycloak |
ALLOWED_HOSTS |
Host names the application will answer for |
CSRF_TRUSTED_ORIGINS |
Origins permitted to submit forms |
DB_HOST, DB_PORT |
Database location, see the profile sections above |
DATA_DIR |
Data directory path inside the container |
MESH_NODE_SEED |
Unique identifier for this site |
MESH_MQTT_BROKER_URL |
Message broker address, central deployments only |
GUNICORN_WORKERS |
Worker process count, default 3 |
4. Build the image¶
docker compose build
For installations without build capability, load a supplied archive instead:
docker load -i cvdlink-django.tar
5. Start the application¶
docker compose up -d
Two containers start: the web application and the federation listener. Static files are collected automatically at start.
6. Apply database migrations¶
After the first start, apply the migrations:
docker compose exec web python manage.py migrate
Where the installation uses a restored database whose schema must not be altered automatically, review the pending migrations before applying them:
docker compose exec web python manage.py showmigrations
7. Create the first account¶
In a standalone deployment, create an administrative account:
docker compose exec web python manage.py createsuperuser
Subsequent users may register themselves at /auth/register/.
In a central deployment, accounts are managed in Keycloak and this step does not apply.
8. Verify the installation¶
Confirm both containers are running¶
docker compose ps
Both the web and federation containers must show a status beginning Up.
Confirm the application responds¶
curl -I http://localhost:48000/
A redirect to the sign-in route is the expected result for an unauthenticated request.
Confirm the deployment configuration¶
docker compose exec web python manage.py check --deploy
Review any warnings reported. Warnings about DEBUG or cookie security indicate
configuration that must be corrected before the installation is used in production.
Confirm sign-in¶
Open the application in a browser:
http://<application host>:48000/
In a central deployment, the browser is redirected to Keycloak and returns to the
application after successful authentication. In a standalone deployment, the sign-in
form is presented at /auth/login/.
Confirm the database connection¶
docker compose logs web | grep -iE "error|could not connect|refused"
No output is expected.
Reverse proxy¶
Where the application is reached over HTTPS, a reverse proxy terminates TLS and forwards requests to port 48000. The proxy must forward the original protocol and host, as the application relies on these to construct correct URLs and to enforce secure cookies.
The following headers are required:
X-Forwarded-Proto the original protocol, https
X-Forwarded-Host the original host name
The public host name must appear in ALLOWED_HOSTS and the public origin in
CSRF_TRUSTED_ORIGINS, otherwise requests are rejected.
Routine operations¶
| Operation | Command |
|---|---|
| Restart | docker compose restart |
| Stop | docker compose down |
| View application logs | docker compose logs -f web |
| View federation logs | docker compose logs -f mesh |
| Apply configuration changes | docker compose up -d |
| Rebuild after a code change | docker compose up -d --build |
| Rebuild without cache | docker compose build --no-cache |
Changes to .env.docker take effect when the containers are recreated with
docker compose up -d. A restart alone does not reload the file.
Troubleshooting¶
The application reports a disallowed host¶
The host name used in the browser is not listed in ALLOWED_HOSTS. Add it, then
recreate the containers.
Form submissions are rejected¶
The origin is not listed in CSRF_TRUSTED_ORIGINS. Add the full origin, including the
scheme, then recreate the containers. Where a reverse proxy is in use, confirm it
forwards the X-Forwarded-Proto header.
Static files are missing¶
Static files are collected at container start. Collect them manually if required:
docker compose exec web python manage.py collectstatic --noinput
Compose reports the configuration file is missing¶
docker compose down requires the environment file referenced by the compose file to
be present. If it has been moved, restore it before stopping the containers, or stop
them directly:
docker stop cvdlink-django-prod cvdlink-django-prod-mesh
docker rm cvdlink-django-prod cvdlink-django-prod-mesh
Sign-in through Keycloak fails¶
Confirm that the realm is reachable from the container, that the client identifier matches a client configured in the realm, and that the client permits the redirect URI of this installation. Confirm the system clock is correct, as token validation is time sensitive.
Federation status shows no peer sites¶
Confirm that MESH_MQTT_ENABLED is true, that the broker address is correct and
reachable, and that MESH_NODE_SEED differs from that of every other site. Inspect the
federation container logs.
Uploaded files are not visible¶
Confirm the data directory is mounted and writable:
docker compose exec web ls -la /home/simavi/data
If the directory is owned by a different user and not writable, correct its ownership on the host.
Next steps¶
Install the remaining components required for the deployment: Models, UI, and P2P.