Skip to content
Snippets Groups Projects
Commit eeab7190 authored by Christos Diamantakis's avatar Christos Diamantakis
Browse files

Edit README.md

parent 692b2c00
No related branches found
No related tags found
No related merge requests found
# ENACT Dataspace API - Setup & Deployment Guide # ENACT Dataspace API
A multi-tenant EDC (Eclipse Dataspace Connector) API for managing dataspace connectors, assets, and data exchange workflows with Keycloak authentication. A multi-tenant EDC (Eclipse Dataspace Connector) API for managing dataspace connectors, assets, and data exchange workflows with Keycloak authentication.
## 📋 Prerequisites ## 📋 Prerequisites
- Docker & Docker Compose - **Docker & Docker Compose** - Container orchestration
- Linux/Unix environment - **Linux/Unix environment** - Development and deployment
- Domain name (for production) - **Domain name** - For production deployment
- SSL certificates (for production) - **SSL certificates** - HTTPS configuration
- Keycloak instance (for authentication) - **Keycloak instance** - Authentication provider
## 🏗️ Project Structure ## 🏗️ Project Structure
```
enact-dataspace/ enact-dataspace/
├── app/ # Node.js application ├── app/ # Node.js application
│ ├── app.js # Main application file │ ├── app.js # Main application file
...@@ -28,6 +31,7 @@ enact-dataspace/ ...@@ -28,6 +31,7 @@ enact-dataspace/
├── docker-compose.production.yml # Production container setup ├── docker-compose.production.yml # Production container setup
├── .env # Production environment variables ├── .env # Production environment variables
└── logs/ # Application logs └── logs/ # Application logs
```
## 🚀 Quick Start ## 🚀 Quick Start
...@@ -41,12 +45,21 @@ cd enact-dataspace ...@@ -41,12 +45,21 @@ cd enact-dataspace
# Create required directories # Create required directories
mkdir -p logs ssl mkdir -p logs ssl
chmod 755 logs ssl chmod 755 logs ssl
2. Environment Configuration ```
Create your .env file in the root directory:
bashcp .env.production .env ### 2. Environment Configuration
Create your `.env` file in the root directory:
```bash
cp .env.production .env
nano .env nano .env
Required Environment Variables: ```
bash# Security & Encryption
#### Required Environment Variables
```bash
# Security & Encryption
ENCRYPTION_KEY=<64-character-hex-key> # Generate with: openssl rand -hex 32 ENCRYPTION_KEY=<64-character-hex-key> # Generate with: openssl rand -hex 32
SESSION_SECRET=<base64-session-secret> SESSION_SECRET=<base64-session-secret>
EDC_WEBHOOK_SECRET=<webhook-secret> EDC_WEBHOOK_SECRET=<webhook-secret>
...@@ -71,17 +84,26 @@ DOMAIN=your-domain.com ...@@ -71,17 +84,26 @@ DOMAIN=your-domain.com
PROTOCOL=https PROTOCOL=https
DISABLE_AUTH=false DISABLE_AUTH=false
MULTI_TENANT_MODE=true MULTI_TENANT_MODE=true
3. SSL Certificates (Production) ```
Place your SSL certificates in the ssl/ directory:
bash# Copy your certificates ### 3. SSL Certificates (Production)
Place your SSL certificates in the `ssl/` directory:
```bash
# Copy your certificates
cp your-fullchain.pem ssl/fullchain.pem cp your-fullchain.pem ssl/fullchain.pem
cp your-private-key.pem ssl/privkey.pem cp your-private-key.pem ssl/privkey.pem
# Set correct permissions # Set correct permissions
chmod 644 ssl/fullchain.pem chmod 644 ssl/fullchain.pem
chmod 600 ssl/privkey.pem chmod 600 ssl/privkey.pem
4. Deploy with Docker Compose ```
bash# Create symlink for easier management
### 4. Deploy with Docker Compose
```bash
# Create symlink for easier management
ln -sf docker-compose.production.yml docker-compose.yml ln -sf docker-compose.production.yml docker-compose.yml
# Start all services # Start all services
...@@ -92,10 +114,16 @@ docker-compose ps ...@@ -92,10 +114,16 @@ docker-compose ps
# View logs # View logs
docker-compose logs -f enact-api docker-compose logs -f enact-api
🛠️ Development Setup ```
Local Development with Authentication Disabled
Create a development .env file: ## 🛠️ Development Setup
bash# Development configuration
### Local Development with Authentication Disabled
Create a development `.env` file:
```bash
# Development configuration
NODE_ENV=development NODE_ENV=development
PORT=3000 PORT=3000
DOMAIN=localhost DOMAIN=localhost
...@@ -108,8 +136,12 @@ MULTI_TENANT_MODE=true ...@@ -108,8 +136,12 @@ MULTI_TENANT_MODE=true
# Use localhost endpoints for development # Use localhost endpoints for development
MINIO_ENDPOINT=localhost MINIO_ENDPOINT=localhost
KEYCLOAK_URL=https://your-keycloak-instance.com KEYCLOAK_URL=https://your-keycloak-instance.com
Running Development Mode ```
bash# Install dependencies
### Running Development Mode
```bash
# Install dependencies
cd app cd app
npm install npm install
...@@ -121,41 +153,64 @@ npm start ...@@ -121,41 +153,64 @@ npm start
# Or run everything in containers # Or run everything in containers
docker-compose -f docker-compose.simple.yml up -d docker-compose -f docker-compose.simple.yml up -d
🔧 Configuration Details ```
Environment Variables Reference
VariableDescriptionRequiredDefaultENCRYPTION_KEYAES-256 encryption key (64 hex chars)✅-KEYCLOAK_URLKeycloak server URL✅-KEYCLOAK_CLIENT_SECRETOAuth client secret✅-DATABASE_URLPostgreSQL connection string✅-DISABLE_AUTHDisable authentication for development❌falseMULTI_TENANT_MODEEnable multi-tenant EDC support❌trueLOG_LEVELLogging level (debug/info/warn/error)❌info ## 🔧 Configuration Details
Port Configuration
ServiceInternal PortExternal PortDescriptionAPI30003000Main application APINginx80/44380/443Reverse proxy & SSL terminationPostgreSQL54325433Database (external for debugging)MinIO9000/9001-Object storage (internal only) ### Environment Variables Reference
🔐 Authentication Setup
Keycloak Configuration | Variable | Description | Required | Default |
|----------|-------------|----------|---------|
| `ENCRYPTION_KEY` | AES-256 encryption key (64 hex chars) | ✅ | - |
| `KEYCLOAK_URL` | Keycloak server URL | ✅ | - |
| `KEYCLOAK_CLIENT_SECRET` | OAuth client secret | ✅ | - |
| `DATABASE_URL` | PostgreSQL connection string | ✅ | - |
| `DISABLE_AUTH` | Disable authentication for development | ❌ | `false` |
| `MULTI_TENANT_MODE` | Enable multi-tenant EDC support | ❌ | `true` |
| `LOG_LEVEL` | Logging level (debug/info/warn/error) | ❌ | `info` |
### Port Configuration
| Service | Internal Port | External Port | Description |
|---------|---------------|---------------|-------------|
| API | 3000 | 3000 | Main application API |
| Nginx | 80/443 | 80/443 | Reverse proxy & SSL termination |
| PostgreSQL | 5432 | 5433 | Database (external for debugging) |
| MinIO | 9000/9001 | - | Object storage (internal only) |
Create Realm: Set up a realm in your Keycloak instance ## 🔐 Authentication Setup
Create Client: Configure OAuth2 client with:
Client ID: connector-1 (or your chosen ID) ### Keycloak Configuration
Access Type: confidential
Valid Redirect URIs: https://your-domain.com/*
Web Origins: https://your-domain.com
1. **Create Realm**: Set up a realm in your Keycloak instance
2. **Create Client**: Configure OAuth2 client with:
- **Client ID**: `connector-1` (or your chosen ID)
- **Access Type**: confidential
- **Valid Redirect URIs**: `https://your-domain.com/*`
- **Web Origins**: `https://your-domain.com`
3. **Client Credentials**: Note the client secret for your `.env` file
Client Credentials: Note the client secret for your .env file ### Token Verification
Token Verification
The application supports multiple authentication methods: The application supports multiple authentication methods:
Keycloak JWT tokens (production) - **Keycloak JWT tokens** (production)
API keys (service-to-service) - **API keys** (service-to-service)
Disabled auth (development only) - **Disabled auth** (development only)
🗄️ Database Setup ## 🗄️ Database Setup
Automatic Initialization
The database schema is automatically created on first run using scripts in init-scripts/:
01-init-db.sql - Basic tables and indexes ### Automatic Initialization
02-seed-data.sql - Initial data (if needed)
Manual Database Management The database schema is automatically created on first run using scripts in `init-scripts/`:
bash# Connect to database
- `01-init-db.sql` - Basic tables and indexes
- `02-seed-data.sql` - Initial data (if needed)
### Manual Database Management
```bash
# Connect to database
docker-compose exec postgres psql -U enact_user -d enact_dataspace docker-compose exec postgres psql -U enact_user -d enact_dataspace
# Backup database # Backup database
...@@ -163,27 +218,35 @@ docker-compose exec postgres pg_dump -U enact_user enact_dataspace > backup.sql ...@@ -163,27 +218,35 @@ docker-compose exec postgres pg_dump -U enact_user enact_dataspace > backup.sql
# Restore database # Restore database
docker-compose exec -T postgres psql -U enact_user -d enact_dataspace < backup.sql docker-compose exec -T postgres psql -U enact_user -d enact_dataspace < backup.sql
🌐 Nginx Configuration ```
SSL/TLS Setup
## 🌐 Nginx Configuration
### SSL/TLS Setup
The included nginx configuration provides: The included nginx configuration provides:
SSL termination with modern TLS settings - SSL termination with modern TLS settings
HTTP to HTTPS redirection - HTTP to HTTPS redirection
Security headers (HSTS, CSP, etc.) - Security headers (HSTS, CSP, etc.)
Rate limiting for API endpoints - Rate limiting for API endpoints
Reverse proxy to Node.js application - Reverse proxy to Node.js application
### Custom Nginx Configuration
Custom Nginx Configuration Edit `nginx/nginx.conf` to customize:
Edit nginx/nginx.conf to customize:
Rate limiting rules - Rate limiting rules
Security headers - Security headers
Proxy settings - Proxy settings
SSL cipher suites - SSL cipher suites
📊 Monitoring & Maintenance ## 📊 Monitoring & Maintenance
Health Checks
bash# Application health ### Health Checks
```bash
# Application health
curl https://your-domain.com/health curl https://your-domain.com/health
# Database health # Database health
...@@ -191,8 +254,12 @@ docker-compose exec postgres pg_isready -U enact_user ...@@ -191,8 +254,12 @@ docker-compose exec postgres pg_isready -U enact_user
# Container status # Container status
docker-compose ps docker-compose ps
Log Management ```
bash# View application logs
### Log Management
```bash
# View application logs
docker-compose logs enact-api docker-compose logs enact-api
# View nginx logs # View nginx logs
...@@ -203,8 +270,12 @@ docker-compose logs -f ...@@ -203,8 +270,12 @@ docker-compose logs -f
# Log rotation (recommended for production) # Log rotation (recommended for production)
# Configure logrotate for docker logs # Configure logrotate for docker logs
Backup Strategy ```
bash#!/bin/bash
### Backup Strategy
```bash
#!/bin/bash
# backup.sh - Regular backup script # backup.sh - Regular backup script
# Backup database # Backup database
...@@ -215,9 +286,14 @@ cp .env "env-backup-$(date +%Y%m%d)" ...@@ -215,9 +286,14 @@ cp .env "env-backup-$(date +%Y%m%d)"
# Backup SSL certificates # Backup SSL certificates
tar -czf "ssl-backup-$(date +%Y%m%d).tar.gz" ssl/ tar -czf "ssl-backup-$(date +%Y%m%d).tar.gz" ssl/
🚦 API Usage ```
Multi-Tenant EDC Management
bash# Register a new EDC connector ## 🚦 API Usage
### Multi-Tenant EDC Management
```bash
# Register a new EDC connector
curl -X POST https://your-domain.com/api/connectors/register \ curl -X POST https://your-domain.com/api/connectors/register \
-H "Authorization: Bearer $TOKEN" \ -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
...@@ -237,8 +313,12 @@ curl -H "Authorization: Bearer $TOKEN" \ ...@@ -237,8 +313,12 @@ curl -H "Authorization: Bearer $TOKEN" \
# Health check all connectors # Health check all connectors
curl -X POST -H "Authorization: Bearer $TOKEN" \ curl -X POST -H "Authorization: Bearer $TOKEN" \
https://your-domain.com/api/connectors/health https://your-domain.com/api/connectors/health
Data Exchange Workflow ```
bash# Complete workflow example
### Data Exchange Workflow
```bash
# Complete workflow example
curl -X POST https://your-domain.com/api/v1/data-exchange/complete-workflow \ curl -X POST https://your-domain.com/api/v1/data-exchange/complete-workflow \
-H "Authorization: Bearer $TOKEN" \ -H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
...@@ -250,10 +330,16 @@ curl -X POST https://your-domain.com/api/v1/data-exchange/complete-workflow \ ...@@ -250,10 +330,16 @@ curl -X POST https://your-domain.com/api/v1/data-exchange/complete-workflow \
"description": "Test data exchange", "description": "Test data exchange",
"dataUrl": "https://api.example.com/data" "dataUrl": "https://api.example.com/data"
}' }'
🐛 Troubleshooting ```
Common Issues
Authentication Not Working ## 🐛 Troubleshooting
bash# Check environment variables
### Common Issues
#### Authentication Not Working
```bash
# Check environment variables
docker-compose exec enact-api env | grep -E "(DISABLE_AUTH|KEYCLOAK)" docker-compose exec enact-api env | grep -E "(DISABLE_AUTH|KEYCLOAK)"
# Verify Keycloak connectivity # Verify Keycloak connectivity
...@@ -261,8 +347,12 @@ curl -s "https://your-keycloak.com/realms/your-realm/.well-known/openid_configur ...@@ -261,8 +347,12 @@ curl -s "https://your-keycloak.com/realms/your-realm/.well-known/openid_configur
# Check application logs # Check application logs
docker-compose logs enact-api | grep -i "auth\|error" docker-compose logs enact-api | grep -i "auth\|error"
Database Connection Issues ```
bash# Test database connectivity
#### Database Connection Issues
```bash
# Test database connectivity
docker-compose exec postgres psql -U enact_user -c "SELECT version();" docker-compose exec postgres psql -U enact_user -c "SELECT version();"
# Check database logs # Check database logs
...@@ -270,8 +360,12 @@ docker-compose logs postgres ...@@ -270,8 +360,12 @@ docker-compose logs postgres
# Verify environment variables # Verify environment variables
echo $DATABASE_URL echo $DATABASE_URL
Container Issues ```
bash# Restart specific service
#### Container Issues
```bash
# Restart specific service
docker-compose restart enact-api docker-compose restart enact-api
# Complete restart # Complete restart
...@@ -279,8 +373,12 @@ docker-compose down && docker-compose up -d ...@@ -279,8 +373,12 @@ docker-compose down && docker-compose up -d
# Clean restart (removes volumes) # Clean restart (removes volumes)
docker-compose down -v && docker-compose up -d docker-compose down -v && docker-compose up -d
SSL/HTTPS Issues ```
bash# Test SSL certificate
#### SSL/HTTPS Issues
```bash
# Test SSL certificate
openssl x509 -in ssl/fullchain.pem -text -noout openssl x509 -in ssl/fullchain.pem -text -noout
# Check nginx configuration # Check nginx configuration
...@@ -288,8 +386,12 @@ docker-compose exec nginx nginx -t ...@@ -288,8 +386,12 @@ docker-compose exec nginx nginx -t
# View nginx error logs # View nginx error logs
docker-compose logs nginx docker-compose logs nginx
Performance Tuning ```
bash# Monitor container resources
### Performance Tuning
```bash
# Monitor container resources
docker stats docker stats
# Database performance monitoring # Database performance monitoring
...@@ -300,9 +402,14 @@ docker-compose exec postgres psql -U enact_user -c " ...@@ -300,9 +402,14 @@ docker-compose exec postgres psql -U enact_user -c "
# Application metrics endpoint # Application metrics endpoint
curl https://your-domain.com/metrics curl https://your-domain.com/metrics
🔄 Updates & Upgrades ```
Application Updates
bash# Pull latest code ## 🔄 Updates & Upgrades
### Application Updates
```bash
# Pull latest code
git pull origin main git pull origin main
# Rebuild and restart # Rebuild and restart
...@@ -311,8 +418,12 @@ docker-compose up -d enact-api ...@@ -311,8 +418,12 @@ docker-compose up -d enact-api
# Check if migrations are needed # Check if migrations are needed
docker-compose logs enact-api | grep -i "migration\|schema" docker-compose logs enact-api | grep -i "migration\|schema"
Dependency Updates ```
bash# Update Node.js dependencies
### Dependency Updates
```bash
# Update Node.js dependencies
cd app cd app
npm update npm update
npm audit fix npm audit fix
...@@ -320,42 +431,57 @@ npm audit fix ...@@ -320,42 +431,57 @@ npm audit fix
# Rebuild container # Rebuild container
cd .. cd ..
docker-compose build enact-api docker-compose build enact-api
📚 Additional Resources ```
## 📚 Additional Resources
Eclipse Dataspace Connector Documentation - [Eclipse Dataspace Connector Documentation](https://eclipse-edc.github.io/Connector/)
Keycloak Administration Guide - [Keycloak Administration Guide](https://www.keycloak.org/docs/latest/server_admin/)
Docker Compose Reference - [Docker Compose Reference](https://docs.docker.com/compose/)
Nginx Configuration Guide - [Nginx Configuration Guide](https://nginx.org/en/docs/)
## 🆘 Support
🆘 Support
For issues and questions: For issues and questions:
Check logs: docker-compose logs 1. **Check logs**: `docker-compose logs`
Verify configuration: Environment variables and certificates 2. **Verify configuration**: Environment variables and certificates
Test connectivity: Database, Keycloak, external services 3. **Test connectivity**: Database, Keycloak, external services
Review documentation: This README and component docs 4. **Review documentation**: This README and component docs
🔒 Security Considerations ## 🔒 Security Considerations
Production Checklist
### Production Checklist
Strong passwords for all services
Valid SSL certificates with proper chain - [ ] Strong passwords for all services
Firewall rules restricting access - [ ] Valid SSL certificates with proper chain
Regular backups automated - [ ] Firewall rules restricting access
Log monitoring configured - [ ] Regular backups automated
Security headers enabled in Nginx - [ ] Log monitoring configured
API rate limiting configured - [ ] Security headers enabled in Nginx
Database access restricted - [ ] API rate limiting configured
Secrets management (consider using Docker secrets) - [ ] Database access restricted
Regular security updates for all components - [ ] Secrets management (consider using Docker secrets)
- [ ] Regular security updates for all components
Security Headers
### Security Headers
The nginx configuration includes: The nginx configuration includes:
Strict-Transport-Security - `Strict-Transport-Security`
X-Frame-Options - `X-Frame-Options`
X-Content-Type-Options - `X-Content-Type-Options`
X-XSS-Protection - `X-XSS-Protection`
Referrer-Policy - `Referrer-Policy`
Customize these in `nginx/nginx.conf` as needed for your security requirements.
---
## Contributing
Contributions are welcome! Please read our [Contributing Guidelines](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests.
## License
Customize these in nginx/nginx.conf as needed for your security requirements. This project is licensed under the Eclipse Public License 2.0 - see the [LICENSE](LICENSE) file for details.
\ No newline at end of file \ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment