Skip to content

Deployment

After the Docker images are loaded and the application is deployed using docker compose, it is crucial to monitor the containers to ensure they are running as expected. This guide provides detailed steps for monitoring and troubleshooting your containers post-deployment.

Hostname and Kafka Settings Configuration

Before deploying the application, you must configure the hostname parameters in your docker-compose.yml file to ensure proper external access and Kafka connectivity.

Required Configuration Parameters

Backend Service Settings

You need to update these environment variables in the backend service:

  • ALLOWED_HOSTS: Specifies which hosts are allowed to connect to the backend service
  • PUBLIC_HOST: Defines the public URL for API access

Kafka Service Settings

You need to update this environment variable in the kafka service:

  • KAFKA_ADVERTISED_LISTENERS: Configures how Kafka advertises itself to clients for external access

Configuration Steps

Step 1: Identify Your Server Information

Before making changes, gather the following information:

  • Public IP Address: Your server's external IP (e.g., 203.0.113.10)
  • Internal IP Address: Your server's private network IP (e.g., 192.168.1.100)
  • Domain Name: If you have a domain pointing to your server (e.g., apifort.yourdomain.com)

Step 2: Update Backend Service Configuration

In your docker-compose.yml file, locate the backend service and update these lines:

YAML
backend:
  environment:
    ALLOWED_HOSTS: apifort.yourdomain.com,localhost:9050
    PUBLIC_HOST: http://apifort.yourdomain.com
YAML
backend:
  environment:
    ALLOWED_HOSTS: 203.0.113.10,localhost:9050
    PUBLIC_HOST: http://203.0.113.10

Step 3: Update Kafka Service Configuration

In your docker-compose.yml file, locate the kafka service and update the KAFKA_ADVERTISED_LISTENERS line:

YAML
kafka:
  environment:
    KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:9092,INTERNAL2://192.168.1.100:9095,EXTERNAL://203.0.113.10:9094

Step 4: Update for identigro

In your docker-compose.yml file, locate the identigro service and update the Keycloak__Url line:

YAML
identigro:
  environment:
    Keycloak__Url: http://apifort.yourdomain.com

Step 4: Update Keycloak Configuration (if using HTTPS)

If you're using HTTPS, uncomment and update these lines in the keycloak service:

YAML
keycloak:
  environment:
    KC_HOSTNAME: apifort.yourdomain.com
    KC_PROXY: edge

Configuration Examples

Complete Example with Domain Name

YAML
# Backend service configuration
backend:
  environment:
    ALLOWED_HOSTS: apifort.example.com,localhost:9050
    PUBLIC_HOST: http://apifort.example.com

strike-backend:
    ALLOWED_HOSTS: apifort.example.com,localhost:9050

# Kafka service configuration  
kafka:
  environment:
    KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:9092,INTERNAL2://10.0.1.100:9095,EXTERNAL://203.0.113.10:9094

# Keycloak service configuration (for HTTPS)
keycloak:
  environment:
    KC_HOSTNAME: apifort.example.com
    KC_PROXY: edge

Verification Commands

After updating the configuration, verify your settings:

Bash
# Get your public IP
curl ifconfig.me

# Get your internal IP
hostname -I
Bash
# Test Kafka external listener
docker exec -it kafka /opt/kafka/bin/kafka-topics.sh --list --bootstrap-server YOUR_PUBLIC_IP:9094

Deploying the Application

To start the application, use the following commands depending on your operating system:

Bash
cd apifort
docker compose up -d
PowerShell
cd apifort
docker compose up -d
  • The docker compose up -d command will start the application in detached mode, allowing it to run in the background.

Monitoring Containers

After executing the docker compose up -d command, it is essential to monitor the status of your containers to ensure they are running smoothly and to detect any potential issues.

Checking Container Status

To view the status of all containers, use the following command:

Bash
docker compose ps
PowerShell
docker compose ps
  • This command will display a list of all containers along with their current status (e.g., "Up", "Exited", etc.).
  • Look for any containers that are not in the "Up" state, as this may indicate an issue with the deployment.

Viewing Logs

If a container is not running as expected or has exited, you can inspect its logs to identify the root cause of the problem. Use the following command to view logs:

Bash
docker compose logs <service-name>
PowerShell
docker compose logs <service-name>
  • Replace <service-name> with the name of the problematic service.
  • Review the logs for error messages or warnings that might indicate the source of the issue.

Real-Time Log Monitoring

To monitor logs in real-time for all containers, use the following command:

Bash
docker compose logs -f
PowerShell
docker compose logs -f
  • The -f flag allows you to follow the logs as they are being generated.

Service-Specific Log Monitoring

Monitor specific critical services:

Bash
docker compose logs -f backend
Bash
docker compose logs -f kafka
Bash
docker compose logs -f postgresql

Troubleshooting Common Issues

1. Container Exited Immediately

  • Use docker compose logs <container_name> to check for errors in the logs.
  • Verify that all required environment variables are correctly configured in the docker-compose.yml file.
  • Check if the container has sufficient resources allocated.

2. Port Conflicts

  • If a container fails to start due to a port conflict, ensure that the ports specified in the docker-compose.yml file are not already in use by another service.
  • Use netstat -tulpn | grep <port> (Linux) or netstat -an | findstr <port> (Windows) to check port usage.

3. Database Connection Issues

  • If the application cannot connect to the database, ensure that:
  • The database container is running and accessible.
  • The credentials in the docker-compose.yml file are correct.
  • The network configuration allows communication between containers.
  • Test database connectivity: docker exec -it postgresql psql -U appuser -d apifort

4. Kafka Connection Issues

  • Verify Kafka is running: docker compose ps kafka
  • Check Kafka logs: docker compose logs kafka
  • Test Kafka connectivity from inside the network: docker exec -it kafka /opt/kafka/bin/kafka-topics.sh --list --bootstrap-server kafka:9092
  • Test external Kafka access: docker exec -it kafka /opt/kafka/bin/kafka-topics.sh --list --bootstrap-server YOUR_PUBLIC_IP:9094

5. Hostname Resolution Issues

  • Verify your domain DNS settings point to the correct IP address
  • Test domain resolution: nslookup your-domain.com
  • Ensure firewall rules allow traffic on required ports (80, 443, 9094, 9095)

6. Resource Limitations

  • If containers are restarting frequently, check the system resources (CPU, memory, disk space) using the docker stats command.
  • Monitor individual container resource usage: docker stats <container_name>

7. Incorrect Configuration

  • Ensure that all parameters in the docker-compose.yml file are correctly configured, especially for services like MinIO, PostgreSQL, and the application.
  • Validate YAML syntax: docker compose config

Health Check Commands

Verify that critical services are responding correctly:

Bash
curl -f http://localhost:9050/api/actuator/health
# or with your domain
curl -f http://your-domain.com:9050/api/actuator/health
Bash
curl -f http://localhost:8080/auth/health/ready
Bash
docker exec -it postgresql psql -U appuser -tAc 'select 1' -d apifort

Stopping and Restarting Containers

If you need to stop and restart the containers, use the following commands:

Bash
docker compose down
PowerShell
docker compose down
Bash
docker compose up -d
PowerShell
docker compose up -d
Bash
docker compose restart <service-name>

Additional Commands for Container Management

Resource Monitoring

Bash
# View resource usage for all containers
docker stats

# View resource usage for specific container
docker stats <container_name>

# View system resource usage
df -h  # Disk usage
free -h  # Memory usage
top  # CPU usage

Container Inspection

Bash
# Inspect container details
docker inspect <container_name>

# Execute commands inside container
docker exec -it <container_name> /bin/bash

# View container processes
docker exec -it <container_name> ps aux

Network Troubleshooting

Bash
# List Docker networks
docker network ls

# Inspect network details
docker network inspect apifort-network

# Test connectivity between containers
docker exec -it backend ping postgresql
docker exec -it backend ping kafka

Best Practices for Monitoring

Regular Monitoring Tasks

  • Daily: Check container status using docker compose ps
  • Weekly: Review container logs for errors or warnings
  • Monthly: Monitor resource usage trends and optimize if necessary

Automated Monitoring

  • Set up log rotation to prevent disk space issues
  • Configure alerts for container failures
  • Monitor resource usage thresholds
  • Implement health check endpoints monitoring

Security Considerations

  • Regularly update container images
  • Monitor access logs for suspicious activity
  • Ensure proper firewall configuration
  • Use strong passwords and rotate them regularly

Performance Optimization

  • Monitor response times and optimize slow queries
  • Adjust container resource limits based on usage patterns
  • Implement caching strategies where appropriate
  • Regular database maintenance and optimization

Quick Reference Commands

Bash
# Essential monitoring commands
docker compose ps                    # Check container status
docker compose logs -f              # Follow all logs
docker compose logs <service>       # View specific service logs
docker stats                        # Monitor resource usage
docker compose restart <service>    # Restart specific service
docker compose down && docker compose up -d  # Full restart

# Troubleshooting commands
docker compose config               # Validate configuration
docker exec -it <container> bash    # Access container shell
docker inspect <container>          # Detailed container info
netstat -tulpn | grep <port>       # Check port usage (Linux)

Remember to replace placeholder values (like your-domain.com, 203.0.113.10, 192.168.1.100) with your actual server information before deployment.