`docker-compose up` vs. `docker compose up` with environment variables

[Choonster]

This is a cross-post from StackOverflow.

I'm trying to deploy a three-node Elasticsearch cluster to an Azure Container Instance using Docker Compose. I'm loosely following this example from the Elasticsearch documentation and this tutorial from the ACI documentation.

When I try to deploy to ACI using the standard docker-compose command I see in most documentation, I get an error saying that it's not supported:

> docker-compose -f .\docker-compose.yml -f .\docker-compose.production.yml up
ERROR: The platform targeted with the current context is not supported.
Make sure the context in use targets a Docker Engine.

When I try to deploy to ACI using the docker compose (no hyphen) command from the ACI documentation, I get an error because it's not loading the environment variables from the .env file:

> docker compose -f .\docker-compose.yml -f .\docker-compose.production.yml up
1 error(s) decoding:

* error decoding 'Volumes[1]': invalid spec: certs:: empty section between colons

Is there a way to load the .env file when using docker compose, or use docker-compose with ACI? I don't fully understand where the docker compose command comes from and why it's different to docker-compose, it doesn't seem to be a standard docker command or a CLI extension that I can see.

These are the relevant files:

docker-compose.yml

# Based on example from Elasticsearch documentation:
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker

version: '3.8'
services:

  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.0
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - ELASTIC_PASSWORD=$ELASTIC_PASSWORD 
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=$CERTS_DIR/es01/es01.key
      - xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.http.ssl.certificate=$CERTS_DIR/es01/es01.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate 
      - xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.transport.ssl.certificate=$CERTS_DIR/es01/es01.crt
      - xpack.security.transport.ssl.key=$CERTS_DIR/es01/es01.key
    volumes:
      - data01:/usr/share/elasticsearch/data
      - 'certs:$CERTS_DIR'
    ports:
      - 127.0.0.1:9200:9200
    networks:
      - elastic
    healthcheck:
      test: curl --cacert $CERTS_DIR/ca/ca.crt -s https://localhost:9200 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
      interval: 30s
      timeout: 10s
      retries: 5
    ulimits:
      memlock:
        soft: -1
        hard: -1
    
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.0
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.security.http.ssl.key=$CERTS_DIR/es02/es02.key
      - xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.http.ssl.certificate=$CERTS_DIR/es02/es02.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate 
      - xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.transport.ssl.certificate=$CERTS_DIR/es02/es02.crt
      - xpack.security.transport.ssl.key=$CERTS_DIR/es02/es02.key
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/elasticsearch/data
      - 'certs:$CERTS_DIR'
    networks:
      - elastic
  
  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.0
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.security.http.ssl.key=$CERTS_DIR/es03/es03.key
      - xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.http.ssl.certificate=$CERTS_DIR/es03/es03.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate 
      - xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.transport.ssl.certificate=$CERTS_DIR/es03/es03.crt
      - xpack.security.transport.ssl.key=$CERTS_DIR/es03/es03.key
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/elasticsearch/data
      - 'certs:$CERTS_DIR'
    networks:
      - elastic

  wait_until_ready:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.0
    command: /usr/bin/true
    depends_on: {"es01": {"condition": "service_healthy"}}

volumes:
  data01:
  data02:
  data03:
  certs:

networks:
  elastic:
    driver: bridge

docker-compose.production.yml:

x-volume: &volume
  driver: azure_file
  driver_opts:
    share_name: acishare
    storage_account_name: <My Storage Account Name>

volumes:
  data01:
    <<: *volume
  data02:
    <<: *volume
  data03:
    <<: *volume
  certs:
    <<: *volume

.env

COMPOSE_PROJECT_NAME=es
CERTS_DIR=/usr/share/elasticsearch/config/certificates 
ELASTIC_PASSWORD=<Default Password>

[Choonster]

Based on this GitHub issue comment, it seems that docker-compose is the original Compose project, its source code lives in the docker/compose repository.

docker compose is a new project that implements the Compose specification and supports ECS and ACI, but doesn't yet support local deployment. Its source code lives in the docker/compose-cli repository.

According to this issue, docker compose doesn't support .env files yet.

[dkf]

@Choonster As the next commenter put it:

So to be clear, users should expect to have two commands in Docker Desktop for the foreseeable future:

• docker compose
• docker-compose

[Jaime]

So, is containerization:

• the innovative way to decouple applications from infrastructure and allow development and operations to make choices independently

or

• the new place to hide all of your dependencies that you have no intention of ever updating, no matter how many security vulnerabilities are discovered

Of course the answer is "yes". And apparently, the future is going to be very much like the past. Some day we'll look back on docker as the new Windows and say "How the hell did we get locked into this crap?"

[dkf]

@Jaime said in `docker-compose up` vs. `docker compose up` with environment variables:

Of course the answer is "yes".

I think you were looking for