Docker and Development Environments: Setting up for Node.js on MacOS

We always had our local development environments setup with Docker on top of Vagrant. While this has worked out just fine, the overhead of building and maintaining them started to increase significantly.

Then here comes containerization with Docker gaining popularity. So, after a few months of random discussions with colleague Bez (who is by the way a Senior Software Engineer AND a really sly Magician) and with the wide availability of Docker for Mac (they just released it sometime on the last week of July last year), I decided to take it up a notch and see how well it would work.

Before we start

Here are a couple of requirements we need installed on our host machine:

File and Directory Structure

Take a look at this overview:

/some-random-project
+-- docker-compose.yml
+-- docker-sync.yml
+-- some-random-app-folder
    +-- bin
    |   +-- start.sh
    +-- Dockerfile
    +-- package.json
    +-- index.js

As you notice, the Dockerfile is under some random app folder. This was intended to, as creating an additional "service" or app within the project can be containerized. This is done so by simply adding in the app folder within the project and with their own Dockerfile.

What is that start.sh file? We'll get to that later.

The Dockerfile

FROM node:latest

ENV PROJECT_ROOT /opt/app

RUN apt-get update \
  && apt-get install -y man postgresql-client-9.4 \
  && rm -rf /var/lib/apt/lists/* \
  && npm install -g pm2

RUN mkdir -p /tmp/app

COPY package.json /tmp/app/package.json

RUN cd /tmp/app/ && npm install --silent

WORKDIR $PROJECT_ROOT

COPY bin /opt/bin

CMD ["/opt/bin/start.sh"]

Since this is for a Node.js project, an official image of Node from Docker Hub should do it quite nicely. Hence, the FROM node:latest line.

We then setup an environment variable for our project's root within the container, that would be ENV PROJECT_ROOT /opt/app.

Then we'll install some dependencies that we need for the project. In this case, we just want a postgresql-client and a global install for the npm package pm2 to run our Node.js project.

RUN apt-get update \
  && apt-get install -y man postgresql-client-9.4 \
  && rm -rf /var/lib/apt/lists/* \
  && npm install -g pm2

We'll also cleanup our package lists as they take up some space and we don't need any more to bloat up our soon-to-be-built Docker image.

Also did you notice how we string the commands into a single RUN statement instead of declaring one RUN statement for each? Since Docker creates a new layer to the image for every statement, doing this will minimize the amount of layers the image will have and thus speed things up and reduce image size.

The next 3 commands are used as a way for dependency files to be started always on a blank slate on build, forcing Docker to not use the cache when we change our application's dependencies:

RUN mkdir -p /tmp/app

COPY package.json /tmp/app/package.json

RUN cd /tmp/app/ && npm install --silent

Then set our work directory for the start.sh default command that we'll be tackling soon enough:

WORKDIR $PROJECT_ROOT

Next we copy over our bin folder to the container and set it as our default for the container when it executes.

COPY bin /opt/bin

CMD ["/opt/bin/start.sh"]

The docker-compose.yml configuration

Feeling accomplished after setting up our base image, we then proceed to configuring our links to supporting containers and other services we might have in mind.

For this we'll use docker-compose which is included in the Docker for Mac app.

version: '2'

services:
  postgres:
    image: postgres
    environment:
      - POSTGRES_PASSWORD=1234
      - POSTGRES_USER=postgres
      - POSTGRES_DB=some-random-db
    ports:
      - '5432:5432'
    volumes:
      - pgdata:/var/lib/postgresql/data

  some-random-app:
    build: ./some-random-app-folder
    ports:
      - '3000:3000'
    volumes:
      - some-random-app-sync:/opt/app:rw
    links:
      - postgres

volumes:
  pgdata: {}
  some-random-app-sync:
    external: true

Breaking it down, the postgres service configuration is pretty much self-explanatory. On the other hand, there is one neat thing to take note from the main some-random-app service: the volumes section. It refers to an external volume that will be provided by docker-sync.

Setting up the docker-sync

So why not use Docker's native syntax for mounting a directory from the local host as a data volume? Though it is quite useful and direct without the need for 3rd party modules, it is just REALLY SLOW and SUPER CPU INTENSIVE! Probaly on a very small scale project it could work out fine, but for bigger projects that will handle hundreds of files, it just doesn't cut it.

Thus, we define another configuration file responsible for providing the some-random-app-sync volume, the docker-sync.yml:

version: '2'

syncs:
  some-random-app-sync:
    src: './some-random-app-folder/'
    dest: '/opt/app'
    sync_host_ip: 'localhost'
    sync_host_port: 10871
    sync_group: root
    sync_userid: 0
    sync_strategy: 'rsync'

Some notes to keep in mind:

The sync name must be globally unique just like how container ports are and should NOT match your real application container name.
The sync_host_port should be a unique port.

The start.sh shell script

What is this start.sh shell script for? Take a look at it's content:

#!/usr/bin/env bash

if [ ! -d $PROJECT_ROOT/node_modules ]; then
  cp -a /tmp/app/node_modules $PROJECT_ROOT
fi

pm2-dev start "$PROJECT_ROOT/index.js"

First, it checks to see if the node_modules dependencies folder exists within the project's root. If not, it copies over the dependencies from the /tmp/app folder within the container (remember those 3 commands in the Dockerfile) to the project's root. This helps us keep the dependencies up-to-date and out of our local host, only existing within the container.

Second, we invoked the pm2-dev command to start the app's index.js file to run in development mode.

Don't forget to set the permissions for start.sh to 0755.

Sample index.js and package.json

Here is a sample index.js file I picked up from Express.

var express = require('express')
var app = express()

app.get('/', function(req, res) {
  res.send('Hello World!')
})

app.listen(3000, function() {
  console.log('Listening on port: 3000...')
})

And here's the package.json that accompanies it.

{
  "dependencies": {
    "express": "^4.14.0"
  }
}

Firing it up!

Now to get the show running, run the command docker-sync start which creates and starts the sync containers, watchers and the sync itself. The command will start a long-running process that runs in the shell's foreground. Pressing Ctrl-C will stop the containers (but it will not remove them). If you wish to run them in the background, you can press Ctrl-Z, followed by executing the bg command. You can bring it back to the foreground by executing the fg command anytime within the same shell session.

Then we'd have to open another tab on the terminal and run docker-compose up to finally get the Docker images built and the application containers initialized.

Don't we have anything that can just run them both in one command? YES we have! Thankfully, docker-sync has a tool called docker-sync-stack. Using this, you can start the sync service and docker compose with one single command.

So to start our project, just run:

$ docker-sync-stack start

This is very convenient so you only need one shell and one command to start working and then CTRL-C to stop.

You can then access your project in the browser and should see a Hello World text from http://localhost:3000/.

When you are finished, you could actually clean up with:

$ docker-sync-stack clean

This cleans the sync-service like docker-sync clean and also removes the application stack like docker-compose down.

There you have it, we now have a very functional Docker local development environment for Node.js on MacOS without the need for Vagrant. Isn't it amazing?