Docker

Workshop

Workshop

Docker Basics

Basics

Command

Description

docker version

Display docker version of client and server

docker info

Display various docker information and configuration

docker

Display all available commands

New Management Command Line

docker <command> <sub-command> (options)

Command

Description

docker container run --publish 80:80 nginx

Create a NGINX container in the foreground

docker container run --publish 80:80 --detach nginx

Create a NGINX container in the background

docker container ls

List all active containers

docker container stop 7e4

Stop a container 7e4

docker container ls -a

List all containers (active and inactive)

docker container run --publish 80:80 --detach --name webhost nginx

Create a NGINX container with given name webhost

docker container logs webhost

Show logs of the container webhost

docker container top webhost

Display running processes of the container webhost

docker container --help

Display all container’s sub-commands

docker container rm 860 74e 957

Delete container 860, 74e, and 957 (non-active containers only)

docker container rm -f 860

Forced delete container 860 even it is running

What’s Happened?

  1. Look for image locally
  2. Look for image in remote repository (Docker Hub is the default repo)
  3. Download the latest version by default
  4. Create a new container and prepare to start
  5. Give a virtual IP on a private network inside docker engine
  6. Open port 80 on host and forward to port 80 in container
  7. Start container by using the CMD in Dockerfile

Old Command Line

Command

Description

docker run --name mongo -d mongo

Create a new mongo DB container named mongo

docker ps

List all active containers

docker top mongo

Display running processes of the container mongo

docker stop mongo

Stop the container mongo

docker start mongo

Start the container mongo

Manage Multiple Containers

docker container run -d -p 3306:3306 --name db -e MYSQL_RANDOM_ROOT_PASSWORD=true mysql

Create a MySQL container

  • in detach/background (-d) mode
  • publish port 3306 on host to 3306 on container
  • assign name db
  • set environment variable MYSQL_RANDOM_ROOT_PASSWORD to true to generate root password and print out in the log

Command

Description

docker container run -d --name webserver -p 8080:80 httpd

Create a new HTTPd (Apache) container

docker container run -d --name proxy -p 80:80 nginx

Create a new NGINX container

docker container inspect mysql

See information on how the container mysql started in JSON format

docker container stats

See live stream of container statistics

Shell Inside Containers

docker container run -it --name proxy nginx bash

Create a new NGINX container named proxy and run bash command right after and keep terminal opened (-it.)
*Container will stop after exit

Ubuntu Container

docker container run -it --name ubuntu ubuntu

Try more commands:

apt-get update  
apt-get install -y curl  
curl google.com  

Command

Description

docker container start -ai ubuntu

Start the container ubuntu and open shell

docker container exec -it mysql bash

Create a new process and run bash inside container mysql

docker pull alpine

Pull the image alpine

docker image ls

List all local images

docker container run -it alpine sh

Create a new alpine container and open sh (bash is not available in alpine)

docker container run --rm -it centos:7

Create CentOS v7 container and open shell. Container is removed after exit.

docker container run --rm -it ubuntu

Create Ubuntu latest version and open shell. Container is removed after exit.

Docker Networking

Default Network

  • Each container connected to a private virtual network ‘bridge’ or ‘docker0’
  • Each virtual network routes through NAT firewall on host IP i.e. containers can goes out to the internet via the host machine
  • All containers can talk to each other without -p
  • Best practice is to create a new virtual network for each app (which may containing multiple containers)
  • “Batteries includes, but removable”

Command

Description

docker container run -p 80:80 --name webhost -d nginx

Create a new NGINX container named webhost and map port 80—>80

docker container port webhost

See port mapping for container webhost

docker container inspect --format '' webhost

See IP address of the container webhost

ifconfig en0

See IP address of local machine

docker network ls

List all networks attached to Docker

docker network inspect bridge

See which containers attach to the network bridge

Creating a New Virtual Network

Command

Description

docker network create my_app_net

Create a new bridge network named my_app_net

docker container run -d --name new_nginx --network my_app_net nginx

Create a new container in the specified networt my_app_net

docker network connect webhost my_app_net

Connect container webhost to network my_app_net

docker container inspect webhost

See which network the container webhost connect to

docker network disconnect my_app_net webhost

Disconnect container webhost from network my_app_net

DNS

Because containers are always moving, come and go, all the times. Should not rely on IPs but names.

Command

Description

docker container run -d --name new_nginx --network my_app_net nginx:alpine

Create NGINX container named new_nginx and attach my_app_net

docker container run -d --name my_nginx --network my_app_net nginx:alpine

Create NGINX container named my_nginx and attach my_app_net

docker network inspect my_app_net

See both containers are attached to my_app_net

docker container exec -it my_nginx ping new_nginx

Ping new_nginx from my_nginx

docker container exec -it new_nginx ping my_nginx

Ping my_nginx from new_nginx

DNS Round Robin

Command

Description

docker network create dude

Create a new network named dude

docker container run -d --network dude --net-alias search elasticsearch:2

Create a new Elasticsearch container and attach to network dude with alias search

docker container run -d --network dude --net-alias search elasticsearch:2

Create another Elasticsearch container and attach to network dude with alias search

docker container run --rm --network dude alpine nslookup search

See NS entries mapped to DNS search

docker container run --rm --network dude centos:7 curl -s search:9200

Get Elastic search result from DNS search

Docker Image

Visit Docker Hub

Try to select image with more pulls and stars

Command

Description

docker pull centos

Download the latest (default) version of CentOS image

docker pull nginx:1.11

Download the version 1.11 of NGINX image

docker image ls

See all downloaded images

Official Images

Image Layer

  • Docker always stores only one copy of an image layer.
  • Image is checked via SHA hash for identical validation.
  • When running a container, Docker create a R/W layer on top of an image

Command

Description

docker history nginx:latest

See history of image nginx:’latest

docker image inspect nginx

Display metadata of the image nginx e.g. exposed port, available environment variable, commands executed when a container is created

Image Tagging and Pushing

  • Image can be uniquely referred by <user>/<image>:<tag>
  • <user> is omitted for official images
  • Tag is a label to image ID. One image ID can have many tags

Command

Description

docker image tag nginx pacroy/nginx

Create a new tag

docker image push pacroy/nginx

Upload tag to Docker Hub

cat .docker/config.json

See local Docker config

docker image tag pacroy/nginx pacroy/nginx:testing

Create another tag

docker image push pacroy/nginx:testing

Upload tag which layers already exist

Dockerfile

A recipe to create your own image.

Basic

Command

Description

FROM

Base image

WORKDIR

Works like cd

ENV

Key-value of environment variables

RUN

Commands to execute when building image *Use && to add more commands to make sure changes are put on the same layer

EXPOSE

Ports to expose

CMD

Commands to execute when running/starting container

COPY

Copy file from host into image

Docker Build

When rebuilding the image, only image layer that changed are rebuilt.

Command

Description

docker image build -t customnginx .

Build a new image tag customnginx from Dockerfile in current directory

docker image build -t nginx-with-html .

Build a new image tag nginx-with-html from Dockerfile in current directory

docker container run -p 80:80 --rm nginx-with-html

Run a container of the new image tag

docker image tag nginx-with-html:latest pacroy/nginx-with-html:latest

Re-tagging by creating a new tag

Dockerfile Example

# Instructions from the app developer
# - you should use the 'node' official image, with the alpine 6.x branch
FROM node:6-alpine

# - this app listens on port 3000, but the container should launch on port 80
#  so it will respond to http://localhost:80 on your computer
EXPOSE 3000

# - then it should use alpine package manager to install tini: 'apk add --update tini'
# - then it should create directory /usr/src/app for app files with 'mkdir -p /usr/src/app'
RUN apk add --update tini && \
    mkdir -p /usr/src/app

WORKDIR /usr/src/app

# - Node uses a "package manager", so it needs to copy in package.json file
COPY package.json package.json

# - then it needs to run 'npm install' to install dependencies from that file
# - to keep it clean and small, run 'npm cache clean --force' after above
RUN npm install && npm cache clean --force

# - then it needs to copy in all files from current directory
COPY . .

# - then it needs to start container with command 'tini -- node ./bin/www'
# - in the end you should be using FROM, RUN, WORKDIR, COPY, EXPOSE, and CMD commands
CMD [ "tini", "--", "node", "./bin/www" ]

Data Volume

 

VOLUME in Dockerfile

  • Tells docker to create a new volume and mount at the specified location
  • Can also be inspected (docker image inspect mysql)

Command

Description

docker container run -d --name mysql -e MYSEL_ALLOW_EMPTY_PASSWORD=true mysql

Create a new MySQL container

docker volume ls

List all created volumes

docker container inspect mysql

See which volume is mounted

Named Volume

Command

Description

docker container run -d --name mysql \
-e MYSEL_ALLOW_EMPTY_PASSWORD=true \
-v mysql-db:/var/lib/mysql \
mysql

Create a new MySQL container with named volume

docker container run -d --name mysql2 \
-e MYSEL_ALLOW_EMPTY_PASSWORD=true \
-v mysql-db:/var/lib/mysql \
mysql

Create another MySQL container using the same volume

docker volume create test-volume

Create a new volume named test-volume

Bind Mounting

  • Two locations pointing to the same thing
  • Can’t specified in Dockerfile
  • Must be specified at docker run

Command

Description

docker container run -d --name nginx \
-p 80:80 \
-v $(pwd):/usr/share/nginx/html \
nginx

Create a new NGINX container with bind mount from current directory to /usr/share/nginx/html in the container

docker run -p 80:4000 \
-v $(pwd):/site \
bretfisher/jekyll-serve

Create a new Jekyll container with bind mount from current directory to /site

Useful Commands

Command

Description

Reference

uname -a

Print linux kernel version

whatsmyos.com

lsb_release -a

Print linux distro and version

askubuntu.com

cat /etc/*release

Print linux distro and version

whatsmyos.com

docker ps -a --filter \
volume=VOLUME_NAME_OR_MOUNT_POINT

Determine what containers use the docker volume

 

openssl req -x509 -newkey rsa:2048 \
-keyout key.pem -out cert.pem \
-days 30000 -nodes

Generate self-signed certificates

krishnasrinivas/wetty

ssh-keygen -t rsa -b 4096 \
-C “your_comment_or_email”

Generate SSH key

Connecting to GitHub with SSH

Workshop

Docker Compose

Basics

Why

  • configure relationships between containers
  • save our docker container run settings in easy-to-read file
  • create one-liner developer environment startups

Components

Comprised of 2 separate but related things

    1. YAML-formatted file that describes our solution options for:
    • containers
    • networks
    • volumes 2. A CLI tool docker-compose used for local dev/test automation with those YAML files

docker-compose.yml

  • Compose YAML format has it’s own versions: 1, 2, 2.1, 3, 3.1
  • YAML file can be used with docker-compose command for local docker automation or..
  • With docker directly in production with Swarm (as of v1.13)
  • docker-compose --help
  • docker-compose.yml is default filename, but any can be used with docker-compose -f

Template

version: '3.1'  # if no version is specificed then v1 is assumed. Recommend v2 minimum

services:  # containers. same as docker run
  servicename: # a friendly name. this is also DNS name inside network
    image: # Optional if you use build:
    command: # Optional, replace the default CMD specified by the image
    environment: # Optional, same as -e in docker run
    volumes: # Optional, same as -v in docker run
  servicename2:

volumes: # Optional, same as docker volume create

networks: # Optional, same as docker network create

Example 1

This create a Jekyll service.

version: '2'

# same as 
# docker run -p 80:4000 -v $(pwd):/site bretfisher/jekyll-serve

services:
  jekyll:
    image: bretfisher/jekyll-serve
    volumes:
      - .:/site
    ports:
      - '80:4000'

Example 2

This creates a wordpress service.

version: '2'

services:

  wordpress:
    image: wordpress
    ports:
      - 8080:80
    environment:
      WORDPRESS_DB_HOST: mysql
      WORDPRESS_DB_NAME: wordpress
      WORDPRESS_DB_USER: example
      WORDPRESS_DB_PASSWORD: examplePW
    volumes:
      - ./wordpress-data:/var/www/html

  mysql:
    image: mariadb
    environment:
      MYSQL_ROOT_PASSWORD: examplerootPW
      MYSQL_DATABASE: wordpress
      MYSQL_USER: example
      MYSQL_PASSWORD: examplePW
    volumes:
      - mysql-data:/var/lib/mysql

volumes:
  mysql-data:
	

Docker Compose CLI

  • CLI tool comes with Docker for Windows/Mac, but separate download for Linux
  • Not a production-grade tool but ideal for local development and test
  • Two most common commands are
    • docker-compose up # setup volumes/networks and start all containers
    • docker-compose down # stop all containers and remove cont/vol/net
  • If all your projects had a Dockerfile and docker-compose.yml then “new developer onboarding” would be:
    • git clone github.com/some/software
    • docker-compose up

Demo

Clone https://github.com/BretFisher/udemy-docker-mastery/tree/master/compose-sample-2 docker-compose up for running in the foreground docker-compose up -d for running in the backgrounddocker-compose logs to see the log docker-compose ps to see list of containers docker-compose topto see list of processes docker-compose down to stop and remove the service

Assignment: Writing A Compose File

  • Build a basic compose file for a Drupal content management system website. Docker Hub is your friend
  • Use the drupal image along with the postgres image
  • Use ports to expose Drupal on 8080 so you can localhost:8080
  • Be sure to set POSTGRES_PASSWORD for postgres
  • Walk though Drupal setup via browser
  • Tip: Drupal assumes DB is localhost, but it’s service name
  • Extra Credit: Use volumes to store Drupal unique data

Answer

version: '2'

services: 
  drupal:
    image: drupal
    ports: 
      - 8080:80
    volumes: 
      - drupal-modules:/var/www/html/modules
      - drupal-profiles:/var/www/html/profiles
      - drupal-sites:/var/www/html/sites
      - drupal-themes:/var/www/html/themes
  postgres:
    image: postgres
    environment: 
      - POSTGRES_PASSWORD=yourpasswd

volumes: 
  drupal-modules:
  drupal-profiles:
  drupal-sites:
  drupal-themes:
Workshop

Docker Swarm

Swarm Basics

Containers Everywhere = New Problems

  • How do we automate container lifecycle?
  • How can we easily scale out/in/up/down?
  • How can we ensure our containers are re-created if they fail?
  • How can we replace containers without downtime (blue/green deploy)?
  • How can we control/track where containers get started?
  • How can we create cross-node virtual networks?
  • How can we ensure only trusted servers run our containers?
  • How can we store secrets, keys, passwords and get them to the right container (and only that container)?

Swarm Mode: Built-In Orchestration

  • Swarm Mode is a clustering solution built inside Docker
  • Not related to Swarm “classic” for pre-1.12 versions
  • Added in 1.12 (Summer 2016) via SwarmKit toolkit
  • Enhanced in 1.13 (January 2017) via Stacks and Secrets
  • Not enabled by default, new commands once enabled
    • docker swarm
    • docker node
    • docker service
    • docker stack
    • docker secret

Swarm Architecture

    • Swarm Mode

Swarm Topology

Nginx On Swarm

Swarm Architecture

Swarm Initialization

  • docker info | grep Swarm See whether swarm is active
  •  docker swarm init Activate swarm mode

docker swarm init: What Just Happened?

  • Lots of PKI and security automation
    • Root Signing Certificate created for our Swarm
    • Certificate is issued for first Manager node
    • Join tokens are created
  • Raft database created to store root CA, configs and secrets
    • Encrypted by default on disk (1.13+)
    • No need for another key/value system to hold orchestration/secrets
    • Replicates logs amongst Managers via mutual TLS in “control plane”
  • docker node ls List nodes in swarm 
  • docker swarm join-token worker Get join token for worker node

Test Swarm

  • docker service create alpine ping 8.8.8.8 Create alpine container and ping 8.8.8.8 
  • docker service ls List services
  • docker service ps <service_name> List of processes (or containers) running for the given services
  • docker service update <service_name> --replicas 3 Update the service to run with 3 containers
  • docker container rm -f <container_name> Try to remove the container and the orchestrator will spin it up automatically
  •  docker service rm <service_name> Remove the given service

Docker Class (Basic)

Course Materials

Playing with Docker

Extras

Container Behind Nginx Server

Add the following location to /etc/nginx/conf.d/default.conf

    location /wetty {
        proxy_pass http://wetty:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 43200000;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
    }

Sample default.conf

server {
    listen 80;
    listen [::]:80;
    server_name _;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    listen [::]:443;

    server_name  localhost;

    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS";
    ssl_prefer_server_ciphers on;

    ssl_certificate     /etc/nginx/cert.pem;
    ssl_certificate_key /etc/nginx/key.pem;
    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

    #location / {
    #    root   /usr/share/nginx/html;
    #    index  index.html index.htm;
    #}

    location /wetty {
        proxy_pass http://wetty:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 43200000;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
    }

    location / {
        proxy_pass http://wikijs:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 43200000;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}

How to Deploy Docker Swarm on Azure

1. Deploy

1) Go to https://store.docker.com/editions/community/docker-ce-azure and select preferred version. It will lunch Azure console.