December 08, 2017

JSON on Docker

What is JSON?


JSON is short for JavaScript Object Notation. It implements a lightweight data interchange format based on a subset of JavaScript language. JSON provides a way to store information such that it is easy for machines to parse and generate. Docker uses JSON as its default Logging Driver to communicate information.


JSON is a way to store information in an organized, easy-to-access manner. It is used primarily to transmit data, as an alternative to XML.

As an example of how Docker leverages JSON, here's an excerpt of the docker inspect <Name | ID> output:

$ docker inspect 978d
[
    {
        "Id": "sha256:978d85d02b87aea199e4ae8664f6abf32fdea331884818e46b8a01106b114cee",
        "RepoTags": [
            "debian:latest"
        ],
        "Container": "4799c1aee3356a0d8e51a1e6e48edc1c4ca224e55750e26916f917bdecd96079",
        "ContainerConfig": {
            "Hostname": "ed11f485244a",
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "CMD [\"/bin/bash\"]"
            ],
        },
    }
]

As the above example shows, JSON is built on two structures:
  • Name/Value pairs, delimited by comma
    • NAME:VALUE, NAME:VALUE,…
    • e.g. "Hostname": "ed11f485244a"
  • Ordered list of values
    • realized as an array, vector, list, or sequence
    • e.g. ["/bin/sh","-c","#(nop) ","CMD [\"/bin/bash\"]"]

In JSON, there are several data structures, including:
  • Array
    • An associative array of values
    • begins with [ (left bracket) and ends with ] (right bracket)
    • Values are separated by , (comma)
  • Object
    • Begins with { (left curly brace) and ends with } (right curly brace)
    • An unordered set of name/value pairs
    • Name and value separated by : (colon)
    • Name/Value pairs delimited by , (comma)
    • Object
      • {string : value,…}
  • Value
    • string
    • number
    • object
    • array
    • true
    • false
    • null

JSON’s design goal is simplicity and universality. It aims to be trivial to generate and parse.

References:

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Command Flowchart

Docker Command Flowchart


Inspired by a post at www.troubleshooters.com this flowchart illustrates some of the Docker objects and the commands that apply to them.

For example, to preserve changes made in a container object and convert it into an image, use the docker commit command.


References:

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker network

Docker Commands: docker network


docker network connect

Connect a running container to a network.

Use docker inspect 00db80208c35 to confirm the network connection. In this case, the container is connected to two networks:


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker inspect

Docker Commands: docker inspect


docker inspect

Return low-level information on Docker objects

The target of this command is an object that can be identified via a Name or an ID, e.g. image, container, network, service, etc.

The output of the command is information about the object displayed as a JSON array.

@ubuntu:~$ docker inspect wizardly_jang
[
    {
        "Id": "c794e33bda6bfa60cdc039795ad7712c62df68ca5f8a6d14b906a6a06bc08e43",
        "Created": "2017-04-01T06:02:04.840341671Z",
        "Path": "nginx",
        "Args": [
            "-g",
            "daemon off;"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 4584,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2017-04-29T17:24:19.743601367Z",
            "FinishedAt": "2017-04-29T17:23:24.201882506Z"
        },
        "Image": "sha256:6b914bbcb59e49851990e064568dceee4d53a462f316ec36207599c12ae9ba65",
        "ResolvConfPath": "/var/lib/docker/containers/c794e33bda6bfa60cdc039795ad7712c62df68ca5f8a6d14b906a6a06bc08e43/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/c794e33bda6bfa60cdc039795ad7712c62df68ca5f8a6d14b906a6a06bc08e43/hostname",
        "HostsPath": "/var/lib/docker/containers/c794e33bda6bfa60cdc039795ad7712c62df68ca5f8a6d14b906a6a06bc08e43/hosts",
        "LogPath": "/var/lib/docker/containers/c794e33bda6bfa60cdc039795ad7712c62df68ca5f8a6d14b906a6a06bc06e43/c794e33bda6bfa60cdc039795ad7712c62df68ca5f8a6d14b906a6a06bc08e43-json.log",
        "Name": "/wizardly_jang",
        "RestartCount": 0,
        "Driver": "aufs",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {
                "80/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8080"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Name": "aufs",
            "Data": null
        },
        "Mounts": [],
        "Config": {
            "Hostname": "c794e33bda6b",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "443/tcp": {},
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NGINX_VERSION=1.11.10-1~jessie"
            ],
            "Cmd": [
                "nginx",
                "-g",
                "daemon off;"
            ],
            "ArgsEscaped": true,
            "Image": "nginx",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "a4e88604a69fad8bf3fc99c97444992c98e47dca983024edb2e55072db77560c",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "443/tcp": null,
                "80/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "8080"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/a4e88604a69f",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "a9a5fd1166429b6086accd128055e9992f772c929e8539225d2d57abb3741fa4",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "0aeb282b8e82b97e693e77e5ae0237e9b32ebae09b6bf0510457ec6c516c2a0a",
                    "EndpointID": "a9a5fd1166429b6086accd128055e9992f772c929e8539225d2d57abb3741fa4",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02"
                }
            }
        }
    }
]
@ubuntu:~$

To output a specific field, add the --format or -f option.

Use docker inspect --format "{{.NetworkSettings.IPAddress}}" <container ID> to view the IP Address section of the docker inspect output:


References:

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker history

Docker Commands: docker history


docker history <image_name>

Show the history of an image. In effect, it identifies the "layers" in an image.


Note: There is an online tool, imagelayers.io that can be used to visualize the layers of an image.

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker diff

Docker Commands: docker diff


docker diff <container ID>

Inspect the changes made to a container's filesystem.



Note:
A Added File
C Changed File
D Deleted File

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker exec

Docker Commands: docker exec


docker exec

Run a new process in an already running container

Use docker exec to run a secondary process inside a container.

Let's say the Nginx web proxy container is running in detached (-d) mode, you can use docker exec to start another process in that container. In the example below, the process status command, ps is run inside the Nginx container.

Note: If the container is stopped, it must first be started with docker start.


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker commit

Docker Commands: docker commit


docker commit <container ID>

Containers by default are ephemeral and stateless. Changes made while in the container are discarded when the container is removed. One way to persist container updates or configuration changes, is to "freeze" the container, i.e. convert it back into an image.

The docker commit command creates a new image, incorporating any changes made while a container. You can: start a container, configure it to taste, then commit those changes into a new Docker image:


The new container (started from this newly generated image) reflects the changes configured in the original container.


Note: Building an image via docker commit is not considered a best practice; it is not repeatable or self-documenting, like using docker build and the Dockerfile.

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker build

Docker Commands: docker build


docker build -t <DockerID>/<ImageName> <Dockerfile name>

To build a new image using a Dockerfile.


Using docker build is the preferred way to build a Docker Image. The build instructions are laid out in the Dockerfile, which allows an automated, documented and repeatable way to generate a specific Docker image.

Associated with the docker build command is its context. The build’s context is the files at a specified location: PATH or URL.

PATH is a directory on your local filesystem
URL is a Git repository location

A context is processed recursively, i.e. any subdirectories in the PATH and any submodules in the URL are processed for objects.

By default the build instructions are read from a file called Dockerfile at the root (or top level) of your context. In the example below,  if the docker build command is run from a subdirectory called Files, this becomes its context. The Docker daemon searches this directory and any subdirectories for objects it needs, e.g. Dockerfile.


Note: if the Dockerfile is located outside the context, use the -f option to specify the Dockerfile, e.g. $ docker build -f /path/to/a/Dockerfile .

Note: You can specify a repository (and tag) at which to save the new image, e.g. $ docker build -t neokobo/alp01data:1.0 .

Reference:

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker info

Docker Commands: docker info


docker info

Display system-wide Docker information


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

May 24, 2017

Docker: Troubleshoot This

Troubleshoot This

  1. Describe the command sequences below and explain the error message.
  2. How would you fix it?



Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker search

Docker Commands: docker search


docker search <ImageName>

Looks like command line version of a Docker Hub search


Note:
NAME is the image name.
Names in the format <UserID>/<ImageName> represent images uploaded by non-official sources.
STARS represent the number of likes for a specific image.
OFFICIAL identifies official vendor images.

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker attach

Docker Commands: docker attach


docker attach <container>

Attach to a running container.

First make sure the container is running. If its stopped, start it, then attach to it.


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker ps

Docker Commands: docker ps


docker ps -a

List all containers (running or not).

docker ps lists only the running containers.


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker rmi

Docker Commands: docker rmi


docker rmi <image ID>

Remove one (or more) images




Note: You can identify the image(s) to remove using REPOSITORY instead of the IMAGE ID.

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker rm

Docker Commands: docker rm


docker rm <container ID>

Remove one (or more) containers



Note: You can identify the container(s) to remove using either CONTAINER ID or NAMES:


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

May 13, 2017

Docker Commands: docker run

Docker Commands: docker run


docker run debian ls -ls

What does the above command do?

When you supply the run argument, the Docker daemon finds the image (debian in this case), creates the container and then runs ls -ls command in that container.

In this case, ls -ls is an argument passed on to the container executable (debian), and you see the following:


Note: If the image does not exist locally, an attempt is made to automatically download it from the repository:


docker run -it alpine /bin/sh
When you run this command, Docker daemon does the following:
  1. Runs the alpine image: If the image exists locally, Docker daemon uses it for the new container. Otherwise, Docker Engine pulls it from a registry, e.g. Docker Hub
  2. Creates a new container: Docker allocates a filesystem and mounts a read-write layer on top of the image.
  3. Configures the container: Creates a network bridge interface and attaches an available IP address from a pool
  4. Executes the starting command: Runs the default executable or in this case, /bin/sh from the command line
  5. Manages the data I/O stream: Connects and logs standard input, output and error streams

Running the run command with the -it flags attaches us to an interactive tty in the container. Now you can run as many commands in the container as you want. Take some time to run your favorite commands in the alpine container.


  1. Running docker ps will show if any containers are currently active (running)
  2. docker images lists images available on the local host: nginx, ubuntu, debian, alpine
  3. With docker run, Docker Engine starts the local alpine image running as a container, in interactive mode (-i) and attaches a TTY device (-t) for I/O. After the container starts, it runs the application, in this case the Linux shell, /bin/sh.
    Note: in some cases, you can combine multiple options (e.g. -i -t) into one (e.g. -it).
  4. A couple of things happening behind the scenes before the prompt:
    • Filesystem allocated and mounted as read-write layer
    • A network is attached. Since no network was specified, the default, bridge network driver interface is created. This allows the container to communicate with the local host
    • Allocates an available IP address from a pool
    • Executes the default executable or the override command, in this case, /bin/sh
    • Enables the standard input, output and error streams to allow I/O access with the container

docker run alpine echo "hello from alpine"

In this case, the Docker daemon starts the alpine container, which runs the echo command with the "hello from alpine" argument. The container then immediately exits.

docker run –name web01 -d -p 8080:80 nginx

Starts the nginx web server, names it web01. This allows it to be identified by this name or the automatically generated 64-character ID. It also maps port 80 of the container to port 8080 of the host machine; exposing port 8080 and allowing access via http://localhost:8080 or http://<ip_address:8080>


Note: Nginx (pronounced "engine-x") is an open source reverse proxy server for HTTP, HTTPS, SMTP, POP3, and IMAP protocols, as well as a load balancer, HTTP cache, and a web server (origin server). It runs on Linux, BSD variants, macOS X, Solaris, AIX, HP-UX.
The above command create a container environment with its own isolated: process space, network stack and file system, names it web01, starts the nginx application running in detached mode (or as a daemon) and exposes container port 80 as local port 8080.

docker run --rm
Creates a transient container, i.e. the container is removed after it exits. This automatically runs the equivalent of $ docker rm <containerID> after the container is exited.


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker images

Docker Commands: docker images


$ docker images

Lists all images on the local host


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker help

Docker Commands: docker help


docker run --help

See a list of all flags supported by the run argument.

In general, you can append the --help option to any Docker command for more information, e.g. docker --help or docker <command> --help


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker pull

Docker Commands: docker pull

docker pull <image>

Docker will connect to the Docker Hub and attempt to pull, i.e. download and install, an <image> locally.

E.g. docker pull ubuntu downloads and installs the latest version of the image named ubuntu from Docker Hub.


Note: The above command downloads the most up-to-date version of ubuntu image, or to be technically correct, it pulls (or downloads) from the Docker Hub, the ubuntu image that is either untagged or has the tag latest.

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.