May 24, 2017

Docker: Troubleshoot This

Troubleshoot This

  1. Describe the command sequences below and explain the error message.
  2. How would you fix it?



Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker search

Docker Commands: docker search


docker search <ImageName>

Looks like command line version of a Docker Hub search


Note:
NAME is the image name.
Names in the format <UserID>/<ImageName> represent images uploaded by non-official sources.
STARS represent the number of likes for a specific image.
OFFICIAL identifies official vendor images.

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker attach

Docker Commands: docker attach


docker attach <container>

Attach to a running container.

First make sure the container is running. If its stopped, start it, then attach to it.


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker ps

Docker Commands: docker ps


docker ps -a

List all containers (running or not).

docker ps lists only the running containers.


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker rmi

Docker Commands: docker rmi


docker rmi <image ID>

Remove one (or more) images




Note: You can identify the image(s) to remove using REPOSITORY instead of the IMAGE ID.

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker rm

Docker Commands: docker rm


docker rm <container ID>

Remove one (or more) containers



Note: You can identify the container(s) to remove using either CONTAINER ID or NAMES:


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

May 13, 2017

Docker Commands: docker run

Docker Commands: docker run


docker run debian ls -ls

What does the above command do?

When you supply the run argument, the Docker daemon finds the image (debian in this case), creates the container and then runs ls -ls command in that container.

In this case, ls -ls is an argument passed on to the container executable (debian), and you see the following:


Note: If the image does not exist locally, an attempt is made to automatically download it from the repository:


docker run -it alpine /bin/sh
When you run this command, Docker daemon does the following:
  1. Runs the alpine image: If the image exists locally, Docker daemon uses it for the new container. Otherwise, Docker Engine pulls it from a registry, e.g. Docker Hub
  2. Creates a new container: Docker allocates a filesystem and mounts a read-write layer on top of the image.
  3. Configures the container: Creates a network bridge interface and attaches an available IP address from a pool
  4. Executes the starting command: Runs the default executable or in this case, /bin/sh from the command line
  5. Manages the data I/O stream: Connects and logs standard input, output and error streams

Running the run command with the -it flags attaches us to an interactive tty in the container. Now you can run as many commands in the container as you want. Take some time to run your favorite commands in the alpine container.


  1. Running docker ps will show if any containers are currently active (running)
  2. docker images lists images available on the local host: nginx, ubuntu, debian, alpine
  3. With docker run, Docker Engine starts the local alpine image running as a container, in interactive mode (-i) and attaches a TTY device (-t) for I/O. After the container starts, it runs the application, in this case the Linux shell, /bin/sh.
    Note: in some cases, you can combine multiple options (e.g. -i -t) into one (e.g. -it).
  4. A couple of things happening behind the scenes before the prompt:
    • Filesystem allocated and mounted as read-write layer
    • A network is attached. Since no network was specified, the default, bridge network driver interface is created. This allows the container to communicate with the local host
    • Allocates an available IP address from a pool
    • Executes the default executable or the override command, in this case, /bin/sh
    • Enables the standard input, output and error streams to allow I/O access with the container

docker run alpine echo "hello from alpine"

In this case, the Docker daemon starts the alpine container, which runs the echo command with the "hello from alpine" argument. The container then immediately exits.

docker run –name web01 -d -p 8080:80 nginx

Starts the nginx web server, names it web01. This allows it to be identified by this name or the automatically generated 64-character ID. It also maps port 80 of the container to port 8080 of the host machine; exposing port 8080 and allowing access via http://localhost:8080 or http://<ip_address:8080>


Note: Nginx (pronounced "engine-x") is an open source reverse proxy server for HTTP, HTTPS, SMTP, POP3, and IMAP protocols, as well as a load balancer, HTTP cache, and a web server (origin server). It runs on Linux, BSD variants, macOS X, Solaris, AIX, HP-UX.
The above command create a container environment with its own isolated: process space, network stack and file system, names it web01, starts the nginx application running in detached mode (or as a daemon) and exposes container port 80 as local port 8080.

docker run --rm
Creates a transient container, i.e. the container is removed after it exits. This automatically runs the equivalent of $ docker rm <containerID> after the container is exited.


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker images

Docker Commands: docker images


$ docker images

Lists all images on the local host


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker help

Docker Commands: docker help


docker run --help

See a list of all flags supported by the run argument.

In general, you can append the --help option to any Docker command for more information, e.g. docker --help or docker <command> --help


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Commands: docker pull

Docker Commands: docker pull

docker pull <image>

Docker will connect to the Docker Hub and attempt to pull, i.e. download and install, an <image> locally.

E.g. docker pull ubuntu downloads and installs the latest version of the image named ubuntu from Docker Hub.


Note: The above command downloads the most up-to-date version of ubuntu image, or to be technically correct, it pulls (or downloads) from the Docker Hub, the ubuntu image that is either untagged or has the tag latest.

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Docker Hub: Tags

Docker Hub: tags


To see a list of tags or version identifiers associated with an image, connect to Docker Hub:


and see its Description, e.g.:


Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

Installation Requirements

Installation Requirements


Platform:
  • Linux:
    • Any distribution running version 3.10+ of the Linux kernel
    • Specific instructions are available for most Linux distributions, including RHEL, Ubuntu, SuSE, and many others.
  • Microsoft Windows:
    • Windows Server 2016
    • Windows 10
  • Cloud:
    • Amazon EC2
    • Google Compute Engine
    • Microsoft Azure
    • Rackspace
References:

May 08, 2017

Docker Word Cloud

Docker Word Cloud 28 12 24 36 12 16 28 20 18 20 104 112 144 20 60 56 24 156 144 128 36 140 148 40 196 112 64 56 296 232 72

Be sure to comment, follow and/or recommend this blog!

WordClouds.com was used to create this Docker Word Cloud.

Creative Commons License Licensed under a Creative Commons Attribution 4.0 International License.

April 29, 2017

Relevant Linux Features: Go Forth And Explore

Go Forth And Explore


You have just lightly scratched the surface… being fresh from a pedicure. You haven't touched any of the items below. The good news is that if you don't already know them, chances are you will not need them… at least not right away:

2>&1 Redirect STDERR and STDOUT to the same place
Metacharacter A character that, when unquoted, separates words.  One of the following:
              |  & ; ( ) < > space tab
Operators ==, !=, =~, …
control operator A token that performs a control function.  It is one of the following symbols:
              || & && ; ;; ( ) | |& <newline>
Process Status 'ps' command, e.g. 'ps -A'
Shell scripting
Compound commands && - e.g. apt-get update && apt-get install -y openssh-server
List - (list)
{ list; }
((expression))
[[…]]
See "man bash"
xargs build and execute command lines from standard input
docker ps -aq | xargs docker rm -f
vi, sed, nawk Various editors and text processing tools

Relevant Linux Features: UNIX Domain Socket

UNIX Domain Socket


UNIX domain socket, also known as IPC (inter-process communication) socket is a data communications endpoint for exchanging data between processes executing on the same host.

Unlike a regular shell pipeline, or pipe (|), which allows communication between command and in only one direction, the UNIX domain socket is bi-directional, i.e. it enables a two-way communications channel.

Similar concepts:
  • TCP Sockets
    • enables bi-directional communication channel between two endpoints over the Internet
  • Pipes
    • one-way communication channel between commands on the local host
    • a sequence of processes chained together by their standard streams, where the standard output (stdout) of one process is fed to the next process through its standard input (stdin)
  • FIFO
    • First In First Out
    • Also known as a named pipe
    • unidirectional communication channel between two processes on the local host
    • Can be accessed by two processes, one to write data, the other to read data
    • Can be created and named by commands: mkfifo or mknod

UNIX domain socket allows processes on the same host can communicate.
All communication occurs entirely within the operating system kernel
Unix domain sockets use the file system as their address name space.
A UNIX domain socket is known by a pathname where the client and server agree on the pathname used for the communication

The /var/run/docker.sock is an implementation of the UNIX domain socket, used by Docker. The Docker daemon listens on it by default and it enables communication between the Docker daemon and containers.


References:

April 25, 2017

Relevant Linux Features: Exit Status

Exit Status


On Unix and Linux systems, every command ends with an exit status (also known as return status or exit code). Exit status is an integer value ranging from 0 to 255.

By default, a command that ends "successfully" has an exit status of zero, 0.
A command that ends with a "failure" has a non-zero (1 - 255) exit status.

Note: Success and failure, with respect to exit status is relative. By default if a command does what it's expected to do, on exit it sets a zero, 0, exit status. E.g. If the directory /var/log/apt exists, the command ls /var/log/apt will end successfully and result in an exit status of 0. However if the argument, in this case a directory, is not accessible the ls command will "fail" and leave a non-zero exit status:


By convention, success results in an exit status of zero, however commands are generally free to decide what non-zero integer, between 1 and 255 to use. In the above example, ls uses an exit status of 2 to reflect that a directory is not accessible. And docker chooses an exit status of 125 to reflect that the image is not accessible from the repository.

Commands are free to choose which value to use to reflect success or failure. However there are some reserved value that have special meaning, defined here: http://www.tldp.org/LDP/abs/html/exitcodes.html

A command writes its exit status into the ? shell variable, accessible via $?. This variable can hold one value at a time, as such it is overwritten when the next command exits.

To read the previous command's exit status, use the command, echo $?.

Summary:

0 the exit status of a command on success
1 - 255 the exit status of a command on failure
? holds the exit status of the last command executed
$? reads the exit status of the last command executed

References:

Relevant Linux Features: Control Operator

Control Operator


A Control Operator is a token that performs a control function.  It is one of the following symbols: || & && ; ;; ( ) | |& <newline>. We will focus on only && and || control operators in this article.

On occasion you might need to group Docker commands. Let's see a few ways to do this in Linux with three of the control operators.

Control operators Description
; Semicolon - delimits commands in a sequence
Used to run multiple commands one after the other
Similar to hitting ENTER after each command:

$ docker run --rm -it debian bash -c "ls /var; sleep 1; ls /"

Run the container and execute the three commands one after the other, separated by ; (semicolon)
&& AND - runs commands conditionally, on success
has the form A && B where B is run if, and only if A succeeds, i.e. if A returns an exit status of zero.


$ apt-get update && apt-get install -y openssh-server

This runs the second command, apt-get install -y openssh-server, IF AND ONLY IF the first command, apt-get update succeeded.
|| OR - runs command conditionally, on failure
has the form A || B where B is run if, and only if A fails, i.e. if A returns a non-zero exit status


This runs the second command, IF AND ONLY IF, the first command fails. In this example, since the first command, false will always fail, i.e. return a non-zero exit status, the second command, true, runs and sets the zero exit status.

April 22, 2017

Relevant Linux Features: Pipe

Relevant Linux Features: Pipe

The pipe is implemented with the  "|" symbol. It takes the output (stdout) of the command on the left and sends it as input (stdin)  for the command on the right:


In the example below, docker run --help is the first command. Its output is used as input to the more command, which displays the output, one screen at a time:

Note: stderr (standard error) is NOT passed through the pipe. I.e. we are not able to pass stderr through the pipe as we can stdout.

Relevant Linux Features: Command Substitution

Command substitution - $(command)

In command substitution, the shell runs command, however instead of displaying the output of command, it stores the output in a variable. You can then pass that variable as input to another command.

The syntax of command substitution is $(command) or the older `command`, using back-ticks.

Let's say you want to remove the most recent container running. You can use docker ps -a which lists all containers, starting with the most recent, then copy the Container ID into the docker rm <Container ID> command:

 
Alternatively, you can use Command Substitution and let the system do some of the work for you. The following command runs docker ps -lq which gets the ID of the most recent container, it passes that ID to the docker rm command: $ docker rm $(docker ps -lq):

Relevant Linux Features: Standard I/O

Relevant Linux Features: Standard I/O

stdin, stdout, stderr:
Linux recognizes three input/output streams:
  • STDIN
    • standard input
    • by default input to a command comes from the keyboard
    • STDIN can be redirected to come from other than the keyboard, e.g. a file or a device
      • use the "<" symbol to redirect input, e.g. command < file
      • e.g. to send the contents of file, text001 as INPUT to the command pr and offset each line by 5 spaces:
        $ pr --indent=5 < text001
  • STDOUT
    • standard output
    • by default, output of a command is sent to the terminal
    • STDOUT can be redirected to go to other than the terminal, e.g. a file or a device
      • use the ">" symbol to redirect output, e.g. command > file
      • e.g. send the output of command, ls to a file, text001 instead of the terminal:
        $ ls > text001
  • STDERR
    • standard error
    • by default, error from a command is sent to the terminal
    • STDERR can be redirected to go to other than the terminal, e.g. a file or a device
      • use the "2>" symbol to redirect error, e.g. command 2> file
      • e.g. send the error of command, ls to a file, capture.err instead of the terminal:
        $ ls file 2> capture.err

Relevant Linux Features: Redirection

Relevant Linux Features: Redirection

Linux allows I/O to be redirected away from the default source or target.

The default source of STDIN is the keyboard, i.e. by default a command expects to get its input from the keyboard. To send input to a command, from a file, use the "<" redirection symbol.

The default target of STDOUT is the terminal or screen, i.e. by default a command expects to send its output to the screen. To redirect it elsewhere, use the ">" symbol:

Note: "command > file" sends the output to a file, "file". If "file" already exists, any existing content is overwritten. To instead append the output to the end of the file, use ">>" instead, i.e. "command >> file".

The default target of STDERR is the screen, i.e. by default a command expects to send its error output to the screen. To redirect it elsewhere, use the "2>" symbol:

Note: "command 2> file" send the output to a new file, "file". If "file" already exists, any existing content is overwritten. To instead append any new output to the end of the file, use "2>>" instead, i.e. "command 2>> file".