Ruan Bekker's Blog

From a Curious mind to Posts on Github

Deploy Docker Swarm Using Ansible

In this setup we will use Ansible to Deploy Docker Swarm.

With this setup, I have a client node, which will be my jump box, as it will be used to ssh with the docker user to my swarm nodes with passwordless ssh access.

The repository for the source code can be found on my Github Repository

Pre-Check

Hosts file:

1
2
3
4
5
$ cat /etc/hosts
10.0.8.2 client
192.168.1.10 swarm-manager
192.168.1.11 swarm-worker-1
192.168.1.12 swarm-worker-2

SSH Config:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$ cat ~/.ssh/config 
Host client
  Hostname client
  User root
  IdentityFile /tmp/key.pem
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

Host swarm-manager
  Hostname swarm-manager
  User root
  IdentityFile /tmp/key.pem
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

Host swarm-worker-1
  Hostname swarm-worker-1
  User root
  IdentityFile /tmp/key.pem
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

Host swarm-worker-2
  Hostname swarm-worker-2
  User root
  IdentityFile /tmp/key.pem
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

Install Ansible:

1
2
3
$ apt install python-setuptools -y
$ easy_install pip
$ pip install ansible

Ensure passwordless ssh is working:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ ansible -i inventory.ini -u root -m ping all
client | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
swarm-manager | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
swarm-worker-2 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
swarm-worker-1 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

Deploy Docker Swarm

1
2
3
4
5
6
7
$ ansible-playbook -i inventory.ini -u root deploy-swarm.yml 
PLAY RECAP 

client                     : ok=11   changed=3    unreachable=0    failed=0   
swarm-manager              : ok=18   changed=4    unreachable=0    failed=0   
swarm-worker-1             : ok=15   changed=1    unreachable=0    failed=0   
swarm-worker-2             : ok=15   changed=1    unreachable=0    failed=0   

SSH to the Swarm Manager and List the Nodes:

1
2
3
4
5
$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
0ead0jshzkpyrw7livudrzq9o *   swarm-manager       Ready               Active              Leader              18.03.1-ce
iwyp6t3wcjdww0r797kwwkvvy     swarm-worker-1      Ready               Active                                  18.03.1-ce
ytcc86ixi0kuuw5mq5xxqamt1     swarm-worker-2      Ready               Active                                  18.03.1-ce

Test Application on Swarm

Create a Nginx Demo Service:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$ docker network create --driver overlay appnet
$ docker service create --name nginx --publish 80:80 --network appnet --replicas 6 nginx
$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
k3vwvhmiqbfk        nginx               replicated          6/6                 nginx:latest        *:80->80/tcp

$ docker service ps nginx
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE            ERROR               PORTS
tspsypgis3qe        nginx.1             nginx:latest        swarm-manager       Running             Running 34 seconds ago                       
g2f0ytwb2jjg        nginx.2             nginx:latest        swarm-worker-1      Running             Running 34 seconds ago                       
clcmew8bcvom        nginx.3             nginx:latest        swarm-manager       Running             Running 34 seconds ago                       
q293r8zwu692        nginx.4             nginx:latest        swarm-worker-2      Running             Running 34 seconds ago                       
sv7bqa5e08zw        nginx.5             nginx:latest        swarm-worker-1      Running             Running 34 seconds ago                       
r7qg9nk0a9o2        nginx.6             nginx:latest        swarm-worker-2      Running             Running 34 seconds ago   

Test the Application:

1
2
3
4
5
6
7
8
9
10
$ curl -i http://192.168.1.10
HTTP/1.1 200 OK
Server: nginx/1.15.0
Date: Thu, 14 Jun 2018 10:01:34 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 05 Jun 2018 12:00:18 GMT
Connection: keep-alive
ETag: "5b167b52-264"
Accept-Ranges: bytes

Delete the Service:

1
2
3

$ docker service rm nginx
nginx

Delete the Swarm:

1
2
3
4
5
6
$ ansible-playbook -i inventory.ini -u root delete-swarm.yml 

PLAY RECAP 
swarm-manager              : ok=2    changed=1    unreachable=0    failed=0   
swarm-worker-1             : ok=2    changed=1    unreachable=0    failed=0   
swarm-worker-2             : ok=2    changed=1    unreachable=0    failed=0   

Ensure the Nodes is removed from the Swarm, SSH to your Swarm Manager:

1
2
$ docker node ls
Error response from daemon: This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.

Setup a 3 Node Ceph Storage Cluster on Ubuntu 16

For some time now, I wanted to do a setup of Ceph, and I finally got the time to do it. This setup was done on Ubuntu 16.04

What is Ceph

Ceph is a storage platform that implements object storage on a single distributed computer cluster and provides interfaces for object, block and file-level storage.

  • Object Storage:

Ceph provides seemless access to objects via native language bindings or via the REST interface, RadosGW and also compatible for applications written for S3 and Swift.

  • Block Storage:

Ceph’s Rados Block Device (RBD) provides access to block device images that are replicated and striped across the storage cluster.

  • File System:

Ceph provides a network file system (CephFS) that aims for high performance.

Our Setup

We will have 4 nodes. 1 Admin node where we will deploy our cluster with, and 3 nodes that will hold the data:

  • ceph-admin (10.0.8.2)
  • ceph-node1 (10.0.8.3)
  • ceph-node2 (10.0.8.4)
  • ceph-node3 (10.0.8.5)

Host Entries

If you don’t have dns for your servers, setup the /etc/hosts file so that the names can resolves to the ip addresses:

1
2
3
4
10.0.8.2 ceph-admin
10.0.8.3 ceph-node1
10.0.8.4 ceph-node2
10.0.8.5 ceph-node3

User Accounts and Passwordless SSH

Setup the ceph-system user accounts on all the servers:

1
2
$ useradd -d /home/ceph-system -s /bin/bash -m ceph-system
$ passwd ceph-system

Setup the created user part of the sudoers that is able to issue sudo commands without a pssword:

1
2
$ echo "ceph-system ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/ceph-system
$ chmod 0440 /etc/sudoers.d/ceph-system

Switch user to ceph-system and generate SSH keys and copy the keys from the ceph-admin server to the ceph-nodes:

1
2
3
4
5
6
$ sudo su - ceph-system
$ ssh-keygen -t rsa -f ~/.ssh/id_rsa -P ""
$ ssh-copy-id ceph-system@ceph-node1
$ ssh-copy-id ceph-system@ceph-node2
$ ssh-copy-id ceph-system@ceph-node3
$ ssh-copy-id ceph-system@ceph-admin

Pre-Requisite Software:

Install Python and Ceph Deploy on each node:

1
2
$ sudo apt-get install python -y
$ sudo apt install ceph-deploy -y

Note: Please skip this section if you have additional disks on your servers.

The instances that im using to test this setup only has one disk, so I will be creating loop block devices using allocated files. This is not recommended as when the disk fails, all the (files/block device images) will be gone with that. But since im demonstrating this, I will create the block devices from a file:

I will be creating a 12GB file on each node

1
2
$ sudo mkdir /raw-disks
$ sudo dd if=/dev/zero of=/raw-disks/rd0 bs=1M count=12288

The use losetup to create the loop0 block device:

1
$ sudo losetup /dev/loop0 /raw-disks/rd0

As you can see the loop device is showing when listing the block devices:

1
2
3
$ lsblk
NAME      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0       7:0    0   12G  0 loop

Install Ceph

Now let’s install ceph using ceph-deploy to all our nodes:

1
2
$ sudo apt update && sudo apt upgrade -y
$ ceph-deploy install ceph-admin ceph-node1 ceph-node2 ceph-node3

The version I was running at the time:

1
2
$ ceph --version
ceph version 10.2.9

Initialize Ceph

Initialize the Cluster with 3 Monitors:

1
$ ceph-deploy new ceph-node1 ceph-node2 ceph-node3

Add the initial monitors and gather the keys from the previous command:

1
$ ceph-deploy mon create-initial

At this point, we should be able to scan the block devices on our nodes:

1
2
3
$ ceph-deploy disk list ceph-node3
[ceph-node3][INFO  ] Running command: sudo /usr/sbin/ceph-disk list
[ceph-node3][DEBUG ] /dev/loop0 other

Prepare the Disks:

First we will zap the block devices and then prepare to create the partitions:

1
2
3
4
5
$ ceph-deploy disk zap ceph-node1:/dev/loop0 ceph-node2:/dev/loop0 ceph-node3:/dev/loop0
$ ceph-deploy osd prepare ceph-node1:/dev/loop0 ceph-node2:/dev/loop0 ceph-node3:/dev/loop0
[ceph_deploy.osd][DEBUG ] Host ceph-node1 is now ready for osd use.
[ceph_deploy.osd][DEBUG ] Host ceph-node2 is now ready for osd use.
[ceph_deploy.osd][DEBUG ] Host ceph-node3 is now ready for osd use.

When you scan the nodes for their disks, you will notice that the partitions has been created:

1
2
3
$ ceph-deploy disk list ceph-node1
[ceph-node1][DEBUG ] /dev/loop0p2 ceph journal, for /dev/loop0p1
[ceph-node1][DEBUG ] /dev/loop0p1 ceph data, active, cluster ceph, osd.0, journal /dev/loop0p2

Now let’s activate the OSD’s by using the data partitions:

1
$ ceph-deploy osd activate ceph-node1:/dev/loop0p1 ceph-node2:/dev/loop0p1 ceph-node3:/dev/loop0p1

Redistribute Keys:

Copy the configuration files and admin key to your admin node and ceph data nodes:

1
$ ceph-deploy admin ceph-admin ceph-node1 ceph-node2 ceph-node3

If you would like to add more OSD’s (not tested):

1
2
3
4
$ ceph-deploy disk zap ceph-node1:/dev/loop1 ceph-node2:/dev/loop1 ceph-node3:/dev/loop1
$ ceph-deploy osd prepare ceph-node1:/dev/loop1 ceph-node2:/dev/loop1 ceph-node3:/dev/loop1
$ ceph-deploy osd activate ceph-node2:/dev/loop1p1:/dev/loop1p2 ceph-node2:/dev/loop1p1:/dev/loop1p2 ceph-node3:/dev/loop1p1:/dev/loop1p2
$ ceph-deploy admin ceph-node1 ceph-node2 ceph-node3

Ceph Status:

Have a look at your cluster status:

1
2
3
4
5
6
7
8
9
10
$ sudo ceph -s
    cluster 8d704c8a-ac19-4454-a89f-89a5d5b7d94d
     health HEALTH_OK
     monmap e1: 3 mons at {ceph-node1=10.0.8.3:6789/0,ceph-node2=10.0.8.4:6789/0,ceph-node3=10.0.8.5:6789/0}
            election epoch 10, quorum 0,1,2 ceph-node2,ceph-node3,ceph-node1
     osdmap e14: 3 osds: 3 up, 3 in
            flags sortbitwise,require_jewel_osds
      pgmap v29: 64 pgs, 1 pools, 0 bytes data, 0 objects
            100 MB used, 18298 MB / 18398 MB avail
                  64 active+clean

Everything looks good. Also change the permissions on this file, on all the nodes in order to execute the ceph, rados commands:

1
$ sudo chmod +r /etc/ceph/ceph.client.admin.keyring

Storage Pools:

List your pool in your Ceph Cluster:

1
2
$ rados lspools
rbd

Let’s create a new storage pool called mypool:

1
2
$ ceph osd pool create mypool 32 32
pool 'mypool' created

Let’s the list the storage pools again:

1
2
3
$ rados lspools
rbd
mypool

You can also use the ceph command to list the pools:

1
2
3
$ ceph osd pool ls
rbd
mypool

Create a Block Device Image:

1
$ rbd create --size 1024 mypool/disk1 --image-feature layering

List the Block Device Images under your Pool:

1
2
$ rbd list mypool
disk1

Retrieve information from your image:

1
2
3
4
5
6
7
8
9
$ rbd info mypool/disk1
rbd image 'disk1':
        size 1024 MB in 256 objects
        order 22 (4096 kB objects)
        block_name_prefix: rbd_data.1021643c9869
        format: 2
        features: layering
        flags:
        create_timestamp: Thu Jun  7 23:48:23 2018

Create a local mapping of the image to a block device:

1
2
$ sudo rbd map mypool/disk1
/dev/rbd0

Now we have a block device available at /dev/rbd0. Go ahead and mount it to /mnt:

1
$ sudo mount /dev/rbd0 /mnt

We can then see it when we list our mounted disk partitions:

1
2
3
4
$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        19G   13G  5.2G  72% /
/dev/rbd0       976M  1.3M  908M   1% /mnt

We can also resize the disk on the fly, let’s resize it from 1GB to 2GB:

1
2
$ rbd resize mypool/disk1 --size 2048
Resizing image: 100% complete...done.

To grow the space we can use resize2fs for ext4 partitions and xfs_growfs for xfs partitions:

1
2
3
4
5
$ sudo resize2fs /dev/rbd0
resize2fs 1.42.13 (17-May-2015)
Filesystem at /dev/rbd0 is mounted on /mnt; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/rbd0 is now 524288 (4k) blocks long.

When we look at our mounted partitions, you will notice that the size of our mounted partition has been increased in size:

1
2
3
4
$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        19G   13G  5.2G   72% /
/dev/rbd0       2.0G  1.5M  1.9G   1% /mnt

Object Storage RadosGW

Let’s create a new pool where we will store our objects:

1
2
$ ceph osd pool create object-pool 32 32
pool 'object-pool' created

We will now create a local file, push the file to our object storage service, then delete our local file, download the file as a file with a different name, and read the contents:

Create the local file:

1
$ echo "ok" > test.txt

Push the local file to our pool in our object storage:

1
$ rados put objects/data/test.txt ./test.txt --pool object-pool

List the pool (note that this can be executed from any node):

1
2
$ $ rados ls --pool object-pool
objects/data/test.txt

Delete the local file, download the file from our object storage and read the contents:

1
2
3
4
5
6
$ rm -rf test.txt

$ rados get objects/data/test.txt ./newfile.txt --pool object-pool

$ cat ./newfile.txt
ok

View the disk space from our storage-pool:

1
2
3
4
5
6
$ rados df --pool object-pool
pool name                 KB      objects       clones     degraded      unfound           rd        rd KB           wr        wr KB
object-pool                1            1            0            0            0            0            0            1            1
  total used          261144           37
  total avail       18579372
  total space       18840516

Resources:

Hello World Programs in Different Languages

This post will demonstrate running hello world programs in different languages and also providing return time statistics

C++

Code

1
2
3
4
5
6
7
8
#include <iostream>
using namespace std;

int main()
{
    std::cout << "Hello, World!" << std::endl;
    return 0;
}

Compile:

1
$ c++ hello_cpp.cpp -o hello_cpp

Run:

1
2
3
4
5
6
$ time ./hello_cpp
Hello, World!

real  0m0.005s
user  0m0.001s
sys     0m0.001s

Golang:

Code

1
2
3
4
5
6
7
package main

import "fmt"

func main() {
  fmt.Println("Hello, World!")
}

Compile:

1
$ go build hello_golang.go

Run:

1
2
3
4
5
6
time ./hello_golang
Hello, World!

real  0m0.006s
user  0m0.001s
sys     0m0.003s

Python

Code:

1
2
#!/usr/bin/env python
print("Hello, World!")

Make it executable:

1
$ chmod +x ./hello_python.py

Run:

1
2
3
4
5
6
$ time ./hello_python.py
Hello, World!

real  0m0.033s
user  0m0.015s
sys     0m0.010s

Ruby

Code:

1
2
#!/usr/bin/env ruby
puts "Hello, World!"

Make it executable:

1
$ chmod +x ./hello_ruby.rb

Run:

1
2
3
4
5
6
$ time ./hello_ruby.rb
Hello, World!

real  0m0.136s
user  0m0.080s
sys     0m0.024s

Java

Code:

1
2
3
4
5
public class hello_java {
    public static void main(String[] args) {
        System.out.println("Hello, World!");
    }
}

Compile:

1
$ javac hello_java.java

Run:

1
2
3
4
5
6
$ time java hello_java
Hello, World!

real  0m0.114s
user  0m0.086s
sys     0m0.023s

Resource:

Setup a Peer to Peer VPN With VPNCloud on Ubuntu

So I got 3 Dedicated Servers each having its own Static IP and I wanted a way to build a private network between these servers.

The Scenario:

3 Servers with the following IP’s (not real IP addresses):

1
2
3
- Server 1: 52.1.99.10
- Server 2: 52.1.84.20
- Server 3: 52.1.49.30

So I want to have a private network, so that I can have the following internal network:

1
2
3
- Server 1: 10.0.1.1
- Server 2: 10.0.1.2
- Server 3: 10.0.1.3

A couple of years ago, I accomplished the end goal using GRE Tunnels, which works well, but wanted to try something different.

VPNCloud

So I stumbled upon VPNCloud.rs, which is a peer to peer VPN. Their description, quoted from their Github page:

“VpnCloud is a simple VPN over UDP. It creates a virtual network interface on the host and forwards all received data via UDP to the destination. VpnCloud establishes a fully-meshed VPN network in a peer-to-peer manner. It can work on TUN devices (IP based) and TAP devices (Ethernet based).”

This is exactly what I was looking for.

Setting up a 3 node Private Network:

Given the IP configuration above, we will setup a Private network between our 3 hosts.

Do some updates then grab the package from Github and install VPNCloud:

1
2
3
$ apt update && apt ugprade -y
$ wget https://github.com/dswd/vpncloud.rs/releases/download/v0.8.1/vpncloud_0.8.1_amd64.deb
$ dpkg -i ./vpncloud_0.8.1_amd64.deb

Let’s start the configuration on Server-1, this config should also be setup on the other 2 servers, the config will remain the same, except for the ifup command. The other servers will look like:

1
2
Server-2: -> ifup: "ifconfig $IFNAME 10.0.1.2/24 mtu 1400"
Server-3: -> ifup: "ifconfig $IFNAME 10.0.1.3/24 mtu 1400"

Getting back to the Server-1 config:

1
$ vim /etc/vpncloud/private.net

Example Config that I am using:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# each vpn running on their own port
port: 3210

# members of our private network
peers:
  - srv2.domain.com:3210
  - srv3.domain.com:3210

# timeouts
peer_timeout: 1800
dst_timeout: 300

# token that identifies the network and helps to distinguish from other networks
magic: "76706e01"

# pre shared key
shared_key: "VeryStrongPreSharedKey_ThatShouldBeChanged"

# encryption
crypto: aes256

# device info
device_name: "vpncloud%d"
device_type: tap

# vpn modes: hub / switch / router / normal
mode: normal

# subnet to be used for our private network
subnets:
  - 10.0.1.0/24

# command to setup the network
ifup: "ifconfig $IFNAME 10.0.1.1/24 mtu 1400"
ifdown: "ifconfig $IFNAME down"

# user/group owning the process
user: "root"
group: "root"

Repeat the config on the other servers.

Start the VPN Service:

Restart the VPNCloud Service on all the Servers:

1
$ service vpncloud@private start

Check the status:

1
$ service vpncloud@private status

Check if the interface is up:

1
2
3
4
5
6
7
8
$ ifconfig vpncloud0
vpncloud0 Link encap:Ethernet  HWaddr aa:bb:cc:dd:ee:ff
          inet addr:10.0.1.1  Bcast:10.0.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1400  Metric:1
          RX packets:55 errors:0 dropped:0 overruns:0 frame:0
          TX packets:71 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:5046 (5.0 KB)  TX bytes:5526 (5.5 KB)

Ping the 3rd server via the private network:

1
2
3
4
5
6
7
8
9
$ ping -c 3 10.0.1.3
PING 10.0.1.2 (10.0.1.3) 56(84) bytes of data.
64 bytes from 10.0.1.3: icmp_seq=1 ttl=64 time=0.852 ms
64 bytes from 10.0.1.3: icmp_seq=2 ttl=64 time=0.831 ms
64 bytes from 10.0.1.3: icmp_seq=3 ttl=64 time=0.800 ms

--- 10.0.1.3 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2028ms
rtt min/avg/max/mdev = 0.800/0.827/0.852/0.039 ms

Awesome service, please check their Github Repo out.

Add a Authentication Header to Your Python Flask App

We will write a simple Python Flask application that requires authentication in order to respond with a 200 HTTP Status code.

Python Flask Application:

Our Python Flask application will require the Header x-api-key dhuejso2dj3d0 in the HTTP Request, to give us a 200 HTTP Status code, if not, we will respond with a 401 Unauthorized Response:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route('/')
def index():
    headers = request.headers
    auth = headers.get("X-Api-Key")
    if auth == 'asoidewfoef':
        return jsonify({"message": "OK: Authorized"}), 200
    else:
        return jsonify({"message": "ERROR: Unauthorized"}), 401

if __name__ == '__main__':
    app.run()

To get the headers, you can use headers.get("X-Api-Key") or headers["X-Api-Key"]

Create a virtual environment, install flask and run the app:

1
2
3
4
5
6
7
8
9
$ virtualenv .venv
$ source .venv/bin/activate
$ python app.py
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Requests to our App:

Let’s first make a request with no headers, which should then give us a 401 Unautorhized response:

1
2
3
4
5
6
7
8
9
$ curl -i http://localhost:5000

HTTP/1.0 401 UNAUTHORIZED
Content-Type: application/json
Content-Length: 33
Server: Werkzeug/0.14.1 Python/3.6.5
Date: Fri, 01 Jun 2018 07:26:25 GMT

{"message":"ERROR: Unauthorized"}

Now let’s include the authentication token in our headers. If the string is the same as the one in the code, we should see a 200 HTTP Response:

1
2
3
4
5
6
7
8
9
$ curl -i -H 'x-api-key: asoidewfoef' http://localhost:5000

HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 29
Server: Werkzeug/0.14.1 Python/3.6.5
Date: Fri, 01 Jun 2018 07:27:03 GMT

{"message":"OK: Authorized"}

Note:

From a best practice, its not a good decision to hard code sensitive details in your code, but rather read that from an encrypted database and store that in your applications environment variables, and let your application read from the environment variables, something like that :D

Clearing Up Disk Space on Docker Swarm by Removing Unused Data With Prune

After some time, your system can run out of disk space when running a lot of containers / volumes etc. You will find that at times, you will have a lot of unused containers, stopped containers, unused images, unused networks that is just sitting there, which consumes data on your nodes.

One way to clean them is by using docker system prune.

Check Docker Disk Space

The command below will show the amount of disk space consumed, and how much is reclaimable:

1
2
3
4
5
6
$ docker system df
TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
Images              229                 125                 23.94GB             14.65GB (61%)
Containers          322                 16                  8.229GB             8.222GB (99%)
Local Volumes       77                  41                  698MB               19.13MB (2%)
Build Cache                                                 0B                  0B

Removing Unsued Data:

By using Prune, we can remove the unused resources that is consuming data:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ docker system prune

WARNING! This will remove:
        - all stopped containers
        - all networks not used by at least one container
        - all dangling images
        - all build cache
Are you sure you want to continue? [y/N] y

Deleted Containers:
a3d7db158e065d0c86160fd5d688875f8b7435848ea91db57ed007
47890dcfea4a105f43e790dd8ad3c6d7c4ad7e738186c034d7a46b

Deleted Networks:
traefik-net
app_appnet

Deleted Images:
deleted: sha256:5b9909c10e93afec
deleted: sha256:d81eesdfihweo3rk

Total reclaimed space: 14.18GB

For related Docker posts.

SSH Tools That Comes in Handy When Dealing With Multiple Servers

When dealing with a lot of servers where you need to ssh to different servers and especially if they require different authentication from different private ssh keys, it kinda gets annoying specifying the private key you need, when you want to SSH to them.

SSH Config

SSH Config: ~/.ssh/config is powerful!

In this config file, you can specify the remote host, the key, user and the alias, so that when you want to SSH to it, you dont have to use the fully qualified domain name or IP address.

Let’s take for example our server-a with the following details:

  • FQDN: host1.eu.compute.domain.coom
  • User: james
  • PrivateKeyFile: /path/to/key.pem
  • Disable Strict Host Checking

So to access that host, you would use the following command (without ssh config):

1
$ ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -i /path/to/key.pem james@host1.eu.compute.domain.com

Now with SSH Config, open up the config file:

1
$ vim ~/.ssh/config

and declare the host details:

1
2
3
4
5
6
Host host1
  Hostname host1.eu.compute.domain.com
  User james
  IdentityFile /path/to/key.pem
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

Now, if we need to SSH to it, we can do it as simply as:

1
$ ssh host1

as it will pull in the configs from the config that is described from the host alias that you calling from the argument of the ssh binary.

SSH Agent

Another handy tool is ssh-agent, if you have password encryption on your key, everytime you need to ssh, a password will be prompted. A way to get around this is to use the ssh-agent.

We also want to set a TTL to the ssh-agent, as we don’t want it to run forever (unless you want it to). In this case I will let the ssh-agent exit after 2 hours. It will also only run in the shell session from where you execute it. Lets start up our ssh-agent:

1
2
$ eval $(ssh-agent -t 7200)
Agent pid 88760

Now add the private key to the ssh-agent. If your private key is password protected, it will prompt you for the password and after successful verification the key will be added:

1
2
$ ssh-add /path/to/key.pem
Identity added: /path/to/key.pem (/path/to/key.pem)

Multiple Github Accounts:

Here is a great post on how to work with different GitHub Accounts: - https://gist.github.com/jexchan/2351996

Wildcard SSL Certificate With Letsencrypt on Docker Swarm Using Traefik

With Letsencrypt supporting Wildcard certificates is really awesome. Now, we can setup traefik to listen on 443, acting as a reverse proxy and is doing HTTPS Termination to our Applications thats running in our Swarm.

Architectural Design:

At the moment we have 3 Manager Nodes, and 5 Worker Nodes:

  • Using a Dummy Domain example.com which is set to the 3 Public IP’s of our Manager Nodes
  • DNS is set for: example.com A Record to: 52.10.1.10, 52.10.1.11, 52.10.1.12
  • DNS is set for: *.example.com CNAME to example.com
  • Any application that is spawned into our Swarm, will be labeled with a traefik.frontend.rule which will be routed to the service and redirected from HTTP to HTTPS

Create the Overlay Network:

Create the overlay network that will be used for our stack:

1
$ docker network create --driver overlay appnet

Create the Compose Files for our Stacks:

Create the Traefik Service Compose file, we will deploy it in Global Mode, constraint to our Manager Nodes, so that every manager node has a copy of traefik running.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
$ cat > traefik-compose.yml << EOF

version: "3.4"
services:
  proxy:
    image: traefik:latest
    command:
      - "--api"
      - "--entrypoints=Name:http Address::80 Redirect.EntryPoint:https"
      - "--entrypoints=Name:https Address::443 TLS"
      - "--defaultentrypoints=http,https"
      - "--acme"
      - "--acme.storage=/etc/traefik/acme/acme.json"
      - "--acme.entryPoint=https"
      - "--acme.httpChallenge.entryPoint=http"
      - "--acme.onHostRule=true"
      - "--acme.onDemand=false"
      - "--acme.email=me@example.com"
      - "--docker"
      - "--docker.swarmMode"
      - "--docker.domain=example.com"
      - "--docker.watch"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /mnt/traefik/acme.json:/etc/traefik/acme/acme.json
    networks:
      - appnet
    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host
      - target: 8080
        published: 8080
        mode: host
    deploy:
      mode: global
      placement:
        constraints:
          - node.role == manager
      update_config:
        parallelism: 1
        delay: 10s
      restart_policy:
        condition: on-failure
networks:
  appnet:
    external: true

EOF

Create the Application Compose file, in this example we will be deploying a Ghost Blog:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
$ cat > ghost-compose.yml << EOF

version: '3.4'

services:
  blog:
    image: ghost:1.22.7-alpine
    networks:
      - appnet
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints: 
          - node.role == worker
      labels:
        - "traefik.backend.loadbalancer.sticky=false"
        - "traefik.backend.loadbalancer.swarm=true"
        - "traefik.backend=blog-1"
        - "traefik.docker.network=appnet"
        - "traefik.entrypoints=https"
        - "traefik.frontend.passHostHeader=true"
        - "traefik.frontend.rule=Host:blog.example.com"
        - "traefik.port=2368"

networks:
  appnet:
    external: true

EOF

Prepare the Path for Traefik:

We have a replicated volume under our /mnt partition, so that all our managers can read from that path, create the file and provide the sufficient permissions:

1
2
3
$ mkdir -p /mnt/traefik
$ touch /mnt/traefik/acme.json
$ chmod 600 /mnt/traefik/acme.json

Deploy the Stacks:

Deploy the Traefik Stack:

1
$ docker stack deploy -c traefik-compose.yml traefik

Wait until the services are deployed:

1
2
3
$ docker stack services traefik
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
f8ru5gbcgd2v        traefik_proxy       global              3/3                 traefik:latest

Deploy the Application Stack:

1
$ docker stack deploy -c ghost-compose.yml apps

Verify that the Application Stack has been deployed:

1
2
3
$ docker stack services apps
ID                  NAME                MODE                REPLICAS            IMAGE                          PORTS
516zlfs2cfdv        apps_blog           replicated          1/1                 ghost:1.22.7-alpine

At the moment we will have 2 stacks in our Swarm:

1
2
3
4
$ docker stack ls
NAME                SERVICES
apps                1
traefik             1

Test the Application:

Let’s test our blog to see if we get redirected to HTTPS:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ curl -iL http://blog.example.com
HTTP/1.1 302 Found
Location: https://blog.example.com:443/
Date: Mon, 28 May 2018 22:02:41 GMT
Content-Length: 5
Content-Type: text/plain; charset=utf-8

HTTP/1.1 200 OK
Cache-Control: public, max-age=0
Content-Type: text/html; charset=utf-8
Date: Mon, 28 May 2018 22:02:42 GMT
Etag: W/"4166-J2ooSIa8gtTkYjbnr7vnPUFlRJI"
Vary: Accept-Encoding
X-Powered-By: Express
Transfer-Encoding: chunked

Works like a charm! Traefik FTW!

Web Forms With Python Flask and the WTForms Module With Bootstrap

Quick demo with Web Forms using the WTForms module in Python Flask.

Requirements:

Install the required dependencies:

1
$ pip install flask wtforms

Application:

The Application code of the Web Forms Application. Note that we are also using validation, as we want the user to complete all the fields. I am also including a function that logs to the directory where the application is running, for previewing the data that was logged.

app.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
from random import randint
from time import strftime
from flask import Flask, render_template, flash, request
from wtforms import Form, TextField, TextAreaField, validators, StringField, SubmitField

DEBUG = True
app = Flask(__name__)
app.config.from_object(__name__)
app.config['SECRET_KEY'] = 'SjdnUends821Jsdlkvxh391ksdODnejdDw'

class ReusableForm(Form):
    name = TextField('Name:', validators=[validators.required()])
    surname = TextField('Surname:', validators=[validators.required()])

def get_time():
    time = strftime("%Y-%m-%dT%H:%M")
    return time

def write_to_disk(name, surname, email):
    data = open('file.log', 'a')
    timestamp = get_time()
    data.write('DateStamp={}, Name={}, Surname={}, Email={} \n'.format(timestamp, name, surname, email))
    data.close()

@app.route("/", methods=['GET', 'POST'])
def hello():
    form = ReusableForm(request.form)

    #print(form.errors)
    if request.method == 'POST':
        name=request.form['name']
        surname=request.form['surname']
  email=request.form['email']
  password=request.form['password']

        if form.validate():
            write_to_disk(name, surname, email)
            flash('Hello: {} {}'.format(name, surname))

        else:
            flash('Error: All Fields are Required')

    return render_template('index.html', form=form)

if __name__ == "__main__":
    app.run()

HTML Template:

templates/index.html

This will result in a basic web form like this:

Resources:

Generate Random Characters With Python Using Random and String Modules

When generating random characters for whatever reason, passwords, secrets-keys etc, you could use the uuid module, which looks like this:

Random String with UUID
1
2
3
>>> from uuid import uuid4
>>> print("Your string is: {0}".format(uuid4()) )
Your string is: 53a6e1a7-a2c7-488e-bed9-d76662de9c5f

But if you want to be more specific, like digits, letters, capitalization etc, you can use the string and random modules to do so. First we will generate a random string containing only letters:

Random String with letters
1
2
3
4
5
6
7
8
9
>>> from string import ascii_letters, punctuation, digits
>>> from random import choice, randint
>>> min = 12
>>> max = 15
>>> string_format = ascii_letters
>>> generated_string = "".join(choice(string_format) for x in range(randint(min, max)))

>>> print("Your String is: {0}".format(generated_string))
Your String is: zNeUFluvZwED

As you can see, you have a randomized string which will be always at least 12 characters and max 15 characters, which is lower and upper case. You can also use the lower and upper functions if you want to capitalize or lower case your string:

1
2
3
4
5
>>> generated_string.lower()
'zneufluvzwed'

>>> generated_string.upper()
'ZNEUFLUVZWED'

Let’s add some logic so that we can have a more randomized characters with digits, punctuations etc:

Random String with Letters, Punctuations and Digits
1
2
3
4
5
6
7
8
>>> from string import ascii_letters, punctuation, digits
>>> from random import choice, randint
>>> min = 12
>>> max = 15
>>> string_format = ascii_letters + punctuation + digits
>>> generated_string = "".join(choice(string_format) for x in range(randint(min, max)))
>>> print("Your String is: {0}".format(generated_string))
Your String is: Bu>}x_/-H5)fLAr

More Python related blog posts.