Ruan Bekker's Blog

From a Curious mind to Posts on Github

Modern Reverse Proxy With Traefik on Docker Swarm

Traefik is a modern load balancer and reverse proxy built for micro services.

We will build 4 WebServices with Traefik, where we will go through the following scenarios:

  • Hostname Based Routingi (With Path’s and Without)
  • Path Based Routing

Pre-Requisites:

From your DNS Provider add wildcard entries to the Docker Swarm Public IPs:

  • apps.domain.com -> A Record to each Docker Swarm Node
  • *.apps.domain.com => apps.doamin.com

This will allow us to create web applications on the fly.

Static Website with Traefik:

Create Traefik Proxy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
docker service create \
--name traefik \
--constraint 'node.role==manager' \
--publish 80:80 \
--publish 443:443 \
--publish 8080:8080 \
--mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock \
--network appnet \
traefik:camembert \
--docker --docker.swarmmode  \
--docker.domain=apps.domain.com \
--docker.watch \
--logLevel=DEBUG \
--web

Build a WebService with 2 Endpoints:

Our Website will have:

  • /
  • /test/

Our Dockerfile:

1
2
3
4
5
6
7
8
FROM alpine:edge

RUN apk update \
    && apk add lighttpd

ADD htdocs /var/www/localhost/htdocs

CMD ["lighttpd", "-D", "-f", "/etc/lighttpd/lighttpd.conf"]

Our htdocs directory:

1
2
3
4
5
find ./htdocs/
./htdocs/
./htdocs/index.html
./htdocs/test
./htdocs/test/index.html

Building and Push the Image to a Registry of your choice:

1
2
3
docker login registry.gitlab.com
docker build -t registry.gitlab.com/<user>/<repo>/lighttpd:test
docker push registry.gitlab.com/<user>/<repo>/lighttpd:test

Create the 1st Service, No Hostname or Path based specified:

The Service will allow us to view / and /test/ paths, and also enable us to use the service name as the subdomain, or the domain specified in the traefik service:

1
docker service create --name web1 --label 'traefik.port=80'  --network appnet --with-registry-auth registry.gitlab.com/<user>/<repo>/lighttpd:test

Testing the service:

1
2
3
4
$ curl http://web1.apps.domain.com/
<html>
Root Page
</html>
1
2
3
4
$ curl http://web2.apps.domain.com/test/
<html>
Test Page
</html>

and

1
2
3
4
$ curl http://apps.domain.com/test/
<html>
Test Page
</html>

but

1
2
$ curl http://foo.apps.domain.com/test/
404 page not found

Create the 2nd Service, Only 1 Path Based Routing:

This service will only allow us to view the /test/ endpoint:

1
$ docker service create --name website2 --label 'traefik.port=80' --label traefik.frontend.rule="Path: /test/" --network appnet --with-registry-auth registry.gitlab.com/<user>/<repo>/lighttpd:test

Testing the Service:

1
2
$ curl http://web1.apps.domain.com/
404 page not found
1
2
3
4
$ curl http://web2.apps.domain.com/test/
<html>
Test Page
</html>

Hostname Based and Path Based Routing:

1
2
3
4
5
6
$ docker service create \
--name web3 \
--label 'traefik.port=80' \
--label traefik.frontend.rule="Host:apps.domain.com; Path: /test/" \
--network appnet \
--with-registry-auth registry.gitlab.com/rbekker87/docker/lighttpd:u1t-test

Test the / endpoint, which should not work:

1
2
$ curl  http://apps.domain.com/
404 page not found

and the /test/ endpoint:

1
2
3
4
$ curl  http://apps.domain.com/test/
<html>
Test Page
</html>

Also, any other FQDN that is specified will not work as it does not match the traefik.frontend.rule:

1
2
$ curl  http://web3.apps.domain.com/
404 page not found

Strictly Hostname Based Routing and not specifying any paths:

1
2
3
4
5
6
$ docker service create \
--name web4 \
--label 'traefik.port=80' \
--label traefik.frontend.rule="Host:apps.domain.com" \
--network appnet \
--with-registry-auth registry.gitlab.com/<user>/<repo>/lighttpd:u1t-test

Testing the Service:

1
2
3
4
$ curl http://apps.domain.com/
<html>
Root Page
</html>
1
2
3
4
$ curl http://apps.domain.com/test/
<html>
Test Page
</html>

Anything specified other than that, will result in a 404 Response.

Create a ZFS Raidz1 Volume Pool on Ubuntu 16

Setting up ZFS Volume Pool on Ubuntu 16.04

Installation

1
$ sudo apt-get install zfsutils-linux -y

Creating the ZFS Storage Pool

We will create a RAIDZ(1) Volume which is like Raid5 with Single Parity, so we can lose one of the Physical Disks before Raid failure.

Let’s first have a look at our disks that we have on our server:

1
2
3
4
5
6
$ lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0   8G  0 disk
└─xvda1 202:1    0   8G  0 part /
xvdf    202:80   0 100G  0 disk 
xvdg    202:80   0 100G  0 disk 

So we will be creating the volume consisting of /dev/xvdf and /dev/xvdg and we will name our pool: storage-pool

1
$ zpool create storage-pool raidz1 xvdf xvdg -f

Listing Pools

1
2
3
$ zpool list
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
storage-pool   199G   125K   199G         -     0%     0%  1.00x  ONLINE  -

We can also list the volume with zfs:

1
2
3
$ zfs list
NAME            USED  AVAIL  REFER  MOUNTPOINT
storage-pool    125K  199G   19K    /storage-pool

Mounting the Volume:

You will find that the volume is already mounted:

1
2
3
4
$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      7.7G  1.1G  6.7G  14% /
pool            199G  125K  198G   1% /pool

Resources:

See how Brett Kelly from 45 Drives tried to break a Storage Cluster with GlusterFS and ZFS:

Great ZFS Performance Comparison:

Setup MongoDB Client on CentOS 6

I have a bastion host that is still running CentOS6 and epel repos provides mongodb-shell version 2.x and Mlab requires version 3.x

Setup the Repositories

Create the repository:

1
2
3
4
5
6
7
8
$ cat > /etc/yum.repos.d/mongodb.repo << EOF
[mongodb-org-3.4]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/\$releasever/mongodb-org/3.4/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-3.4.asc
EOF

Update the repository index:

1
$ sudo yum update -y

Install MongoDB-Shell

Install the MongoDB Shell Client:

1
$ sudo yum install mongodb-shell -y

Update: Thanks to Rick, when you use CentOS 7, you can install the Shell Client as instructed below:

1
$ sudo yum install mongodb-org-shell -y

Connect to your Remote MongoDB Instance:

1
$ mongo remotedb.mlab.com:27017/<dbname> -u <user> -p <pass>