The other section will be used as a demo, with solving a problem with Amazon CloudWatch Logs. I’m a Massive AWS Fanatic, but when it comes to CloudWatch Logs, I’m not so big of a fan of that specific service. Especially when you use Docker Swarm for AWS and have your logdriver set to CloudWatch Logs.
The Problem I have with CloudWatch Logs
When you point to your CloudWatch LogGroups, you can search for your streams, and in my case searching for a specific swarm service, but you can’t sort by date, like this:
This makes it really tedious when trying to search find your logs in a quick way.
Python Flask to the Resque
We will create a Python Flask application that retrieves your data about all your Docker Swarm Services and Container Id’s running on each node. For this demonstration, I have hard coded the services and container id’s, but using it in a real environment, you can utilise the Docker API or some logic that retrieves it from a datastore where a process populates it to.
The Application Code will do the following:
- returns a list of your swarm services (mock data in the code)
- when you select a service, it will get a list of the container ids and run through a for loop unsing jinja templates and display them in table format
- when you select the containerId, it will populate the containerId to the cloudwatch logs filter, giving you the exact logstream which you are looking for
this will do a redirect to the AWS Console, and you will see the data in the sorted time of interest
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
Filtering the Data
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Once you search for a specific keyword on the service you are looking for the output should more or less look like the following: