In this post we will use Grafana Promtail to collect all our logs and ship it to Grafana Loki.
About
We will be using Docker Compose and mount the docker socket to Grafana Promtail so that it is aware of all the docker events and configure it that only containers with docker labels logging=promtail
needs to be enabled for logging, which will then scrape those logs and send it to Grafana Loki where we will visualize it in Grafana.
Promtail
In our promtail configuration config/promtail.yaml
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
|
You can see we are using the docker_sd_configs
provider and filter only docker containers with the docker labels logging=promtail
and once we have those logs we relabel our labels to have the container name and we also use docker labels like log_stream
and logging_jobname
to add labels to our logs.
Grafana Config
We would like to auto configure our datasources for Grafana and in config/grafana-datasources.yml
we have:
1 2 3 4 5 6 7 8 9 10 |
|
Docker Compose
Then lastly we have our docker-compose.yml
that wire up all our containers:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
|
As you can see with our nginx container we define our labels:
1 2 3 4 5 6 |
|
Which uses logging: "promtail"
to let promtail know this log container’s log to be scraped and logging_jobname: "containerlogs"
which will assign containerlogs to the job label.
Start the stack
If you are following along all this configuration is available in my github repository https://github.com/ruanbekker/docker-promtail-loki .
Once you have everything in place you can start it with:
1
|
|
Access nginx on http://localhost:8080
Then navigate to grafana on http://localhost:3000 and select explore on the left and select the container:
And you will see the logs:
Thank You
Thanks for reading, feel free to check out my website, feel free to subscribe to my newsletter or follow me at @ruanbekker on Twitter.
- Linktree: https://go.ruan.dev/links
- Patreon: https://go.ruan.dev/patreon