Setup Prometheus and Node Exporter on Ubuntu for Epic Monitoring
Prometheus is one of those awesome open source monitoring services that I simply cannot live without. Prometheus is a Time Series Database that collects metrics from services using it’s exporters functionality. Prometheus has its own query language called PromQL and makes graphing epic visualiztions with services such as Grafana a breeze.
What are we doing today
We will install the prometheus service and set up node_exporter to consume node related metrics such as cpu, memory, io etc that will be scraped by the exporter configuration on prometheus, which then gets pushed into prometheus’s time series database. Which can then be used by services such as Grafana to visualize the data.
Other exporters is also available, such as: haproxy_exporter, blackbox_exporter etc, then you also get pushgateway which is used to push data to, and then your exporter configuration scrapes the data from the pushgateway endpoint. In a later tutorial, we will set up push gateway as well.
First, let’s provision our dedicated system users for prometheus and node exporter:
Let’s look at the status to see if everything works as expected:
$ systemctl status prometheus
prometheus.service - Prometheus
Loaded: loaded (/etc/systemd/system/prometheus.service; disabled; vendor preset: enabled)
Active: active (running) since Tue 2019-03-26 11:59:10 UTC; 6s ago
Main PID: 16374 (prometheus)
Tasks: 9 (limit: 4704)
└─16374 /usr/local/bin/prometheus --config.file /etc/prometheus/prometheus.yml --storage.tsdb.path /var/lib/prometheus/ --web.console.templates=/etc/prometheus/consoles --web.console.libraries=
Mar 26 11:59:10 ip-172-31-41-126 prometheus: level=info ts=2019-03-26T11:59:10.893770598Z caller=main.go:655 msg="TSDB started"
Seems legit! Enable the service on startup:
$ systemctl enable prometheus
Install Node Exporter
Now since we have prometheus up and running, we can start adding exporters to publish data into our prometheus time series database. As mentioned before, with node exporter, we will allow prometheus to scrape the node exporter endpoint to consume metrics about the node:
You will find the latest version from their website, which I have added at the top of this post.
Once the config is saved, restart prometheus and have a look at the status if everything is going as expected:
$ systemctl restart prometheus
$ systemctl status prometheus
Nginx Reverse Proxy
Let’s add a layer of security and front our setup with a nginx reverse proxy, so that we don’t have to access prometheus on high ports and we have the option to enable basic http authentication. Install nginx:
$ apt install nginx apache2-utils -y
Create the authentication file:
$ htpasswd -c /etc/nginx/.htpasswd admin
Create the nginx site configuration, this will tel nginx to route connections on port 80, to reverse proxy to localhost, port 9090, if authenticated:
$ rm /etc/nginx/sites-enabled/default
$ vim /etc/nginx/sites-enabled/prometheus.conf