In this post we will setup the Elastic Stack with Elasticsearc, Kibana and APM . The APM Server (Application Performance Metrics) which will receive the metric data from the application side, and is then pushed to apm indices on Elasticsearch.
This will be a 2 post blog on APM:
- 1) Setup the Elastic Stack with Elasticsearch, Kibana and APM Server - this post
- 2) Setup a Python Flask application with the APM Agent
What is APM
From their website APM is described as: “Elastic APM is an application performance monitoring system built on the Elastic Stack. It allows you to monitor software services and applications in real time, collecting detailed performance information on response time for incoming requests, database queries, calls to caches, external HTTP requests, etc.”
You get metrics like average, p99 response times etc, and also have insights when errors occur, it even allows you to look at the stacktrace, poinpointing on which line of your code it ocurred etc.
APM Agents:
The APM Agents will be loaded inside your application, application metrics will then be pushed to the APM Server (which we will setup in this post), which then gets pushed to Elasticsearch and is then consumed by Kibana.
At the time of writing, the APM Agents are supported in the following languages:
- Node.js
- Django
- Flask
- Ruby on Rails
- Rack
- RUM
- Golang
- Java
Setup the Elastic Stack
One thing to note, every service in your Elastic Stack needs to be running on the same version. In this post we will setup Elasticsearch, APM and Kibana all running on version 6.4.3
Setup the Pre-Requirements:
Elasticsearch depends on Java, se we will go ahead and setup the repositories:
1 2 3 4 5 |
|
Verify that Java is installed:
1 2 3 4 |
|
Setup Kernel parameters for Elasticsearch:
1 2 |
|
Setup Elasticsearch:
Search for the latest versions (when already having elasticsearch, either upgrade or install apm on the same version as elasticsearch/kibana):
1 2 3 |
|
Install Elasticsearch:
1
|
|
Configure Elasticsearch to lock the memory on startup:
1
|
|
Enable Elasticsearch on startup and start the service:
1 2 3 |
|
Install Kibana:
Install Kibana version 6.4.3
:
1
|
|
For demonstration, I will configure Kibana to listen on all interfaces on port 5601
, but note this will enable access for everyone, you can [follow this blogpost] to setup a Nginx Reverse Proxy to upstream to localhost on port 5601.
Since this demonstration we are using Elasticsearch locally, so if you have a remote cluster, configuration can be applied where needed.
1 2 |
|
Enable Kibana on startup and start the service:
1 2 |
|
Install the APM Server
Install APM Server version 6.4.3
:
1
|
|
Since we have everything locally, the configuration can be kept as is, but if you need to configure the elasticsearch or kibana hosts, it can be done via /etc/apm-server/apm-server.yml
Then once Kibana and Elasticsearch is started, load the mapping templates, enable and start the service:
1 2 3 |
|
Ensure all the services are running with netstat -tulpn
and port 9200
, 9300
, 5601
and 8300
should be listening
Access Your Elastic Stack
Access Kibana on your routable endpoint on port 5601
and you should see something like this:
Configuring a Application to push metrics to APM
In the next post I will setup a Python Flask Application on APM