As many of you might know, when you deploy a ELK stack on Amazon Web Services, you only get E and K in the ELK stack, which is Elasticsearch and Kibana. Here we will be dealing with Logstash on EC2.
What will we be doing
In this tutorial we will setup a Logstash Server on EC2, setup a IAM Role and Autenticate Requests to Elasticsearch with an IAM Role, setup Nginx so that logstash can ship logs to Elasticsearch.
I am not fond of working with access key’s and secret keys, and if I can stay away from handling secret information the better. So instead of creating a access key and secret key for logstash, we will instead create a IAM Policy that will allow the actions to Elasticsearch, associate that policy to an IAM Role, set EC2 as a trusted entity and strap that IAM Role to the EC2 Instance.
Then we will allow the IAM Role ARN to the Elasticsearch Policy, then when Logstash makes requests against Elasticsearch, it will use the IAM Role to assume temporary credentials to authenticate. That way we don’t have to deal with keys. But I mean you can create access keys if that is your preferred method, I’m just not a big fan of keeping secret keys.
The benefit of authenticating with IAM, allows you to remove a reverse proxy that is another hop to the path of your target.
Create the IAM Policy:
Create a IAM Policy that will allow actions to Elasticsearch:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Create Role logstash-system-es with “ec2.amazonaws.com” as trusted entity in trust the relationship and associate the above policy to the role.
Authorize your Role in Elasticsearch Policy
Head over to your Elasticsearch Domain and configure your Elasticsearch Policy to include your IAM Role to grant requests to your Domain:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Install Logstash on EC2
I will be using Ubuntu Server 18. Update the repositories and install dependencies:
1 2 3 4 5
As logstash requires Java, install the the Java OpenJDK Runtime Environment:
Verify that Java is installed:
1 2 3 4
Now, install logstash and enable the service on boot:
1 2 3
Install the Amazon ES Logstash Output Plugin
For us to be able to authenticate using IAM, we should use the Amazon-ES Logstash Output Plugin. Update and install the plugin:
I like to split up my configuration in 3 parts, (input, filter, output).
Let’s create the input configuration:
1 2 3 4 5 6
Our filter configuration:
1 2 3 4 5 6 7 8 9 10
And lastly, our output configuration:
1 2 3 4 5 6 7 8 9
Note that the
aws_ directives has been left empty as that seems to be the way it needs to be set when using roles. Authentication will be assumed via the Role which is associated to the EC2 Instance.
If you are using access keys, you can populate them there.
Tail the logs to see if logstash starts up correctly, it should look more or less like this:
1 2 3 4 5 6
As you noticed, I have specified
/var/log/nginx/access.log as my input file for logstash, as we will test logstash by shipping nginx access logs to Elasticsearch Service.
Start the service:
Make a GET request on your Nginx Web Server and inspect the log on Kibana, where it should look like this: