This tutorial will guide you how to use the Bulk API with Elasticsearch, this is great for when having a dataset that contains a lot of documents, where you want to insert them into elasticsearch in bulk uploads.
The Dataset
We will be using a dataset from elastic that contains 1000 documents that holds account data.
Getting the Dataset:
1
|
|
Preview the data:
1 2 3 4 5 6 7 8 9 10 11 |
|
Using the Bulk API:
We will ingest the data to our bank_accounts
index, and to the account
type:
1
|
|
When it’s done, have a look at the indices:
1 2 3 |
|
Doing a search and display one document:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
|
Demo Recording:
This has also been reccored, which can be viewed here:
Using Bulk with Auto Generated ID’s
As you might know when you do a POST request to the type, the _id
field gets auto populated. Timo, one of my friends had the requirement to use the Bulk API to post auto generated Id’s and not the static id’s that is given in the example dataset.
I have answered this on Elastic’s discuss page: https://discuss.elastic.co/t/looking-for-working-example-data-set-to-bulk-index-into-es6/128678/3
I will provide the steps below as well:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
The original file:
1 2 3 4 5 |
|
Removing the initial metadata:
1 2 |
|
Previewing the destination file:
1 2 3 4 5 |
|
Looking at my current indices:
1 2 3 |
|
Ingesting the data via Bulk API:
1
|
|
Looking at my indices to verify that the index exist:
1 2 3 4 |
|
Looking at one document: :smiley:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
|