Today I was using Elasticsearch Curator to reindex indices that was created on a daily basis, to reindex all the data to one index. I used this route as the old data will not be accessed frequently.
Install Elasticsearch Curator
12
$ docker run -it python:2.7-alpine sh
$ pip install elasticsearch-curator
Create Configs:
Create the curator config:
config.yml
123456789101112131415161718
---# Remember, leave a key empty if there is no value. None will be a string,# not a Python "NoneType"client:hosts:-es.endpoint.comport:443use_ssl:Truessl_no_validate:Falsehttp_auth:admin:passtimeout:30master_only:Falselogging:loglevel:INFOlogfile:logformat:defaultblacklist:['urllib3']
Create the action config:
action-reindex.yml
12345678910111213141516171819202122
---# Remember, leave a key empty if there is no value. None will be a string,# not a Python "NoneType"## Also remember that all examples have 'disable_action' set to True. If you# want to use this action as a template, be sure to set this to False after# copying it.actions:1:description:"Reindexindex-2017.10.{30,31}intonew-index-2017.10"action:reindexoptions:disable_action:Falsewait_interval:9max_wait:-1request_body:source:index:['index-2017.10.*']dest:index:new-index-2017.10filters:-filtertype:none
Create the Elasticsearch Index:
Create the Index where we will reindex the data to:
$ curator --config curator.yml action-reindex.yml
2017-11-22 14:18:15,138 INFO Task "reindex from [index-2017.10.*] to [index-2017.10]" with task_id "Za-sn0z3Q9-75xCMRwJ3-A:15782886" has been running for 928.948195354 seconds
2017-11-22 14:18:24,152 INFO GET https://es.endpoint.com:443/_tasks/Za-sn0z3Q9-75xCMRwJ3-A%3A15782886 [status:200 request:0.005s]2017-11-22 14:18:24,153 INFO Task "reindex from [index-2017.10.*] to [new-index-2017.10]" with task_id "Za-sn0z3Q9-75xCMRwJ3-A:15782886" has been running for 937.962740393 seconds
2017-11-22 14:22:23,171 INFO Action ID: 1, "reindex" completed.
2017-11-22 14:22:23,171 INFO Job completed.