Options for analytics on ES

Greetings,

We have tons of data coming into our ES from devices across the whole US. We need to do a lot of analytical research on this data. We do not want to do this on the live data but would like to do it on copies of it. What would be the best approach of getting this data into another node (research node) separate from the production node on a daily basis. I have already tried a search option as follows

result = elastic_client.search(index="slushies_*", body=search_param)
    response_hits  = result["hits"]["hits"]

and then looping through the result set. This is not scalable and it takes a long time to go through a days worth of data.

Is there other recommended options, such as restoring backups, or some other sort of snapshots?

Thanks!

Perhaps

1 Like

thanks very much @stephenb I will look into this asap

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.