Flush data periodically

Elastic Stack Elasticsearch
1

Hi All,

Please suggest how to configure data flush in elasticsearch periodically.

Regards,
Prateek

2

You can set the refresh interval per index using the APIs.

3

Hey Christian,

Thank you for the update.
As I understand from the link refresh interval could be defined for bulk indexing dynamically but my problem statement is -

Problem Statement- Update the current index with new data for fixed time sample(Remove the previous time data from index ).
Example- Update the last 7 days event log to index only, remove all data earlier than the define time.

Regards,
Prateek

4

Have you considered using time-based indices?

5

Is it possible to schedule the steps to add and remove aliases from older index to new ,so that it remove the dependency of updating manually on a day basis.

Regards,
Prateek

6

Can you please explain your use case/problem in more detail? How are you using aliases?

7

I am trying to replicate below-

POST /_aliases
{
"actions": [
{ "add": { "alias": "logs_current", "index": "logs_2014-10-01" }},
{ "remove": { "alias": "logs_current", "index": "logs_2014-10-02" }},

]
}
The logs would be pushed to elasticsearch on a day basis to date based index and aliases would be changed(directed to current date index) to visualize the data on dashboard.

Regards,
Prateek

8

There is generally no need to create this type of alias when working with Kibana, as it is able to determine which indices that may contain relevant data automatically and only query those indices if you define the index pattern as logs_*. What is it you are looking to achieve using this alias?

9

I was trying to use alias to fetch the data from different date.
Creating index on date basis and defining index pattern as logs_* helped to resolve the problem statement.

Thank you for help.

Regards,
Prateek