Can i "Filter" certain logs into a Special Index?

Elastic Stack Elasticsearch
1

Hey guys,
since i still pretty new to this topic, i wanted to know if the following would be Possible:

I've got a costumer who has different retention periods for different logs.
In this case, the logs concerning the internet searches have to be stored 12 Months instead of 3.

So as i know, i can manage it via the rollover-policies, but the thing is that the Index (Logstash) which holds the data about internet searches, is the same as the verbose output of the virtual cluster. Can i somehow configure it that, that the logs concerning the webfilter will be handled in a different index so i can create a rollover-policy over 12 Months?

Sorry if that's a dumb question. Not my topic usually.

Kind regards,
Moritz Kiesewetter

2

Hey Moritz,

in general you can index into any index, it's merely a question what endpoint you configure for your ingestion layer. If you have different ingestion components, just change the index for that component/customer. If the ingestion component is the same, you could do two things. First specify the index on a per document level. If that does not work due to whatever reason, you could probably just reindex that data into an own index, before deleting the index containing all the customer data. See https://www.elastic.co/guide/en/elasticsearch/reference/7.4/docs-reindex.html

hope this helps as a starter.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.