Need Help in understanding how to convert Datastreams to Regular Index

Elastic Stack Elasticsearch
1

Hi all,
So we are using EFK stack for logging purposes. We have ES at 7.17.23 and filebeat at 7 version. Unfortunately on some of the machines, the filebeat got upgraded to 8 , it forcefully modified the Index template that is connected to the filebeat to have Data Streams enabled. So every day the index is getting created is not regular index but rather its a datastream. I tried forcefully changing back the index template without successfully.
We have a cron job to trigger the snapshots on the regular indices, So we used to that to create the snapshots on the indices created by these data streams but when we tried to restore it back to regular index from the snapshot. I have been unsuccessful. Any suggestions or directions I can follow, I would highly appreciate it.

Thanks,
Kushal

2

Hi @kushalOtter,

I found this older StackOverflow post, which seems to suggest deleting the data stream first and the corresponding index template by running Elasticsearch/Kibana connection on a different port.

3

Not your fix, but this is an example of why you never give your ingester users credentials above just sending data :slight_smile:

1 Like
4

Thanks @jessgarson . I looked at that solution, That will not work, given we have months of data logs and can't delete them. However, we decided it best to snapshot the data Streams for now.

@rugenl Absolutely agree with you. Trying to find the correct balance with managing overhead and productivity at a small company.

Thanks for the replies.

1 Like
5

Thanks for sharing, @kushalOtter. That does make sense, and a snapshot of the data streams seems like a good next step.