Newbie question - need suggestion - NFS share + 1 node

Elastic Stack Elasticsearch
1

Good morning community,
i am a newnubbie to ES and i would like a design suggestion from you.
We need to index 5 years old logs taken from a SIEM from a subset of applications, we have one server (system spec should be defined) and an attached NFS NAS storage of 5 TB . The logs size to be ‘uploaded’ to ES once a day (we though to retrieve and upload the data via API ) each day are few, approx 400 MB per day; so basically 712GB per 5 years. I have the following questions, i hope you can help me understand better and give me some advice:
1- It is feasable to use the 5TB NFS as a index data store nonetheless the few data required to be indexed? As far as i understood NFS datastore are a nono in ES.
2- All the log data from 5 years back should be searchable and indexed; an index of almost 1TB is feasable? Or it is adviced to define an index for each log source and/or for each day/week/month? What are your suggestion about index and shards to be defined?
3- With the data required to be uploaded and indexed, what server (1 node) specs do you suggest? 8cpu and 32GB RAM?
4- If the NFS datastore is not an option how can i use it? Store Index snapshots maybe?

I hope my questions are clear enough and that you can help me on my doubts.
Thanks,

2

NFS is not recommended as it can cause performance problems and I believe potentially also corruption (could not find reference now though).

Use time based indices, e.g. an index per month.

Sounds like a reasonable starting point.

Yes.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.