I am new to ELK [ Elasticsearch Logstash and Kibana].
I have three machines having 30 GB RAM and around 400 GB space
I have millions of file to parse [size is around 300 GB]
I have split the files and placed in three systems.
I installed Elasticsearch Logstash and Kibana in one machine . Then installed Logstash in two other machines.
I redirected the output of logstash in each machine to one elasticsearch machine
It took 6 days to filter out the searchd item[I searched for 10 digit number, timestamp and Name of one API from all these logs]and dispay it in Kibana. Am i doing something wrong here. Is there any other way to speed up/tune this process.
Thanks in Advance, Paul