I have been running my ELK SIEM for a few months now, almost a year actually and my syslogs stopped being shown. I have them coming in again now, but I need to clean up my indices so that it does not stop again. Is there a good description and documentation that I can review to become more aquainted with the difference between deleteting, flushing, and merging indices? I don't want to loose data but I also cannot have the syslogs stop coming in as well.
Thank you for your help and guidance. It is always much appreciated.
Thank you for the response Christian. I will get that information for you on Friday. At this time I do not have remote access to the Elastic Stack SIEM.
I do have one other question that relates to this topic.
Is there a or can the Logstash server send the logstash logs to a folder that is not located on the Logstash server? Ie adding in the Logstash.yml file for the location of the Logfiles \192.168.50.15\C$\Logstash_logs\ Where the ip indicated is not the IP of the Logstash server? We have a log server that has terrabytes of space that we want to send the logs to, which will ensure that it never stops logging.
The API output is very large and I am unable to pull it out of the environment. Is there a specific area to look in the output?
If I am looking to perform a regular removal or clearing of the indexes so that my syslogs do not stop populating every week what is the best way to do that.
As an example I have indexes from 2021 that I want to keep but they are taking up an index and if I remove one or two I then get more syslog data. I want to automate and ensure that syslogs continue to be collected without losing current data that is 6 months old. Data older than 6months is good to be put into cold storage.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.