Hi hoping someone could help point me in the right direction. I just setup the mimecast integration and pulling data in. The problem is with mimecast, and the vast amount of logs it needs to ingest, i am about 7 days behind on the SIEM logs. Is there a way to get caught up?
I am running onprem server and resources seem to be fine on it.
Thanks for any suggestions.
Full disclosure: I'm not very familiar with the Mimecast integration.
It looks like Mimecast stores 30 minutes of logs per file and by default every 5 minutes, Agent asks for the oldest file, then the second oldest file, then the third oldest file, etc until it finally catches up.
Looking at the integration settings I'm wondering if decreasing the interval setting would allow the Agent to catch up more quickly?
Reducing to 3/2/1 minute might decrease the time to catch up by having the Agent wait less time before grabbing the next batch of logs from the Mimecast API.
Hitting the API more often might cause rate limiting so something to watch out for
Thank you for the suggestion. I originally thought it was not able to collect enough data in the interval so I extended it but that did not help.
I took your suggestion and lowered it down to 30s for the Interval and was able to pull data in and it appears to be catching up.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
