Thanks @tudor, this is amazing news... I'm assuming that
ES5.0 wont have plugin support in the way
Logstash currently has, and that it would probably be something like
Do you by chance know any more about this, or have a link to further details?
I clearly misunderstood the purpose of
regxp, and I'll need to look into other options (
Logstash being top of the list).
Thanks @anhlqn, I'm going to look into
nxlog as a solution to the issue, but I have been hoping that I could pipe all my logs directly to my
ES cluster. On a large scale, I feel that
Logstash can be a bottleneck/SPOF (I do wonder if
nxlog would introduce the same issue), and it important to monitor and know when
LS is causing a bottleneck. I'd like to avoid that scenario.
I understand there are methods to cluster
LS but I'm looking for simple design, so that it can scale easier.
Thanks both of you.
One last question to anyone who may know:
Is there anyway to use filebeat with the effect of pre-processing. For example, if
filebeat log input is in properly-formatted
JSON-output, would this assist in splitting?
My understanding is that this wouldn't require any processing, just an understanding of JSON.