Filebeat / logstash "merging" events?

I have setup ELK stack on one machine, and feed logs from filebeat from another machine, the one that hosts the actual logs, to logstash.

Log entries are quite diverse, however they conform to the following structure:

2017-11-14T11:52:44.386-0500 INFO 8646 com.l7tech.traffic: APITransaction - ;service=oauth/clients;RequestUrl=https://oauth2dns:8443/pub/oauth/clientstore/getAll?client_ident=&client_key=da95da9c-2ab1-42a9-8878-d4457c322036&format=&name=&org=&registered_by=&queryOffset=0&filterStatus=&environment=&master=;RequestMethod=GET;RequestSize=0;ResponseSize=;TotalTime=7;RoutingTime=;HttpStatus=0;Request=XXXXXXXXXXXX;Response=YYYYYYYYYYYYYYYY;

Request and Response may be pretty much anything and span multiple lines, therefore I am using multiline in filebeat.yml like this:

pattern: ^(19|20)\\d\\d-(0[1-9]|1[012])-([012]\\d|3[01])T([01]\\d|2[0-3]):([0-5]\\d):([0-5]\\d)$
negate: true
match: after

logstash in turn is configured as follows:

input {
beats {
port => 5044
filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log_level} %{SPACE} %{WORD:length} %{IPORHOST:dns}: %{WORD:transaction_text} - ;%{GREEDYDATA:kvpairs};Request=%{GREEDYDATA:request};Response=%{GREEDYDATA:response};" }
kv {
source => "kvpairs"
value_split => "="
field_split => ";"
# remove_field => [ "kvpairs" ] #Delete the field afterwards
output {
elasticsearch {hosts => ["localhost:9200"] }
stdout {codec => rubydebug }

This usually works, unless the log events are written within the same second or so, when they are "merged" into one single event. In other words, I am sending to requests, a GET and a POST 3-4 seconds apart, everything is just fine, logstash output shows me a GET event followed by a POST. Sending exactly the same 2 requests quickly, sometimes I se one event only GET, POST in logstash output.
I would assume there is something to be tuned up, but whatever I tried, didn't really help.

Thanks with anticipation

No sweat. I figured it out. As usual, check the simple things first: I overlooked the following

"Filebeat supports a subset of the regular expression syntax accepted by RE2. Because we use the POSIX implementation, some patterns are currently not supported."

\d seems not to be supported by Filebeat, therefore I had to settle for the following regexp, and now it works as expected:

pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}-'

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.