Tcp logstash encoder

[visioncoder]

Hello,

I am using tcp-logstash-encoder 4.11 with SpringBoot 1.5.6. to push json log messages to Logstash. Some select messages are coming through fine but most are not. Console is printing all the messages.

The TCP connection itself looks to be good so it does not seem to be the case of dropped messages.

Does Logstash store error records (grokparsefailure) anywhere if it is not specified? Is there a way for me to keep track of "failed" records - the ones that reached Logstash but never made it to ES?

The relevant input and output configurations are below:

  tcp {
        port => 9202
        type => "mbo_lower"
        tags => [ "mbo" ]
        codec => json_lines
    }


 if [host] =~ /^(nameA|nameB).*$/  {
            if [host] =~ /^(nameA|nameB)\w*\-\w*\-\w*\d+[p]\..*$/ {
                elasticsearch {
                    index => "indexA-%{+YYYY.MM.dd}"
                    hosts => [ "xxxx:9200" ]
                    user => "oneUser"
                    password => "onePassword"
                }
            } else {
                elasticsearch {
                    index => "indexB-%{+YYYY.MM.dd}"
                    hosts => [ "xxxx9200" ]
                    user => "oneUser"
                    password => "onePassword"
                }
            }

Please let me know if additional information is required from my end.

Thanks!

[magnusbaeck (Magnus Bäck)]

Have you looked in the Logstash logs? What if you comment out the elasticsearch output and just use a stdout or file output, are you then getting all events?