# Grokparse failure seen with tcp input plugin in logstash pipeline **URL:** \ **Category:** Logstash\ **Created:** [October 26, 2022, 12:20am UTC](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447 "2022-10-26T00:20:01Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Arinjay\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arinjay_jain/32/97753_2.png) [@Arinjay\_Jain](https://discuss.elastic.co/u/Arinjay_Jain)\ **Post date:** [October 26, 2022, 12:20am UTC](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447/1 "2022-10-26T00:20:01Z")
Hi, I have the following logstash pipeline configuration. ```auto input { tcp { port => 5102 codec => plain } } filter { grok { match => {"message" => "%{SYSLOGTIMESTAMP:time} %{DATA:trace_name} %{DATA:node_name} %{DATA:count} %{DATA:thread_id} %{GREEDYDATA:data}"} } } output { file { path => "%{trace_name}.txt" codec => line } } ``` I have written a client program which connects to logstash server instance on port 5102 and sends log data to it which contains multiple lines of data. I wanted to store all the log data having same trace name in its own file. But in some cases I am seeing that the log data is written to a file named "%{trace\_name}.txt" and on debugging further I found that due to grok parse failure. It looks like the data received from TCP socket in input plugin is not processing log data line by line. Whenever the log data received is terminated with a "\n", the grok filter is able to parse the log message successfully, but it fails if the message is truncated. Can someone suggest what configuration needs to be used so that log data received from TCP socket is processed by the grok filter one line at a time. Thanks, Arinjay ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [October 26, 2022, 5:48am UTC](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447/2 "2022-10-26T05:48:40Z")
The "%{trace\_name}.txt" means it's an empty value. Handle errors 1. Check is your grok pattern always OK, use ruby debugger. You might have 2 grok patterns or "trace\_name" as an optional value. 2. Handle the empty value, something like this ```auto if ![{trace_name]{ mutate { add_field => { "[trace_name]" => "/path/filename.txt" } } } ``` 1. Handle grok errors in output ```auto if "_grokparsefailure" in [tags] { file { path => "/path/grok_error.txt" codec => line } } ``` ---
**Author:** ![Arinjay\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arinjay_jain/32/97753_2.png) [@Arinjay\_Jain](https://discuss.elastic.co/u/Arinjay_Jain)\ **Post date:** [October 26, 2022, 6:07am UTC](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447/3 "2022-10-26T06:07:35Z")
Thanks @Rios . I checked the grok pattern in debugger and found that the messages are having 2 different formats. One format has all the fields as defined in the grok pattern above, but other has "count" field missing. Grok parsing is working ok when the message contains all the fields. I think I need to make "count" field optional in the pattern. Can you tell how to make this field optional ? I tried something like below, but it didnt work. ```auto match => {"message" => "%{SYSLOGTIMESTAMP:time} %{DATA:trace_name} %{DATA:node_name} (%{DATA:count})? %{DATA:thread_id} %{GREEDYDATA:data}"} ``` Thanks, Arinjay ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [October 26, 2022, 6:31am UTC](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447/4 "2022-10-26T06:31:48Z")
Space " " means mandatory value. You can use \s\* or %{SPACE}, which means zero or more occurrences of space. Try: `%{DATA:time} %{DATA:trace_name} %{DATA:node_name}%{SPACE}(%{DATA:count})?\s*%{DATA:thread_id} %{GREEDYDATA:data}` ---
**Author:** ![Arinjay\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arinjay_jain/32/97753_2.png) [@Arinjay\_Jain](https://discuss.elastic.co/u/Arinjay_Jain)\ **Post date:** [October 28, 2022, 5:47pm UTC](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447/5 "2022-10-28T17:47:53Z")
Thanks Rios. I was able to fix the grok pattern and see the log messages getting stored with the correct file names. Thanks, Arinjay ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [October 30, 2022, 11:52am UTC](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447/6 "2022-10-30T11:52:14Z")
You welcome. Just keep consistent, use either regex syntax \s+ or LS syntax %{SPACE} in the grok matching. I intentionally mix both to see that is possible. Summary: " " - a single static space, must be separated by only one space character. \s\* - zero or more white spaces. That is: \s matches a space, a tab, a carriage return, a line feed, or a form feed. \s+ - one or more white spaces %{SPACE} - same as \s\*, as is mentioned in [grok patterns](https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 27, 2022, 11:53am UTC](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447/7 "2022-11-27T11:53:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.