# How can split discovery message field? **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [August 9, 2017, 4:26am UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370 "2017-08-09T04:26:35Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![abu.sayeed](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Post date:** [August 9, 2017, 4:26am UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370/1 "2017-08-09T04:26:35Z")
Elasticsearch version 5.5.1 kibana version 5.5.1 Logstash version 5.5.1 Beats version 5.5.1 my message field: message:[INFO] Status =\> SENT | client : [IPDC] | cell : [1746710009] | message-delivery-time : [2017-08-07 09:46:27,807] | Operator: [ROBI] I would like to split according to Status, Client, Operator and message-delivery-time. I wish any expert will help me. Thanks ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [August 9, 2017, 11:43am UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370/2 "2017-08-09T11:43:56Z")
You will have to use the Logstash [grok filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html) or [dissect filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-dissect.html) to parse your message. ---
**Author:** ![abu.sayeed](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Post date:** [August 9, 2017, 12:10pm UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370/3 "2017-08-09T12:10:41Z")
my logstash.conf filter { if [type] == “syslog” { grok { match =\> { “message” =\> “%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_clientip} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}” } add\_field =\> [“received\_at”, “%{@timestamp}”] add\_field =\> [“received\_from”, “%{host}”] } syslog\_pri {} mutate { remove\_field =\> [“type”, “tags”, “input\_type”, “@version”, “beat”, “offset”] } date { match =\> [“syslog\_timestamp”, “MMM d HH:mm:ss”, “MMM dd HH:mm:ss”] } } } my discovery logs format: @timestamp:August 8th 2017, 13:45:54.424 host:vNTDACLSnTALK01 source:/home/local/group/nazdaq/logs/naztech.log message:[INFO] Status =\> SENT | client : [MTB] | cell : [1746710009] | message-delivery-time : [2017-08-07 09:46:27,807] | Operator: [GP] Available field: @timestamp message host message source I need available field: @timestamp message host message source status client operator What can I do? Please anybody help me. thanks ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [August 10, 2017, 1:50pm UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370/4 "2017-08-10T13:50:22Z")
please properly format logs and configs using the `` button. I don't see how your grok pattern can match the log in any way, as you don't use any of the keywords in your grok pattern. You can try with `https://grokdebug.herokuapp.com`. grok is basically build up regular expressions. One has to escape `[` and `|` for example. You can also try to match field by field when testing by adding `.*` to the end of your pattern. Some simple testing: `@timestamp:%{DATA:ts} host:%{DATA:host} source:%{DATA:source} message:\[%{DATA:level}\] Status => %{DATA:what} \| client : \[%{DATA:client}\] \|.*` ---
**Author:** ![abu.sayeed](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Post date:** [August 13, 2017, 10:12am UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370/5 "2017-08-13T10:12:00Z")
filter { mutate { remove\_field =\> ["type", "tags", "input\_type", "@version", "beat", "offset"] } } my discovery logs format: @timestamp:August 8th 2017, 13:45:54.424 host:vNTDACLSnTALK01 source:/home/local/group/nazdaq/logs/naztech.log message:[INFO] Status =\> SENT | client : [MTB] | cell : [1746710009] | message-delivery-time : [2017-08-07 09:46:27,807] | Operator: [GP] Available field: @timestamp message host message source I need available field: @timestamp message host message source **status** **client** **operator** **message-delivery-time** how can I add last four field? thanks ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [August 14, 2017, 12:56pm UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370/6 "2017-08-14T12:56:38Z")
I uses the `.*` pattern at the end of my grok pattern to ignore the rest of the log message. Remove the `.*` pattern and add the correct patterns for the other fields like I've already shown. I only used `DATA` in my grok pattern. `DATA` is just `.*`. That is raw unparsed text. as you have timestamps and numbers you might want to replace data with the appropriate grok pattern types. You are highly encouraged to [use the debugger](https://grokdebug.herokuapp.com) when improving your grok pattern. ---
**Author:** ![abu.sayeed](https://avatars.discourse-cdn.com/v4/letter/a/dec6dc/32.png) [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Post date:** [August 27, 2017, 9:37am UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370/7 "2017-08-27T09:37:12Z")
Again I change logstash filter.conf filter { grok { match =\> { "message" =\> "@timestamp:%{DATA:ts} host:%{DATA:host} source:%{DATA:source} operator:%{DATA:operator} message:[%{DATA:level}] status =\> %{DATA:what} | client : [%{DATA:client}] " } } mutate { remove\_field =\> ["type", "tags", "input\_type", "@version", "beat", "offset"] } } discovery logs: **@timestamp** : August 27th 2017, 15:33:50.603 t **host** : vNTDACLSnTALK01 t **message** : log-level : [INFO], status : [FAIL], client : [IPDC], cell : [1746710009], message\_delivery\_time : [2017-08-14 09:46:27,807], operator: [ROBI] t **source** : /home/local/group/nazdaq/logs/naztech.log but don't show field: **status** **client** **operator** **message-delivery-time** So I need help Thanks ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [August 28, 2017, 2:53pm UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370/8 "2017-08-28T14:53:40Z")
grok is build on regular expressions. Regular expressions are used to match the contents against the regular language describe by the regular expression. When parsing some semi-structured data one must obey order and can not have arbitrary nesting. Your grok pattern uses `|` as separate, but your sample log uses `,`. Also please format using the `` button, I have no idea how correct your event is (like number of lines, whitespace and others...). Not just the separate did change, but also the order of the fields. Maybe the [kv](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html) filter is a better fit for the initial parsing. If you have Logstash configuration/parsing/processing related questions, better use the Logstash forum. There you will find users with much more experience in Logstash then in the filebeat forum. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 25, 2017, 2:54pm UTC](https://discuss.elastic.co/t/how-can-split-discovery-message-field/96370/9 "2017-09-25T14:54:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.