# Grok out a field withing another field **URL:** \ **Category:** Logstash\ **Created:** [March 26, 2018, 8:34pm UTC](https://discuss.elastic.co/t/grok-out-a-field-withing-another-field/125673 "2018-03-26T20:34:01Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\ **Post date:** [March 26, 2018, 8:34pm UTC](https://discuss.elastic.co/t/grok-out-a-field-withing-another-field/125673/1 "2018-03-26T20:34:01Z")
Hello, I have the following message: ```auto [Linux][Baseline][rwbdfrmmop03][Process][srmclient][SiSExclude] ``` I want to extract the whole message as monitor\_name ```auto [Linux][Baseline][rwbdfrmmop03][Process][srmclient][SiSExclude] ``` Then i want to also extract the remote\_host: rwbdfrmmop03 from monitor\_name I tired something like this, but it did not work: ```auto (?(?)(?)(?[a-z0-9])%{GREEDYDATA}) ``` Thanks for the help. ---
**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\ **Post date:** [March 27, 2018, 1:04am UTC](https://discuss.elastic.co/t/grok-out-a-field-withing-another-field/125673/2 "2018-03-27T01:04:20Z")
You may be better working in two phases: first, extract the `monitor_name` (which I am reading to be a sequence of strings, each bound by square brackets), and then, in a separate filter, extract the relevant bits _out_ of the `monitor_name` (I would prefer [dissect][], since it is a lot simpler to understand and faster at extracting when the patterns of the _separators_ is known): ```auto filter { grok { "pattern_definitions" => { "MONITOR_NAME" => "(\[[^]]+\])+" } "match" => { "message" => "%{MONITOR_NAME:monitor_name}" } } dissect { mapping => { "monitor_name" => "[%{os}][%{monitor_type}][%{remote_host}][%{}][%{}][%{}]" } } } ``` I defined the pattern for `MONITOR_NAME` as `(\[[^]]+\])+`, because it matches any sequence of square-bracketed things: ```auto ( # open group \[ # literal open bracket [ # character class ^] # excluding a close bracket ]+ #closes character class, allows repetition \] # literal close bracket )+ # close group, allows repetition ``` Ideally, when you grok, you'll be anchoring your pattern to the beginning of your log line (by prefixing it with the `^` character), so your grok pattern will capture from the beginning of your log message. This _hugely_ improves performance because the regular expression doesn't have to try the match again starting with the second character, and again with the third, and so-on until it finds a match. ---
**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\ **Post date:** [April 10, 2018, 5:12pm UTC](https://discuss.elastic.co/t/grok-out-a-field-withing-another-field/125673/3 "2018-04-10T17:12:34Z")
@yaauie, Thanks for the information. I see that it is extracting the monitor\_name as follows: ```auto Baseline, [Linux][Baseline][rwbdfrmmop03][Process][srmclient][SiSExclude] ``` AS you can see it is extracting the Baseline and setting it as monitor\_name as well. Where in fact the monitor name should only be: ```auto [Linux][Baseline][rwbdfrmmop03][Process][srmclient][SiSExclude] ``` This is the config i have: ```auto if ([source] == "my_source"){ dissect { mapping => { "message" => "[%{}][%{monitor_name}][%{remote_host}][%{}][%{}][%{}]" } } grok { patterns_dir => ["/apps/elk/configurations/grok-patterns/my_app"] match => ["message", "%{my_match}"] } date { match => ["event_timestamp" ,"M/dd/yyyy HH:mm:ss a"] target => "@timestamp" timezone => "America/New_York" } } ``` I attempted to change monitor\_name to application\_name in the dissect fitler, but i do not see the new field being created. Essentially i would like it to be as follows: ```auto monitor_name: [Linux][Baseline][rwbdfrmmop03][Process][srmclient][SiSExclude] (working) ``` ```auto application_name: Baseline (this is not working) ``` ```auto remote_host: rwbdfrmmop03 (working) ``` Thanks! ---
**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\ **Post date:** [April 10, 2018, 10:04pm UTC](https://discuss.elastic.co/t/grok-out-a-field-withing-another-field/125673/4 "2018-04-10T22:04:20Z")
your dissect is capturing a `monitor_name` (the contents of the second set of brackets), which would explain why you have `Baseline` as a value for `monitor_name`. With a file `line.log` containing only your input message with a trailing newline: ```auto [Linux][Baseline][rwbdfrmmop03][Process][srmclient][SiSExclude] ``` And the pipeline configuration I gave originally, with `input` and `output` sections added for completeness: ```auto input { stdin {} } filter { grok { "pattern_definitions" => { "MONITOR_NAME" => "(\[[^]]+\])+" } "match" => { "message" => "%{MONITOR_NAME:monitor_name}" } } dissect { mapping => { "monitor_name" => "[%{os}][%{monitor_type}][%{remote_host}][%{}][%{}][%{}]" } } } output { stdout { codec => rubydebug } } ``` I get the following when I execute, which appears to be what you say you want: ```auto ╭─{ yaauie@castrovel:~/src/elastic/discuss-scratch/125673-grok-dissect-monitor-name } ╰─○ cat line.log | ~/src/elastic/releases/logstash-6.2.2/bin/logstash -f pipeline.conf Sending Logstash's logs to /Users/yaauie/src/elastic/releases/logstash-6.2.2/logs which is now configured via log4j2.properties [2018-04-10T21:57:48,707][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified [2018-04-10T21:57:49,168][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.2.2"} [2018-04-10T21:57:49,521][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600} [2018-04-10T21:57:52,604][INFO][logstash.pipeline] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50} [2018-04-10T21:57:52,959][INFO][logstash.pipeline] Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#"} [2018-04-10T21:57:53,055][INFO][logstash.agent] Pipelines running {:count=>1, :pipelines=>["main"]} { "monitor_name" => "[Linux][Baseline][rwbdfrmmop03][Process][srmclient][SiSExclude]", "@timestamp" => 2018-04-10T21:57:53.022Z, "host" => "castrovel.local", "remote_host" => "rwbdfrmmop03", "monitor_type" => "Baseline", "message" => "[Linux][Baseline][rwbdfrmmop03][Process][srmclient][SiSExclude]", "@version" => "1", "os" => "Linux" } [2018-04-10T21:57:53,808][INFO][logstash.pipeline] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#"} [success (20.000s)] ``` ---
**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\ **Post date:** [April 16, 2018, 3:00pm UTC](https://discuss.elastic.co/t/grok-out-a-field-withing-another-field/125673/5 "2018-04-16T15:00:33Z")
@yaauie, That worked thank you! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 14, 2018, 3:00pm UTC](https://discuss.elastic.co/t/grok-out-a-field-withing-another-field/125673/6 "2018-05-14T15:00:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.