# Please tell me how to get message as well as the content of delimiter and AIR name in output as i am getting only message as output

**URL:** <https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178>\
**Category:** Logstash\
**Created:** [February 9, 2018, 7:06am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178 "2018-02-09T07:06:41Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 9, 2018, 7:06am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/1 "2018-02-09T07:06:41Z")

</div>

input {  
file {  
path =\> "C:\PROJECT\ELK\_STACK\logstash-6.1.2\bin\files\tcp\_1\_logs.log"  
}  
}  
filter {  
grok {  
match =\> ["message", "(?^ Error in sending batch file )(?._)"]  
match =\> ["message", "(? to AIR server: ) (?._) (?permanently. Retry attempts maxed out)"]  
}  
}  
output {  
stdout {  
codec =\> rubydebug  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 9, 2018, 7:37am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/2 "2018-02-09T07:37:17Z")

</div>

Please

- format your configuration snippet as preformatted text using standard Markdown notation or the `</>` toolbar button,
- show what's produced by your stdout output and
- show an example line of input.

---

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 9, 2018, 8:33am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/3 "2018-02-09T08:33:34Z")

</div>

![sample%20input_and_output](https://us1.discourse-cdn.com/elastic/original/3X/d/5/d5c7ec69364502548d250d9d5d518ac82da8f759.png)

_sample code:_

input {  
file {  
path =\> "C:\PROJECT\ELK\_STACK\logstash-6.1.2\bin\files\tcp\_1\_logs.log"  
}  
}  
filter {  
grok {  
match =\> [ "message", "(?^Error in sending batch file )(?._) (?to AIR server: ) (?._)  
(?permanently. Retry attempts maxed out )" ]  
}  
}  
output {  
stdout {  
codec =\> rubydebug  
}  
}

expected output: message + content in delimiter , batchname and AIR name.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 9, 2018, 8:47am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/4 "2018-02-09T08:47:47Z")

</div>

Please have another look at the request in my first bullet.

---

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 9, 2018, 8:56am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/5 "2018-02-09T08:56:53Z")

</div>

format your configuration snippet as preformatted text using standard Markdown notation or the \</\> toolbar button.  
Can you please elaborate this?

---

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 12, 2018, 5:17am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/6 "2018-02-12T05:17:42Z")

</div>

input {  
file {  
path =\> "C:\PROJECT\ELK\_STACK\logstash-6.1.2\bin\files\tcp\_1\_logs.log"  
}  
}  
filter {  
grok {  
match =\> ["message", "(?^Outbound FTP failed for the batch )(?.\*) (?Trying again....)"]  
}

```
grok {
	match => [ "message", "(?<delimeter>^Error in sending batch file )(?<batchname>.*) (?<delimeter>to AIR server: ) (?<AIR name>.*) 
		 (?<delimeter>permanently. Retry attempts maxed out )" ]
}
grok {
	match => ["message","(?<delimeter>^File(s) matching )(?<fileNamePattern>.*) (?<delimeter> not found on remote site. Attempt)(?<attemptCount>.*) (?<delimeter>out of ) (?<downloadFileAttempts>.*)"]
}
grok {
	match => [ "message", "(?<delimeter>^could not find remote file(s) matching ) (?<fileNamePattern>.*) (?<delimeter>after) 
		 (<downloadFileAttempts>.*) (?<delimeter>attempts)" ] 
}
grok {
	match => ["message", "(?<delimeter>^Failed to receive RPT file corresponding to )(?<dat file>.*)"] 
}

grok {
	match => ["message","(?<delimeter>^Inbound FTP for batch:) (?<fileNamePattern>.*) (?<delimeter>permanently. Retry attempts maxed out)"] 
}

grok {
	match => ["message", "(?<fileNamePattern>.*) (?<delimeter> not yet received from AIR & therefore excluding from consolidation report:) (?<inputFile>.*)"]
}		

```

}  
output {  
stdout {  
codec =\> rubydebug  
}  
}

please see the formatted code above and issue is same as i am not getting other contents except message in the output.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 12, 2018, 7:35am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/7 "2018-02-12T07:35:47Z")

</div>

Don't use multiple grok filters. Use a single filter with multiple expressions. Then the expressions will be tried in order until there's a match. There's an example in the grok filter documentation.

In the screenshot you posted earlier (please don't post screenshots if it's possible to copy/paste plain text) the `message` field begins with "ror" instead of "Error". That looks very suspicious.

---

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 12, 2018, 8:47am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/8 "2018-02-12T08:47:02Z")

</div>

filter {  
grok {  
match =\> [ "message", "(?^Error in sending batch file )(?._) (?to AIR server: ) (?._?)  
(?permanently. Retry attempts maxed out )" ]  
match =\> [ "message", "(?^Error in sending batch file )(?._) (?to AIR server: ) (?._)  
(?permanently. Retry attempts maxed out )" ]  
match =\> ["message","(?^File(s) matching )(?._) (? not found on remote site. Attempt)(?._)"]  
match =\> [ "message", "(?^could not find remote file(s) matching ) (?._) (?after)  
(._) (?attempts)" ]  
match =\> ["message", "(?^Failed to receive RPT file corresponding to )(?._)"]  
match =\> ["message","(?^Inbound FTP for batch:) (?._) (?permanently. Retry attempts maxed out)"]  
match =\> ["message", "(?.\*) (? not yet received from AIR & therefore excluding from consolidation report:)"]  
}  
}

Now also output is not coming as desired. Is there any restriction on the use of regex  
(?.\*) multiple times using match in grok filter?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 12, 2018, 1:01pm UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/9 "2018-02-12T13:01:20Z")

</div>

Here's the documentation example I was thinking of:

```auto
filter {
  grok { match => { "message" => ["Duration: %{NUMBER:duration}", "Speed: %{NUMBER:speed}"] } }
}

```

> Now also output is not coming as desired.

So what _do_ you get?

> Is there any restriction on the use of regex  
> (?.\*) multiple times using match in grok filter?

Technically no, but multiple `.*` in the same expression is slow and could easily become ambiguous. Avoid it.

---

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 12, 2018, 6:03pm UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/10 "2018-02-12T18:03:32Z")

</div>

filter {  
grok {  
match =\> { "message" =\> [ "data: %{delimiter:Error in sending batch file}", (?._), "data : %{delimiter:to AIR server:}", (?._),  
"data : %{delimiter: permanently. Retry attempts maxed out}" ] }  
}  
}

please check what is wrong in this syntax?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 12, 2018, 9:02pm UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/11 "2018-02-12T21:02:40Z")

</div>

Unless you format the configuration as preformatted it'll get mangled and I won't see exactly what it looks like.

---

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 13, 2018, 5:05am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/12 "2018-02-13T05:05:42Z")

</div>

```
filter {
	grok {
		match => { "message" => [ "data: %{delimiter:Error in sending batch file}", (?<batchname>.*), "data : %{delimiter:to AIR server:}", (?<AIR name>.*),
				   "data : %{delimiter: permanently. Retry attempts maxed out}" ] }
	}
}

```

here i want output to give content of delimiter + content of AIR name + content of batch name for log like "Error in sending batch file BATCH\_ADJUST\_20180206\_jj005 to AIR server: jjjj permanently. Retry attempts maxed out".  
please suggest me how to get BATCH\_ADJUST\_20180206\_jj005 in batchname, jjjj in AIR name and other information of log in delimiter.  
I hope you get my point.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 13, 2018, 7:10am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/13 "2018-02-13T07:10:58Z")

</div>

What's up with "data:" and "data :" at the beginning of the expression? I don't see it in the log example you posted earlier. Please post it again, as text.

---

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 13, 2018, 8:23am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/14 "2018-02-13T08:23:14Z")

</div>

```
grok {
		match => [ "message", "(?<delimeter>^Error in sending batch file )(?<batchname>.*) (?<delimeter>to AIR server: ) (?<AIR name>.*) 
			     (?<delimeter>permanently. Retry attempts maxed out )" ]
	}

```

here, i want content of delimiter, batchname and AIR Server as output having log with content: "Error in sending batch file BATCH\_ADJUST\_20180206\_jj005 to AIR server: jjjj permanently. Retry attempts maxed out".  
After running this, i am getting only whole message as output but i want seperate batchname, AIR name and delimiter content along with message as ouput.  
please suggest me how to go about it?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 13, 2018, 8:33am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/15 "2018-02-13T08:33:19Z")

</div>

Follow the request I made in my previous post.

---

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 14, 2018, 5:57am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/16 "2018-02-14T05:57:58Z")

</div>

log: Error in sending batch file BATCH\_ADJUST\_20180206\_jj005 to AIR server: jjjj permanently. Retry attempts maxed out

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 14, 2018, 7:15am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/17 "2018-02-14T07:15:13Z")

</div>

And still you don't manage to format the log entry as I requested. Stuff like that matters! If the log you post don't match what you're actually trying to parse my help might be of no use to you and then everyone's time is wasted.

Now, there appears to be two problems with your current expression:

- If your log _really_ begins with "log: " you need to drop the `^`.
- You're being sloppy with the whitespace.

This works:

```
(?<delimeter>Error in sending batch file )(?<batchname>.*) (?<delimeter>to AIR server: )(?<AIR name>.*) (?<delimeter>permanently. Retry attempts maxed out)

```

Notes:

- I have no idea why you're capturing strings into the `delimiter` field in multiple places.
- The repeated use of `.*` is very inefficient and could under some circumstances backfire, resulting in incorrect and surprising matches. Any grok expression with more than one `.*`, `DATA`, or `GREEDYDATA` should be heavily scrutinized.

Over and out.

---

<div class="post-metadata">

**Author:** ![abhi210797](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@abhi210797](https://discuss.elastic.co/u/abhi210797)\
**Post date:** [February 14, 2018, 8:07am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/18 "2018-02-14T08:07:16Z")

</div>

thank you @magnusbaeck for your valuable suggestions and really good advices.🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 14, 2018, 8:07am UTC](https://discuss.elastic.co/t/please-tell-me-how-to-get-message-as-well-as-the-content-of-delimiter-and-air-name-in-output-as-i-am-getting-only-message-as-output/119178/19 "2018-03-14T08:07:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.