# Ingest node message parsing issue

**URL:** https://discuss.elastic.co/t/ingest-node-message-parsing-issue/147173
**Category:** Beats
**Tags:** filebeat
**Created:** [September 4, 2018, 9:03am UTC](https://discuss.elastic.co/t/ingest-node-message-parsing-issue/147173 "2018-09-04T09:03:49Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![miki\_haiat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miki_haiat/32/83361_2.png) [@miki\_haiat](https://discuss.elastic.co/u/miki_haiat)
#### Post date: [September 4, 2018, 9:03am UTC](https://discuss.elastic.co/t/ingest-node-message-parsing-issue/147173/1 "2018-09-04T09:03:49Z")

</div>

Hi ,  
I have some share point log that that i want to use \_ingest in order to process them.  
its works perfect on the grok debugger but im unable to post it without parsing error

this is the grok pattern  
`(?<parsedtime>%{MONTHNUM}/%{MONTHDAY}/%{YEAR} %{HOUR}:%{MINUTE}:%{SECOND}) \t%{DATA:process} \(%{DATA:processcode}\)(\s*)\t%{DATA:tid}(\s*)\t(?<area>.*)(\s*)\t(?<category>.*)(\s*)\t%{WORD:eventID}(\s*)\t%{WORD:level}(\s*)\t%{DATA:eventmessage}\t%{UUID:CorrelationID}`

post it to the server as valid json is a bit tricky ...  
so i tried to url encode the data

```
 "patterns": ["(%3F%3Cparsedtime%3E%25%7BMONTHNUM%7D%2F%25%7BMONTHDAY%7D%2F%25%7BYEAR%7D%20%25%7BHOUR%7D%3A%25%7BMINUTE%7D%3A%25%7BSECOND%7D)%20%5Ct%25%7BDATA%3Aprocess%7D%20%5C(%25%7BDATA%3Aprocesscode%7D%5C)(%5Cs*)%5Ct%25%7BDATA%3Atid%7D(%5Cs*)%5Ct(%3F%3Carea%3E.*)(%5Cs*)%5Ct(%3F%3Ccategory%3E.*)(%5Cs*)%5Ct%25%7BWORD%3AeventID%7D(%5Cs*)%5Ct%25%7BWORD%3Alevel%7D(%5Cs*)%5Ct%25%7BDATA%3Aeventmessage%7D%5Ct%25%7BUUID%3ACorrelationID%7D"]

        },

```

Any suggestion how to post correctly this pattern ?

the error thrown by the server

```
  "caused_by": {
                "type": "json_parse_exception",
                "reason": "Illegal unquoted character ((CTRL-CHAR, code 9)): has to be escaped using backslash to be included in string value\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@17e1d17e; line: 12, column: 39]"
            }
```

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [September 4, 2018, 9:29am UTC](https://discuss.elastic.co/t/ingest-node-message-parsing-issue/147173/2 "2018-09-04T09:29:24Z")

</div>

Could you please share the full pipeline?

---

<div class="post-metadata">

### Author: ![miki\_haiat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miki_haiat/32/83361_2.png) [@miki\_haiat](https://discuss.elastic.co/u/miki_haiat)
#### Post date: [September 4, 2018, 1:48pm UTC](https://discuss.elastic.co/t/ingest-node-message-parsing-issue/147173/3 "2018-09-04T13:48:24Z")

</div>

```
{
    "description": "sharepoint pipeline",
    "processors": [
        {
       "grok": {
        "field": "message",
         "patterns": ["(%3F%3Cparsedtime%3E%25%7BMONTHNUM%7D%2F%25%7BMONTHDAY%7D%2F%25%7BYEAR%7D%20%25%7BHOUR%7D%3A%25%7BMINUTE%7D%3A%25%7BSECOND%7D)%20%5Ct%25%7BDATA%3Aprocess%7D%20%5C(%25%7BDATA%3Aprocesscode%7D%5C)(%5Cs*)%5Ct%25%7BDATA%3Atid%7D(%5Cs*)%5Ct(%3F%3Carea%3E.*)(%5Cs*)%5Ct(%3F%3Ccategory%3E.*)(%5Cs*)%5Ct%25%7BWORD%3AeventID%7D(%5Cs*)%5Ct%25%7BWORD%3Alevel%7D(%5Cs*)%5Ct%25%7BDATA%3Aeventmessage%7D%5Ct%25%7BUUID%3ACorrelationID%7D"]

        }
      
        }
       
    ]
}
```

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [September 4, 2018, 4:16pm UTC](https://discuss.elastic.co/t/ingest-node-message-parsing-issue/147173/4 "2018-09-04T16:16:55Z")

</div>

And also an example log. Sorry for not asking for it previously.

---

<div class="post-metadata">

### Author: ![miki\_haiat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miki_haiat/32/83361_2.png) [@miki\_haiat](https://discuss.elastic.co/u/miki_haiat)
#### Post date: [September 5, 2018, 7:57am UTC](https://discuss.elastic.co/t/ingest-node-message-parsing-issue/147173/5 "2018-09-05T07:57:50Z")

</div>

```
Timestamp Process TID Area Category EventID	Level Message Correlation
09/03/2018 09:14:12.18 w3wp.exe (0x31B4) 0x3F9C	SharePoint Foundation Runtime afu6b	High [Forced due to logging gap, cached @ 09/03/2018 09:14:12.10, Original Level: VerboseEx] No SPAggregateResourceTally associated with thread.
```

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [September 5, 2018, 2:13pm UTC](https://discuss.elastic.co/t/ingest-node-message-parsing-issue/147173/6 "2018-09-05T14:13:11Z")

</div>

The escaping was not correct in your pipeline. It's tricky to port patterns developed with Grok debugger to Filebeat. As it's read from a file using Golang slashes need extra escaping.

So your pattern should look like this:  
`(?<parsedtime>%{MONTHNUM}/%{MONTHDAY}/%{YEAR} %{HOUR}:%{MINUTE}:%{SECOND}) \\t%{DATA:process} \\(%{DATA:processcode}\\)(\\s*)\\t%{DATA:tid}(\\s*)\\t(?<area>.*)(\\s*)\\t(?<category>.*)(\\s*)\\t%{WORD:eventID}(\\s*)\\t%{WORD:level}(\\s*)\\t%{DATA:eventmessage}\\t%{UUID:CorrelationID}`

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 3, 2018, 2:13pm UTC](https://discuss.elastic.co/t/ingest-node-message-parsing-issue/147173/7 "2018-10-03T14:13:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
