# \_grokparsefailure on file parsing

**URL:** <https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 3, 2016, 8:30am UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816 "2016-11-03T08:30:47Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![elasticheart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elasticheart/32/65189_2.png) [@elasticheart](https://discuss.elastic.co/u/elasticheart)\
**Post date:** [November 3, 2016, 8:30am UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816/1 "2016-11-03T08:30:47Z")

</div>

Hi,

I have a multi line log file structured something like;

```
<Aug 02, 2016 11:21:05:049 AM> <dataa> <datab> <datac> <datad> <datae> <dataf> <datag>
 <datah>

```

I have set multiline pattern in filebeat like;

```
multiline.pattern: '^<[A-Za-z_]{3} [[:digit:]]{2}, [[:digit:]]{4} ([[:digit:]]{1}|[[:digit:]]{2}):[[:digit:]]{2}:[[:digit:]]{2}:([[:digit:]]{1}|[[:digit:]]{2}|[[:digit:]]{3}) [A-Z]{2}>'
multiline.negate: true
multiline.match: after
multiline.max_lines: 5000

```

This pushes data to kafka and then logstash consumes from it. The problem is, filebeat is using `\u003c` and `\u003e` instead of `<` and `>` in the message, which makes a `_grokparsefailure` in my logstash. Encoding is utf8 `encoding: utf-8`.

How can I fix this?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 3, 2016, 8:34am UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816/2 "2016-11-03T08:34:36Z")

</div>

It seems unlikely that Logstash's JSON deserializer wouldn't translate \u003c to \<. Please show your grok filter and what a failed events looks like. Use a `stdout { codec => rubydebug }` output and copy/paste its output.

---

<div class="post-metadata">

**Author:** ![elasticheart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elasticheart/32/65189_2.png) [@elasticheart](https://discuss.elastic.co/u/elasticheart)\
**Post date:** [November 3, 2016, 8:39am UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816/3 "2016-11-03T08:39:13Z")

</div>

```
grok {
	match => { "message" => "<(?<timestamp>%{MONTH} %{MONTHDAY}, 20%{YEAR} %{HOUR}:?%{MINUTE}(?::?%{SECOND}) (?:AM|PM))\> <%{GREEDYDATA:dataa}> <%{GREEDYDATA:datab}> <%{GREEDYDATA:datac}> <%{GREEDYDATA:datad}> <%{GREEDYDATA:datae}> <%{GREEDYDATA:dataf}> <%{GREEDYDATA:datag}>\n <%{GREEDYDATA:datah}>" }
}

```

and the output message you can think like a text with `\u003c` and `\u003e`instead of `<` and `>`. It was working fine with version 2, but just now I switched to v5 and testing on it.

---

<div class="post-metadata">

**Author:** ![elasticheart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elasticheart/32/65189_2.png) [@elasticheart](https://discuss.elastic.co/u/elasticheart)\
**Post date:** [November 3, 2016, 8:45am UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816/4 "2016-11-03T08:45:47Z")

</div>

Its not a problem with logstash I think, because in the filebeat log also, its `\u003c` and `\u003e`

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [November 3, 2016, 9:49am UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816/5 "2016-11-03T09:49:01Z")

</div>

Sounds like [https://github.com/elastic/beats/issues/2581](https://github.com/elastic/beats/issues/2581) ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 3, 2016, 9:51am UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816/6 "2016-11-03T09:51:38Z")

</div>

In JSON, \u003c and \< are equivalent so it's totally fine for Filebeat to use \u003c instead of \<. At least Logstash 2.4 handles this just fine:

```nohighlight
$ cat test.config 
input { stdin { codec => json } }
output { stdout { codec => rubydebug } }
$ echo '{"message": "\u003cfoo\u003e"}' | /opt/logstash/bin/logstash -f test.config
Settings: Default pipeline workers: 8
Pipeline main started
{
       "message" => "<foo>",
      "@version" => "1",
    "@timestamp" => "2016-11-03T09:51:09.244Z",
          "host" => "lnxolofon"
}
Pipeline main has been shutdown
stopping pipeline {:id=>"main"}

```

See also:

> <https://stackoverflow.com/questions/28595664/how-to-stop-json-marshal-from-escaping-and>

---

<div class="post-metadata">

**Author:** ![elasticheart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elasticheart/32/65189_2.png) [@elasticheart](https://discuss.elastic.co/u/elasticheart)\
**Post date:** [November 3, 2016, 11:39am UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816/8 "2016-11-03T11:39:02Z")

</div>

I have tried with just console output and this is what I got;

```
{
  "@timestamp": "2016-11-03T11:19:52.393Z",
  "beat": {
    "hostname": "localhost",
    "name": "localhost",
    "version": "5.0.0"
  },
  "input_type": "log",
  "message": "\u003cNov 02, 2016 10:49:42:810 AM\u003e \u003cdataa\u003e \u003cdatab\u003e \u003cdatac\u003e \u003cdatad\u003e \u003cdatae\u003e \u003cdataf\u003e \u003cdatag\u003e\n \u003cdatah\n\u003e",
  "offset": 95625,
  "source": "/logfiles/logfile.log",
  "type": "log"
}
```

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [November 3, 2016, 2:59pm UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816/9 "2016-11-03T14:59:14Z")

</div>

Is there a reason you also opened [Filebeat error. Returns unicode character code instead of symbol](https://discuss.elastic.co/t/filebeat-error-returns-unicode-character-code-instead-of-symbol/64827/1) ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 24, 2016, 8:31am UTC](https://discuss.elastic.co/t/-grokparsefailure-on-file-parsing/64816/10 "2016-11-24T08:31:05Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.