# Beats encodes angle brackets ("\<" and "\>") as \\u003c and \\u003e in JSON output

**URL:** <https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667>\
**Category:** Beats\
**Created:** [September 16, 2016, 4:24am UTC](https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667 "2016-09-16T04:24:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![justintime32](https://avatars.discourse-cdn.com/v4/letter/j/d2c977/32.png) [@justintime32](https://discuss.elastic.co/u/justintime32)\
**Post date:** [September 16, 2016, 4:24am UTC](https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667/1 "2016-09-16T04:24:10Z")

</div>

I'm in the process of setting up filebeat to ship out logs on some of my servers. One of my logs has angle brackets in it ("\<" and "\>"). Once filebeat processes it and outputs its JSON representation, those angle brackets have been replaced with \u003c and \u003e, respectively.

Sample log:

```auto
Sep 15 17:49:02 [26263] <warning> [rest of log omitted]

```

JSON output:

```auto
{
   "@timestamp":"2016-09-16T01:06:24.394Z",
   "beat":{
      "hostname":"[redacted]",
      "name":"[redacted]"
   },
   "input_type":"log",
   "message":"Sep 15 17:49:02 [26263] \u003cwarning\u003e [rest of log omitted]",
   "offset":2229,
   "source":"[redacted]"
}

```

I believe this has to do with Go's JSON library, which encodes like this by default. More info: [https://golang.org/pkg/encoding/json/#Marshal](https://golang.org/pkg/encoding/json/#Marshal)

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 16, 2016, 5:29am UTC](https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667/2 "2016-09-16T05:29:46Z")

</div>

Yes, that's right. Go's JSON marshaller does this by default. Go 1.7 introduces an option to disable this behavior. Is the current behavior a problem?

---

<div class="post-metadata">

**Author:** ![justintime32](https://avatars.discourse-cdn.com/v4/letter/j/d2c977/32.png) [@justintime32](https://discuss.elastic.co/u/justintime32)\
**Post date:** [September 16, 2016, 3:44pm UTC](https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667/3 "2016-09-16T15:44:17Z")

</div>

I believe that this behavior is a problem. IMO, filebeat should be putting the exact log message into the JSON envelope, only changing data necessary for that encapsulation (e.g. escaping double quotation marks). Angle brackets can go into a JSON object as-is.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 17, 2016, 5:40pm UTC](https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667/4 "2016-09-17T17:40:40Z")

</div>

Philosophical musings aside, in what way is the current behavior a problem?

---

<div class="post-metadata">

**Author:** ![justintime32](https://avatars.discourse-cdn.com/v4/letter/j/d2c977/32.png) [@justintime32](https://discuss.elastic.co/u/justintime32)\
**Post date:** [September 19, 2016, 4:30am UTC](https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667/5 "2016-09-19T04:30:22Z")

</div>

In my case, I'm using a JSON parser that does not interpret unicode escape sequences. Logs that include something like "\u003c" in them show up as "u003c" once de-JSONified. I assume it is interpreting "\u" as a literal "u".

I've done a bit of research and I've since discovered that many JSON parsers do interpret these unicode escape sequences, so this may not be a problem for everyone.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 19, 2016, 5:49am UTC](https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667/6 "2016-09-19T05:49:57Z")

</div>

Such escape sequences are part of the JSON standard so I'd expect any JSON parser to support them. That said, the [raison d'être for escaping angle brackets](https://golang.org/pkg/encoding/json/#Marshal),

> to keep some browsers from misinterpreting JSON output as HTML

hardly applies in the Beats case so now that Go 1.7 is out and Beats is already using it I think it's reasonable to disable that kind of escaping. I filed [Disable useless HTML escaping when marshaling JSON · Issue #2581 · elastic/beats · GitHub](https://github.com/elastic/beats/issues/2581) for this.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [September 19, 2016, 8:18am UTC](https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667/7 "2016-09-19T08:18:43Z")

</div>

@magnusbaeck Thanks for opening the issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 7, 2016, 4:24am UTC](https://discuss.elastic.co/t/beats-encodes-angle-brackets-and-as-u003c-and-u003e-in-json-output/60667/8 "2016-10-07T04:24:13Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.