# Filebeat escaping backslashes on its own

**URL:** <https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 15, 2016, 5:22pm UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899 "2016-06-15T17:22:09Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![fiunchinho](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@fiunchinho](https://discuss.elastic.co/u/fiunchinho)\
**Post date:** [June 15, 2016, 5:22pm UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/1 "2016-06-15T17:22:09Z")

</div>

I have a file with the following entry logs. Some Java stack traces with `\n` instead of break lines, so it's only one line per log entry

```auto
2016-05-30 11:25:53,907 WARN 4209 [DiscoveryClient-InstanceInfoReplicator-0] com.netflix.discovery.InstanceInfoReplicator : There was a problem with the instance info replicator {} com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server\n at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:111)\n at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56)\n at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$1.execute(EurekaHttpClientDecorator.java:59)\n at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77)\n at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56)\n at com.netflix.discovery.DiscoveryClient.register(DiscoveryClient.java:806)\n at com.netflix.discovery.InstanceInfoReplicator.run(InstanceInfoReplicator.java:103)\n at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n at java.util.concurrent.FutureTask.run(FutureTask.java:266)\n at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)\n at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)\n at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n at java.lang.Thread.run(Thread.java:745)\n

```

If I tell filebeat to watch this file and send it to logstash (using debug for verbosity), I can see that the break lines are being escaped, so Logstash (and then ElasticSearch) receives the character `\\n` instead of `\n`:

```auto
2016/06/15 16:32:40.250503 publish.go:112: DBG Publish: {
  "@timestamp": "2016-06-15T16:32:35.218Z",
  "beat": {
    "hostname": "225f1d5b0641",
    "name": "225f1d5b0641"
  },
  "count": 1,
  "fields": {
    "type": "app"
  },
  "input_type": "log",
  "message": "2016-05-30 11:25:53,907 WARN 4209 [DiscoveryClient-InstanceInfoReplicator-0] com.netflix.discovery.InstanceInfoReplicator : There was a problem with the instance info replicator {} com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server\\n\tat com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:111)\\n\tat com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56)\\n\tat com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$1.execute(EurekaHttpClientDecorator.java:59)\\n\tat com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77)\\n\tat com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56)\\n\tat com.netflix.discovery.DiscoveryClient.register(DiscoveryClient.java:806)\\n\tat com.netflix.discovery.InstanceInfoReplicator.run(InstanceInfoReplicator.java:103)\\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)\\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)\\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\\n\tat java.lang.Thread.run(Thread.java:745)\\n",
  "offset": 101843,
  "source": "/tmp/app.log",
  "tags": [
  ],
  "type": "log"
}

```

This is a problem because it's really hard to read one-line stack traces on Kibana.  
Is there a way to tell filebeat to avoid escaping this?

Thank you

---

<div class="post-metadata">

**Author:** ![fiunchinho](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@fiunchinho](https://discuss.elastic.co/u/fiunchinho)\
**Post date:** [June 15, 2016, 5:30pm UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/2 "2016-06-15T17:30:54Z")

</div>

This is my filebeat configuration

```auto
filebeat:
  prospectors:
    -
      paths:
        - "/tmp/access.log"

      encoding: utf-8
      input_type: log
      fields:
        type: access
    -
      paths:
        - "/tmp/*.log"

      encoding: utf-8
      input_type: log
      exclude_files: ["access.log"]
      fields:
        type: app

  config_dir:

output:
  logstash:
    enabled: true
    hosts: ["192.168.99.100:5001"]
    index: logstash

shipper:
  tags: []

logging:
  to_syslog: false
  to_files: false

```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 15, 2016, 6:04pm UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/3 "2016-06-15T18:04:52Z")

</div>

> This is a problem because it's really hard to read one-line stack traces on Kibana.

I'm not sure exactly what you mean. Your logs contain `\n` rather than an actual newline character, yes? And you expect those escape sequences to be transformed into the characters they represent so that the stack trace becomes readable?

> Is there a way to tell filebeat to avoid escaping this?

Your example shows a JSON representation of the event that Filebeat sends to Logstash. The JSON specification _requires_ that backslashes are escaped. When Logstash receives this the JSON string will be deserialized back to `\n` and the `\t` that follows `\\n` in the JSON example will be deserialized to an actual tab character.

---

<div class="post-metadata">

**Author:** ![fiunchinho](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@fiunchinho](https://discuss.elastic.co/u/fiunchinho)\
**Post date:** [June 15, 2016, 6:37pm UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/4 "2016-06-15T18:37:21Z")

</div>

Thank you for your reply.

Yes, you got it. I see the `\n' character literally on Kibana, instead of the actual line break. If I directly query ElasticSearch, I can see that the`\n`character is saved, instead of`\n`, so it's like Logstash is not deserializing it?

This is my Logstash configuration

```auto
input {
  beats {
    port => 5001
  }
}
filter {
    grok {
            match => { "message" => "%{TIMESTAMP_ISO8601:date} %{LOGLEVEL:priority} %{NUMBER:pid} \[%{DATA:thread}\] %{DATA:logger} : %{GREEDYDATA:message})\s+\{%{DATA:mdc}\}\s*(%{GREEDYDATA:stacktrace})" }
            overwrite => ["message"]
            add_tag => ["app_log"]
    }
}

```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 15, 2016, 6:40pm UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/5 "2016-06-15T18:40:02Z")

</div>

Logstash does not turn `\n` sequences into newline characters on its own, so if your logs really contain `\n` sequences you need to put in a filter to transform them. Logstash's configuration language has some issues with escaping so I'd be surprised if you'll be able to use the mutate filter's gsub option for this, but you can do it with a ruby filter.

---

<div class="post-metadata">

**Author:** ![fiunchinho](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@fiunchinho](https://discuss.elastic.co/u/fiunchinho)\
**Post date:** [June 15, 2016, 6:42pm UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/6 "2016-06-15T18:42:15Z")

</div>

Thanks again.

Any link where to find out how to go about this ruby filter?

---

<div class="post-metadata">

**Author:** ![fiunchinho](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@fiunchinho](https://discuss.elastic.co/u/fiunchinho)\
**Post date:** [June 15, 2016, 7:32pm UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/7 "2016-06-15T19:32:13Z")

</div>

With

```auto
ruby {
                code => "
                     temp = event['stacktrace']
                     event['stacktrace'] = temp.gsub(/\\n/,'\n')
                "
}

```

I get

```auto
Cannot execute request on any known server\\n\tat com.netflix.discovery.s...

```

But with

```auto
ruby {
                code => "
                     temp = event['stacktrace']
                     event['stacktrace'] = temp.gsub(/\\n/,'n')
                "
}

```

I get

```auto
Cannot execute request on any known servern\tat com.netflix.discovery.s...

```

I don't understand 😖

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 16, 2016, 6:26am UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/8 "2016-06-16T06:26:25Z")

</div>

Yeah, this might be tricky given how Logstash deals with string escaping. Try this:

```
event['stacktrace'] = event['stacktrace'].gsub(92.chr + 'n', 10.chr)

```

(You don't need a temporary variable.)

---

<div class="post-metadata">

**Author:** ![fiunchinho](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@fiunchinho](https://discuss.elastic.co/u/fiunchinho)\
**Post date:** [June 16, 2016, 9:18am UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/9 "2016-06-16T09:18:46Z")

</div>

It works!

Thank you for all your help @magnusbaeck

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2016, 5:22pm UTC](https://discuss.elastic.co/t/filebeat-escaping-backslashes-on-its-own/52899/10 "2016-07-06T17:22:21Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.