# Message Parsing

**URL:** <https://discuss.elastic.co/t/message-parsing/256572>\
**Category:** Logstash\
**Created:** [November 24, 2020, 8:14pm UTC](https://discuss.elastic.co/t/message-parsing/256572 "2020-11-24T20:14:54Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![droidus](https://avatars.discourse-cdn.com/v4/letter/d/13edae/32.png) [@droidus](https://discuss.elastic.co/u/droidus)\
**Post date:** [November 24, 2020, 8:14pm UTC](https://discuss.elastic.co/t/message-parsing/256572/1 "2020-11-24T20:14:54Z")

</div>

Here is what my logstash output looks like:

```auto
output {
  elasticsearch {
    index => "%{[@metadata][beat]}"
    hosts => "192.168.0.103"
  }
}

```

Logstash errors are as follows:

```auto
[2020-11-18T13:08:12,824][WARN][logstash.codecs.jsonlines][main][26da92079e525d4bfdac5a892ff28079c6695bd768a516e8a992f0d588033c05] Received an event that has a different character encoding than you configured. {:text=>"\\u000E\\x97P]...
 [2020-11-18T13:08:12,826][WARN][logstash.codecs.jsonlines][main][26da92079e525d4bfdac5a892ff28079c6695bd768a516e8a992f0d588033c05] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unrecognized token 'z': was expecting ('true', 'false' or 'null')
 at [Source: (String)"z -9\x92\u0001~\u0000/\f\x960l...

```

I added

```auto
codec => plain {
      charset => "ISO-8859-1"
    }

```

But am getting similar error messages:

```auto
JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected character...
Received an event that has a different character encoding than you configured. {:text=>"\\xB6\\xA6}e#\\x

```

---

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [November 26, 2020, 5:56am UTC](https://discuss.elastic.co/t/message-parsing/256572/2 "2020-11-26T05:56:29Z")

</div>

Hi,

I don't think it has something to do with the output - I guess it is connected to the input:

> JSON parse error, original data now in message field

output plugins do not parse the data - they serialize it. Can you show us the complete pipeline?

---

<div class="post-metadata">

**Author:** ![droidus](https://avatars.discourse-cdn.com/v4/letter/d/13edae/32.png) [@droidus](https://discuss.elastic.co/u/droidus)\
**Post date:** [November 30, 2020, 2:29pm UTC](https://discuss.elastic.co/t/message-parsing/256572/3 "2020-11-30T14:29:20Z")

</div>

```auto
input {
    tcp {
    port => 5044
    codec => json
    }
}

filter {
  date {
    match => ["timeMillis", "UNIX_MS"]
  }
}

output {
  elasticsearch {
    index => "%{[@metadata][beat]}"
    hosts => "192.168.0.103"
    codec => plain {
      charset => "ISO-8859-1"
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [December 1, 2020, 6:48am UTC](https://discuss.elastic.co/t/message-parsing/256572/4 "2020-12-01T06:48:53Z")

</div>

Where do you get your data from? According to the character pages:  
\xB6

> The **pilcrow** , **¶** , also called the **paragraph mark** , **paragraph sign** , **paraph** , **alinea** , or **blind P** , is a typographical character marking the start of a paragraph.  
> ]([Pilcrow - Wikipedia](https://en.wikipedia.org/wiki/Pilcrow))

\xA6

> The **vertical bar** , **|** , is a [glyph](https://en.wikipedia.org/wiki/Glyph) with various uses in [mathematics](https://en.wikipedia.org/wiki/Mathematics), [computing](https://en.wikipedia.org/wiki/Computing), and [typography](https://en.wikipedia.org/wiki/Typography). It has many names, often related to particular meanings: [Sheffer stroke](https://en.wikipedia.org/wiki/Sheffer_stroke) (in [logic](https://en.wikipedia.org/wiki/Mathematical_logic)), **verti-bar** , **vbar** , **stick** , **vertical line** , **vertical slash,**  **bar** , **pike** , or **pipe** , and several variants on these names. It is occasionally considered an [allograph](https://en.wikipedia.org/wiki/Allograph) of **broken bar** (see below).

In what character encoding do you receive the data - have you tried setting the encoding on the input instead of the output?

Best regards  
Wolfram

---

<div class="post-metadata">

**Author:** ![droidus](https://avatars.discourse-cdn.com/v4/letter/d/13edae/32.png) [@droidus](https://discuss.elastic.co/u/droidus)\
**Post date:** [December 1, 2020, 1:27pm UTC](https://discuss.elastic.co/t/message-parsing/256572/5 "2020-12-01T13:27:33Z")

</div>

I am sending logs from one of my Linux clients. Here is part of the filebeat.yml from that client:

```auto
- type: log
  # Change to true to enable this input configuration.
  enabled: true
  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/*.log

```

---

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [December 1, 2020, 1:32pm UTC](https://discuss.elastic.co/t/message-parsing/256572/6 "2020-12-01T13:32:25Z")

</div>

> [@droidus](#):
>
> `/var/log/*.log`

Are you sure that the logs are in json format? That would explain the json codec errors...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 29, 2020, 1:32pm UTC](https://discuss.elastic.co/t/message-parsing/256572/7 "2020-12-29T13:32:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.