# About Beats Input

**URL:** <https://discuss.elastic.co/t/about-beats-input/330322>\
**Category:** Logstash\
**Created:** [April 19, 2023, 3:05pm UTC](https://discuss.elastic.co/t/about-beats-input/330322 "2023-04-19T15:05:39Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [April 19, 2023, 3:05pm UTC](https://discuss.elastic.co/t/about-beats-input/330322/1 "2023-04-19T15:05:39Z")

</div>

Hi,  
We occasionally receive log lines like this at the beats input:

`{"level":"info","ts":1681833658.6787357,"caller":"filterprocessor@v0.75.0/traces.go:74","msg":"Span filter configured","env":"","hostname":"tst","tags":["bad_data"],"kind":"processor","name":"xxxxxx","pipeline":"traces","[Include] match_type":"[None]","[Exclude] match_type":"strict"}`

And the input fails with an invalid field reference for "[Include] match\_type". Is there documentation to explains what is acceptable or not at the input as this log line is valid json.

Thx  
D

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 19, 2023, 3:36pm UTC](https://discuss.elastic.co/t/about-beats-input/330322/2 "2023-04-19T15:36:02Z")

</div>

> [@dawiro](#):
>
> Is there documentation to explains what is acceptable or not at the input as this log line is valid json.

I cannot answer that question, but [this](https://github.com/elastic/logstash/issues/11608) github issue discusses the issue and links to at least one PR that tries to address it.

---

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [April 19, 2023, 6:16pm UTC](https://discuss.elastic.co/t/about-beats-input/330322/3 "2023-04-19T18:16:40Z")

</div>

That's interesting though its focus is on the json filter plugin. It seems to me that this is actually an upstream issue with the beats (filebeat) client. It should not be parsing and passing json that will break the downstream beats input. When this happens it cripples throughput.

To be clear, this issue is happening at the input. It never reaches the filter stage.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 19, 2023, 7:23pm UTC](https://discuss.elastic.co/t/about-beats-input/330322/4 "2023-04-19T19:23:11Z")

</div>

> [@dawiro](#):
>
> its focus is on the json filter plugin

Not really, Colin clearly understands that it affects any input/filter that parses valid JSON to create a structure that cannot be encoded into a logstash event.

The PR that it links to makes changes way down in ConvertMap.java, so I would expect that to fix things for anything that creates fields on events.

Try setting config.field\_reference.escape\_style and see.

---

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [April 20, 2023, 8:13am UTC](https://discuss.elastic.co/t/about-beats-input/330322/5 "2023-04-20T08:13:55Z")

</div>

Yes. I see this has a fuller scope that I assumed. Seems like what I need.

---

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [April 20, 2023, 2:40pm UTC](https://discuss.elastic.co/t/about-beats-input/330322/6 "2023-04-20T14:40:54Z")

</div>

OK. Well I tried it:

```auto
config.field_reference.escape_style: percent

```

And it didn't work. Logstash version is 8.4.3.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 18, 2023, 2:41pm UTC](https://discuss.elastic.co/t/about-beats-input/330322/7 "2023-05-18T14:41:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
