# Filebeat integration as file

**URL:** https://discuss.elastic.co/t/filebeat-integration-as-file/279990
**Category:** Logstash
**Created:** [July 29, 2021, 1:34pm UTC](https://discuss.elastic.co/t/filebeat-integration-as-file/279990 "2021-07-29T13:34:33Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![doragon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/doragon/32/92324_2.png) [@doragon](https://discuss.elastic.co/u/doragon)
#### Post date: [July 29, 2021, 1:34pm UTC](https://discuss.elastic.co/t/filebeat-integration-as-file/279990/1 "2021-07-29T13:34:33Z")

</div>

Good afternoon.

I tried to setup a disconnected architecture between production and monitoring, as I have non-connectivity between my production servers and my ELK stack (local network).

I planed to work as follow:

1. filebeat process relevant logs and save it (output plugin) as a file
2. file is synced with the ELK server
3. logstash input filebeat logs

as per elastic documentation, such an approach seems possible ([Configure the File output | Filebeat Reference [7.13] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/file-output.html))

However, I failed to configure logstatsh correctly:  
using the file input plugin (tail mode) makes the process fine BUT beat logs are seen as simple logs and thus logstash puts the filebeat logs as "message" instead of considering them as it would with the beat input plugin.

Is it possible to prevent this from happening?

---

<div class="post-metadata">

### Author: ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)
#### Post date: [July 30, 2021, 2:03pm UTC](https://discuss.elastic.co/t/filebeat-integration-as-file/279990/2 "2021-07-30T14:03:31Z")

</div>

Yes, you need to define the [JSON codec](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-json.html) as your file input in logstash OR use the [JSON filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html) in your logstash configuration and target the `message` field as the `source`

---

<div class="post-metadata">

### Author: ![doragon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/doragon/32/92324_2.png) [@doragon](https://discuss.elastic.co/u/doragon)
#### Post date: [August 2, 2021, 7:25am UTC](https://discuss.elastic.co/t/filebeat-integration-as-file/279990/3 "2021-08-02T07:25:30Z")

</div>

Hi AquaX!

indeed, I did not check the format was JSON .. so it shall work as for ModSecurity log integration.

But it seems there is one core difference between both log files:

- ModSecurity does not contains a "top field" "message" for each JSON entry and works fine
- Filebeat output contains a "top field" "message" for each JSON entry.

if

- I put a JSON filter with "message" at source, as for ModSecurity, it tries to parse message fields, which is a simple string
- I only have JSON codec, it fails with "Can't get text on a START\_OBJECT" errors

(having JSON filter without source fails, but that is expected).

I will try to review this but thanks for the initial input

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 30, 2021, 7:25am UTC](https://discuss.elastic.co/t/filebeat-integration-as-file/279990/4 "2021-08-30T07:25:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
