# Can custom created logs from Logstash be shipped to ES using Filebeat?

**URL:** https://discuss.elastic.co/t/can-custom-created-logs-from-logstash-be-shipped-to-es-using-filebeat/100117
**Category:** Beats
**Tags:** filebeat
**Created:** [September 11, 2017, 6:59pm UTC](https://discuss.elastic.co/t/can-custom-created-logs-from-logstash-be-shipped-to-es-using-filebeat/100117 "2017-09-11T18:59:50Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![NandhaKrishna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nandhakrishna/32/26286_2.png) [@NandhaKrishna](https://discuss.elastic.co/u/NandhaKrishna)
#### Post date: [September 11, 2017, 6:59pm UTC](https://discuss.elastic.co/t/can-custom-created-logs-from-logstash-be-shipped-to-es-using-filebeat/100117/1 "2017-09-11T18:59:50Z")

</div>

I'm not able to GROK a custom log from logstash to ES using FIlebeat. Everytime i create a grok im only able to see the fields that are default in Filebeat. The fields that I grok from the message are not seen.

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [September 11, 2017, 7:11pm UTC](https://discuss.elastic.co/t/can-custom-created-logs-from-logstash-be-shipped-to-es-using-filebeat/100117/2 "2017-09-11T19:11:40Z")

</div>

Can you please provide more information about your setup and logs you are trying to grok.

What error you are seeing?

Please share the config files you are using, sample log messages, and software versions (filebeat, logstash, and elasticsearch).

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [September 11, 2017, 7:34pm UTC](https://discuss.elastic.co/t/can-custom-created-logs-from-logstash-be-shipped-to-es-using-filebeat/100117/4 "2017-09-11T19:34:25Z")

</div>

The message you are trying to grok is JSON so it would be easiest to use a json filter.

```auto
filter {
  json {
    source => "message"
  }
}

```

If you are reading the log file using Filebeat you could do the json decoding there instead using the logstash filter. See [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#config-json](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#config-json)

```auto
filebeat.prospectors:
- paths: [mylog.json]
  json.keys_under_root: true
  json.add_error_key: true
```

---

<div class="post-metadata">

### Author: ![NandhaKrishna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nandhakrishna/32/26286_2.png) [@NandhaKrishna](https://discuss.elastic.co/u/NandhaKrishna)
#### Post date: [September 11, 2017, 8:05pm UTC](https://discuss.elastic.co/t/can-custom-created-logs-from-logstash-be-shipped-to-es-using-filebeat/100117/5 "2017-09-11T20:05:45Z")

</div>

Thank you so much for the quick reply! Just a question, This means that I have to replace the json filter instead of the grok to parse the data right?

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [September 11, 2017, 8:06pm UTC](https://discuss.elastic.co/t/can-custom-created-logs-from-logstash-be-shipped-to-es-using-filebeat/100117/6 "2017-09-11T20:06:36Z")

</div>

Correct, replace the grok filter with the json filter.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 9, 2017, 8:06pm UTC](https://discuss.elastic.co/t/can-custom-created-logs-from-logstash-be-shipped-to-es-using-filebeat/100117/7 "2017-10-09T20:06:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
