# Is setting \`document\_type\` on filebeat a bad idea? Should Logstash be grok'ing and setting it?

**URL:** <https://discuss.elastic.co/t/is-setting-document-type-on-filebeat-a-bad-idea-should-logstash-be-groking-and-setting-it/58714>\
**Category:** Logstash\
**Created:** [August 23, 2016, 4:10pm UTC](https://discuss.elastic.co/t/is-setting-document-type-on-filebeat-a-bad-idea-should-logstash-be-groking-and-setting-it/58714 "2016-08-23T16:10:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![predominant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/predominant/32/11542_2.png) [@predominant](https://discuss.elastic.co/u/predominant)\
**Post date:** [August 23, 2016, 4:10pm UTC](https://discuss.elastic.co/t/is-setting-document-type-on-filebeat-a-bad-idea-should-logstash-be-groking-and-setting-it/58714/1 "2016-08-23T16:10:10Z")

</div>

I'm finalising a deployment for a tenant based system whereby there are various systems pushing logs through logstash.

Having filebeat set document\_type based on prospectors seems like a bad idea. What if people don't set this correctly, or make some mistake? Is it overly taxing to have logstash handle this, adding the document\_type as necessary, and relying on grok's to handle the heavy lifting?

I feel like leaving the responsibility of setting document\_type to filebeat is a bad idea.  
Would love to hear peoples opinions.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 23, 2016, 5:53pm UTC](https://discuss.elastic.co/t/is-setting-document-type-on-filebeat-a-bad-idea-should-logstash-be-groking-and-setting-it/58714/2 "2016-08-23T17:53:33Z")

</div>

> Having filebeat set document\_type based on prospectors seems like a bad idea. What if people don't set this correctly, or make some mistake?

Then they are going to have a bad time and will probably discover their mistake pretty soon.

> Is it overly taxing to have logstash handle this, adding the document\_type as necessary, and relying on grok's to handle the heavy lifting?

Depending on how you write your filters it may cost more to attempt to autodetect the kind of event.

The main reason I require log client to pre-declare the type of events they're sending is because it makes it easier to write the filters and I never have to worry about making sure logs can be unambiguously parsed.

What if you have two clients sending JSON logs, for example? It's trivial to detect that a log most likely is JSON and should be sent through a json filter, but then what? Are you going to look at the presence of certain sentinel fields to determine how to process the event? I'm not saying it's impossible, but I wouldn't want to do it.

---

<div class="post-metadata">

**Author:** ![predominant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/predominant/32/11542_2.png) [@predominant](https://discuss.elastic.co/u/predominant)\
**Post date:** [August 30, 2016, 4:57pm UTC](https://discuss.elastic.co/t/is-setting-document-type-on-filebeat-a-bad-idea-should-logstash-be-groking-and-setting-it/58714/3 "2016-08-30T16:57:31Z")

</div>

Cheers. Thats a perfect answer 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:40am UTC](https://discuss.elastic.co/t/is-setting-document-type-on-filebeat-a-bad-idea-should-logstash-be-groking-and-setting-it/58714/4 "2017-07-06T04:40:51Z")

</div>