# How to conditionaly set type of event

**URL:** <https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906>\
**Category:** Logstash\
**Created:** [December 1, 2017, 10:01am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906 "2017-12-01T10:01:50Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![rinmalavi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rinmalavi/32/24876_2.png) [@rinmalavi](https://discuss.elastic.co/u/rinmalavi)\
**Post date:** [December 1, 2017, 10:01am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906/1 "2017-12-01T10:01:50Z")

</div>

I have more than one type of document per file.  
I want to set them apart with different types.

> [@How to conditionally set the type?](https://discuss.elastic.co/t/how-to-conditionally-set-the-type/32862/5):
>
> I understand thanks. I set type following your example and I end up in ES with a field type (in \_source) and a field \_type in the meta data. They both have the same value, the one I used when setting typein logstash. Should I deduce from this that the field called type in logstash is treated in a special way by the elasticsearch output plugin - by assuming that its value should also be used to populate \_type before sending to ES ?

This topic lead me to writing :

```
replace => { "type" => "%{log_type}"}

```

"log\_type" was added in grok and separates names few events.

What I get is:

a warning

[2017-12-01T09:52:28,089][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"logstash-2017.12.01", :\_type=\>"My\_log\_type", :\_routing=\>nil}, #LogStash::Event:0x5acfcebf], :response=\>{"index"=\>{"\_index"=\>"logstash-2017.12.01", "\_type"=\>"My\_log\_type", "\_id"=\>"z7Z9EWABqNc\_5GGY1bF0", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Rejecting mapping update to [logstash-2017.12.01] as the final mapping would have more than 1 type: [%{log\_type}, My\_log\_type]"}}}}

And in elastic my event has \_type and type "%{log\_type}" as an actual value.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 10:04am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906/2 "2017-12-01T10:04:21Z")

</div>

Elasticsearch 6.0 allows only one type per index, so setting different types will no longer work.

---

<div class="post-metadata">

**Author:** ![rinmalavi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rinmalavi/32/24876_2.png) [@rinmalavi](https://discuss.elastic.co/u/rinmalavi)\
**Post date:** [December 1, 2017, 10:07am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906/3 "2017-12-01T10:07:00Z")

</div>

Should I then set different indexes from this file. Or just have one file generate one type with all fields that are in all types of events?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 10:08am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906/4 "2017-12-01T10:08:29Z")

</div>

That depends on y our data. If you do not have any mapping conflicts, you can store it all in one index. If you have mapping conflicts you may need to split data into multiple indices.

---

<div class="post-metadata">

**Author:** ![rinmalavi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rinmalavi/32/24876_2.png) [@rinmalavi](https://discuss.elastic.co/u/rinmalavi)\
**Post date:** [December 1, 2017, 10:36am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906/5 "2017-12-01T10:36:21Z")

</div>

Thanks!

I will work with different indexes just for separation sake.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 10:38am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906/6 "2017-12-01T10:38:59Z")

</div>

If you go down that route, make sure you do not end up with a large number of very small indices/shards, [as this can be very inefficient](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster).

---

<div class="post-metadata">

**Author:** ![rinmalavi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rinmalavi/32/24876_2.png) [@rinmalavi](https://discuss.elastic.co/u/rinmalavi)\
**Post date:** [December 1, 2017, 10:52am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906/7 "2017-12-01T10:52:47Z")

</div>

I will keep that in mind.  
It probably wont go far.

In my case indices will be weekly with 3 week retention and 8 types at most.  
I will review this in few weeks.

Thanks for your guidance.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 29, 2017, 10:52am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906/8 "2017-12-29T10:52:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
