# New collectd types do not show up in logstash/ES

**URL:** <https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727>\
**Category:** Logstash\
**Created:** [February 25, 2016, 3:03pm UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727 "2016-02-25T15:03:31Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\
**Post date:** [February 25, 2016, 3:03pm UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/1 "2016-02-25T15:03:31Z")

</div>

I've created a collectd exec plugin with new collectd data types to send to logstash+ES, I get the ES documents but amputated without the new data type fields and wonder why. Hints appreciated, TIA.

> # new collectd type:
> 
> $ grep haproxy /usr/share/collectd/types.db  
> haproxy\_backend quecur:COUNTER:0:4294967295,sescur:COUNTER:0:4294967295,sestot:COUNTER:0:4294967295

I expect collectd picks up it's new type correct as I've logged it's output to csv as well and find new multi value type like these:

> epoch,quecur,sescur,sestot  
> 1456412134,0,0,17  
> 1456412144,0,0,17

created from exec plugin by:

> PUTVAL /haproxy/haproxy\_backend-be-web 1456409997:0:0:107

but I see logstash only receiving this from collectd:

> {  
> "host" =\> "hapA",  
> "@timestamp" =\> "2016-02-25T14:39:32.000Z",  
> "plugin" =\> "haproxy",  
> "collectd\_type" =\> "haproxy\_backend",  
> "type\_instance" =\> "be-web",  
> "@metadata" =\> {  
> "input-collectd" =\> "true",  
> "output-esmx" =\> "true",  
> "output-stdout" =\> "false",  
> "output-debug" =\> "true",  
> "esindex" =\> "collectd"  
> }  
> }

Do I need to patch collectd codec somehow to accept new types?

> input {

> udp {  
> port =\> 25826 # 25826 matches port specified in collectd.conf  
> buffer\_size =\> 8192 # 8192 as we support jumboframes upto 9000  
> codec =\> collectd { } # specific Collectd codec to invoke  
> add\_field =\> {'[@metadata][input-collectd]' =\> 'true' '[@metadata][output-es  
> mx]' =\> 'true' '[@metadata][output-stdout]' =\> 'false' }  
> }

---

<div class="post-metadata">

**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\
**Post date:** [February 25, 2016, 3:20pm UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/2 "2016-02-25T15:20:12Z")

</div>

seems it might be fine to sync logstash's type db with end point collectors new data type into here:

/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-collectd-2.0.2/vendor/types.db

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 25, 2016, 5:34pm UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/3 "2016-02-25T17:34:11Z")

</div>

You are correct! You would need to add or update custom types in the `types.db` file. I'd recommend copying that one out to a known location, and edit that version. You can use the [typesdb](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-collectd.html#plugins-codecs-collectd-typesdb) directive in the codec settings to specify the location.

---

<div class="post-metadata">

**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\
**Post date:** [February 26, 2016, 8:08am UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/4 "2016-02-26T08:08:53Z")

</div>

Thanks, do I need all existing types in an element path of typesdb or just add my few custom types to a known path and just add this path to typesdb, ie. will defining typesdb loose default types or will they still be read/known?

---

<div class="post-metadata">

**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\
**Post date:** [February 26, 2016, 8:42am UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/5 "2016-02-26T08:42:40Z")

</div>

Seems I need supply all default types as well

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 26, 2016, 7:52pm UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/6 "2016-02-26T19:52:35Z")

</div>

That is why I suggested copying the source, and editing that 🙂

---

<div class="post-metadata">

**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\
**Post date:** [February 26, 2016, 8:07pm UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/7 "2016-02-26T20:07:34Z")

</div>

Yeap or add an extra element more to typesdb array pointing to default typesdb path... only it' in a versioned path, so an update of this plugin might remove such a path. But yes I copying all default type patterns to my own path and added custom types.

Thanks for point out!

---

<div class="post-metadata">

**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\
**Post date:** [February 26, 2016, 8:11pm UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/8 "2016-02-26T20:11:49Z")

</div>

Guess a RFE for collectd input filter plugin could be to always read in std default typesdb first regardless of an optional typesdb array given thus one only needs to specific custom types and add this path only... just a $.02

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 26, 2016, 8:25pm UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/9 "2016-02-26T20:25:17Z")

</div>

> [@stefws](#):
>
> Guess a RFE for collectd input filter plugin could be to always read in std default typesdb first regardless of an optional typesdb array given

This is problematic in the case that you have _overridden_ one of those values for some reason. This is why it's not done. What _might_ work is to add an array element of `default` or something like that first, and have the codec interpret that to use the included one first. You could always submit an issue or pull request with that feature at [GitHub - logstash-plugins/logstash-codec-collectd](https://github.com/logstash-plugins/logstash-codec-collectd)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:09am UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727/10 "2017-07-06T05:09:32Z")

</div>


