# No custom module fields in Kibana **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [June 25, 2018, 10:09pm UTC](https://discuss.elastic.co/t/no-custom-module-fields-in-kibana/137319 "2018-06-25T22:09:21Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![mvasilenko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvasilenko/32/32886_2.png) [@mvasilenko](https://discuss.elastic.co/u/mvasilenko)\ **Post date:** [June 25, 2018, 10:09pm UTC](https://discuss.elastic.co/t/no-custom-module-fields-in-kibana/137319/1 "2018-06-25T22:09:21Z")
Hello, I'm interested in creating custom Filebeat module, to parse our app log, so I've followed this guide [https://www.elastic.co/elasticon/conf/2018/sf/build-your-own-filebeat-module](https://www.elastic.co/elasticon/conf/2018/sf/build-your-own-filebeat-module), simulated pipeline, grok patterns wokring fine, but no custom fields shows in kibana, what I'm missing here? i'm using filebeat autodiscover feature for docker logs, and other modules, like nginx, working fine, but not mine custom one. ```auto filebeat.autodiscover: providers: - type: docker templates: - condition.or: - contains.docker.container.image: "myimage" config: - module: mymodule access: prospector: type: docker containers.stream: stdout containers.ids: - "${data.docker.container.id}" error: prospector: type: docker containers.stream: stderr containers.ids: - "${data.docker.container.id}" ``` ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [June 26, 2018, 3:05pm UTC](https://discuss.elastic.co/t/no-custom-module-fields-in-kibana/137319/2 "2018-06-26T15:05:24Z")
@mvasilenko Did you try using your module outside the autodiscover context using the log input instead of the docker input? ---
**Author:** ![mvasilenko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvasilenko/32/32886_2.png) [@mvasilenko](https://discuss.elastic.co/u/mvasilenko)\ **Post date:** [June 29, 2018, 6:48am UTC](https://discuss.elastic.co/t/no-custom-module-fields-in-kibana/137319/3 "2018-06-29T06:48:28Z")
@pierhugues thank you for the answer, i've messed with stderr/stdout, but the question remains, what exact actions do i need to perform after generating/editing custom module fields, to load it into kibana/elasticsearch? is it sufficient to update /etc/filebeat/fields.yml and /usr/share/filebeat/module/mymodule? i'm using ubuntu 16 ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [June 29, 2018, 12:48pm UTC](https://discuss.elastic.co/t/no-custom-module-fields-in-kibana/137319/4 "2018-06-29T12:48:49Z")
It should be sufficient to do so. I would still use [this guide](https://www.elastic.co/guide/en/beats/devguide/6.3/filebeat-modules-devguide.html) To make sure you can easily test your new module. ---
**Author:** ![mvasilenko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvasilenko/32/32886_2.png) [@mvasilenko](https://discuss.elastic.co/u/mvasilenko)\ **Post date:** [July 2, 2018, 9:38pm UTC](https://discuss.elastic.co/t/no-custom-module-fields-in-kibana/137319/5 "2018-07-02T21:38:55Z")
@pierhugues ok, i've tested module by adding pipeline to ES `curl -H 'Content-Type: application/json' -XPUT localhost:9200/_ingest/pipeline/filebeat-6.3.0-mymodule-backend -d@/usr/share/filebeat/module/mymodule/backend/ingest/pipeline.json` and simulating in kibana dev tools ```auto POST _ingest/pipeline/filebeat-6.3.0-mymodule-backend/_simulate { "docs" : [ { "_source": { "message": "[2018-07-02 21:06:57 +0000] [INFO] message"} } ] } ``` the output looks good, now my idea is to combine docker autodiscovery feature with my custom module for specific containers, few questions arise - do i need to specify anything in filebeat.yml pipeline section? or it is already applied at autodiscovery/mymodule level - what files need to be copied to remote hosts, in addition to standard filebeat package? ---
**Author:** ![mvasilenko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvasilenko/32/32886_2.png) [@mvasilenko](https://discuss.elastic.co/u/mvasilenko)\ **Post date:** [July 3, 2018, 12:51pm UTC](https://discuss.elastic.co/t/no-custom-module-fields-in-kibana/137319/6 "2018-07-03T12:51:35Z")
Update: looks like adding `filebeat.overwrite_pipelines: true` to `/etc/filebeat/filebeat.yml` fixes this issue another thing i can add - you must to test pipeline like this, not via ES pipeline simulate api ```auto ~/go/src/github.com/elastic/beats/filebeat $ ./scripts/tester/tester -elasticsearch https://elk:9200 \ -pipeline /usr/share/filebeat/module/mymodule/backend/ingest/pipeline.json \ -logfile sample.log --simulate.verbose --verbose ``` Now I'm looking to adopt this model for logging in k8s, any hints? Thank you ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 31, 2018, 1:02pm UTC](https://discuss.elastic.co/t/no-custom-module-fields-in-kibana/137319/7 "2018-07-31T13:02:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.