# Help on showing Custom File beat Module Fields in Kibana

**URL:** https://discuss.elastic.co/t/help-on-showing-custom-file-beat-module-fields-in-kibana/161351
**Category:** Beats
**Tags:** filebeat
**Created:** [December 18, 2018, 3:12pm UTC](https://discuss.elastic.co/t/help-on-showing-custom-file-beat-module-fields-in-kibana/161351 "2018-12-18T15:12:26Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![CoreyF](https://avatars.discourse-cdn.com/v4/letter/c/65b543/32.png) [@CoreyF](https://discuss.elastic.co/u/CoreyF)
#### Post date: [December 18, 2018, 3:12pm UTC](https://discuss.elastic.co/t/help-on-showing-custom-file-beat-module-fields-in-kibana/161351/1 "2018-12-18T15:12:26Z")

</div>

Hello, I have created a custom File Beat Module using [Creating a New Beat](https://www.elastic.co/guide/en/beats/devguide/current/new-beat.html).  
The data is making it to Elasticsearch and I can see it under the “Discover” Tab in Kibana. In Kibana -\> Visualize -\> Visual Builder -\> Time Series I can see the “Count” for the number of records in the index I created with the Custom File Beat. But I cannot see my custom Fields.

Filebeat.yml Config:

```auto
#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /ixialogs/*_WebStatic.log
#============================= Filebeat modules ===============================
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
#==================== Elasticsearch template setting ==========================
setup.template.settings:
  index.number_of_shards: 3
#================================ General =====================================
fields:
   service: WebStatic
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  hosts: ["100.68.2.101:9200"]
  index: "ixia-%{+yyyy}"
  pipeline: WebStatic
#================================ Procesors =====================================
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
#================================ Logging =====================================
setup.template.name: "filebeat-%{[beat.version]}"
setup.template.pattern: "filebeat-%{[beat.version]}-*"
```

Screen Shot of Discover Tab showing records and fields:  
[![](https://i.ibb.co/G7jPB0t/Discover.jpg)](https://ibb.co/hsv8hyZ)

Screen Shot of Visualize showing records in index:  
[![](https://i.ibb.co/qBrXHD2/Visualize-Just-records-in-index.jpg)](https://ibb.co/fMxB3XT)

Screen Shot of Fields not showing:  
[![](https://i.ibb.co/rs1gn6h/Visualize-Fields-not-showing-up.jpg)](https://ibb.co/tXWSys6)

What is my next step to get the Custom Fields to show up in Visualize Graphs?  
Do I need to do the Create a New Dashboard section or did I miss something?

Thanks for the help!  
Corey

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [December 27, 2018, 1:15pm UTC](https://discuss.elastic.co/t/help-on-showing-custom-file-beat-module-fields-in-kibana/161351/2 "2018-12-27T13:15:38Z")

</div>

Do you have a module definition?

If you have a module collecting the data, then do not configure the input and do not configure `output.elasticsearch.pipeline`. It is the module which has this knowledge and configures those.

The field not being shown in the listing looks like a kibana issue. I See you are using the time series visual builder. Have you tried other visualizations (e.g. normal line graph)?

Also double check the kibana index pattern for filebeat under the `Management` tab. The way your filebeat configuration looks like I'm not sure a module is really used. W

Without module and extra fields definitions of the IXIA fields you are missing the field names and types from the template mapping (that is Elasticsearch auto-derives the types for these unknown fields) and you are missing the field names in the kibana index pattern. If that's the case you can try to refresh the index pattern in the kibana `Management` UI, so kibana internally learns about these fields.

---

<div class="post-metadata">

### Author: ![CoreyF](https://avatars.discourse-cdn.com/v4/letter/c/65b543/32.png) [@CoreyF](https://discuss.elastic.co/u/CoreyF)
#### Post date: [January 22, 2019, 7:15pm UTC](https://discuss.elastic.co/t/help-on-showing-custom-file-beat-module-fields-in-kibana/161351/3 "2019-01-22T19:15:53Z")

</div>

Hello, You were correct. I did not have the module loading correctly. I have seen done some re-configuration and testing. Now I believe the Custom Module is loading correctly.

Here is the new "/etc/filebeat/filebeat.yml" file.

```auto
filebeat.inputs:
- type: log
  enabled: false
  paths:
    - /var/log/*.log
filebeat.config.modules:
  enabled: true
  path: ${path.config}/modules.d/*.yml
reload.enabled: false
filebeat.modules:
- module: ixia
setup.template.settings:
  index.number_of_shards: 3
setup.kibana:
  host: "100.68.2.101:5601"
output.elasticsearch:
  hosts: ["100.68.2.101:9200"]
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
```

But I'm still having the same problem.

> [@steffens](#):
>
> Have you tried other visualizations (e.g. normal line graph)?

Yes, I have tried the other visualizations such as line graphs.

Above you mention:

> [@steffens](#):
>
> (that is Elasticsearch auto-derives the types for these unknown fields)

So does that mean there is nothing else that I'm missing in the config setup to create the custom fields to show up in the Kibana Visualization graphs?

Yes, I have refreshed the index pattern in the Kibana Management UI with no luck.

Is there any debug log files that would help?

Thank you....Corey

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [January 23, 2019, 11:44am UTC](https://discuss.elastic.co/t/help-on-showing-custom-file-beat-module-fields-in-kibana/161351/4 "2019-01-23T11:44:00Z")

</div>

For logging see: [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-logging.html](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-logging.html)

If you run filebeat in debug mode (start with `-d '*'` or set log level to debug and set selectors to ["\*"] in config file), you will see each event in the log to be published by filebeat.

I think you are missing indentation on `reload.enabled: false` . Unrelated to your issue, I just noticed.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 20, 2019, 11:44am UTC](https://discuss.elastic.co/t/help-on-showing-custom-file-beat-module-fields-in-kibana/161351/5 "2019-02-20T11:44:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
