# Filebeat decoded JSON fields aren't searchable **URL:** \ **Category:** Kibana\ **Created:** [November 23, 2020, 9:36pm UTC](https://discuss.elastic.co/t/filebeat-decoded-json-fields-arent-searchable/256410 "2020-11-23T21:36:24Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![nmoham](https://avatars.discourse-cdn.com/v4/letter/n/77aa72/32.png) [@nmoham](https://discuss.elastic.co/u/nmoham)\ **Post date:** [November 23, 2020, 9:36pm UTC](https://discuss.elastic.co/t/filebeat-decoded-json-fields-arent-searchable/256410/1 "2020-11-23T21:36:24Z")
Sending logs (JSON nested) using filebeat, the fields are getting created in (confirming from index pattern) , but the decoded fields are not visible in Kibana Discover tab. Sample data ```auto {"id":"5f5a0206-c431-4d76-9e8a-06095a2c2317","name":"Mavent-EPPS","date":"2020-11-22 21:55:33 PST","apmModuleDetailViewData":[{"licenseModuleType":"APM","peakUsage":172,"licenseProvisioned":220}],"nonApmModuleDetailViewData":[{"licenseModuleType":"MACHINE_AGENT","peakUsage":31,"licenseProvisioned":220},{"licenseModuleType":"SIM_MACHINE_AGENT","peakUsage":18,"licenseProvisioned":40}],"apmStackGraphViewData":[{"licenseModuleType":"JAVA","peakUsage":142,"licenseProvisioned":0},{"licenseModuleType":"DOT_NET","peakUsage":30,"licenseProvisioned":0}]} ``` Fields getting created - ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/b/5b7bf0efcff6f8dd86f6b093486cb952ace2736c.png) But not searchable ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/5/a54967679addbde9db5879ca4696127faead002f.jpeg) Filebeat Inputs along with processors - ```auto filebeat.inputs: - type: log enabled: true paths: - /opt/logs/LicenseRulesUtil.log #json.keys_under_root: true #json.add_error_key: true #json.overwrite_keys: false processors: - decode_json_fields: fields: ["message"] process_array: true target: "" max_depth: 5 overwrite_keys: true - timestamp: field: date layouts: - '2006-01-02 15:04:05 PST' test: - '2020-11-22 21:55:33 PST' timezone: "America/Los_Angeles" exclude_files: ['.gz$'] ignore_older: 72h fields: name: appd_license_logs index: appd_license environment: prod fields_under_root: true ``` Any help appreciated .. Thanks ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [November 24, 2020, 4:50am UTC](https://discuss.elastic.co/t/filebeat-decoded-json-fields-arent-searchable/256410/2 "2020-11-24T04:50:23Z")
I'm guessing that's because either these fields are not indexed correctly or because the Kibana index pattern needs refreshing. Can you check your Filebeat index mappings for the missing fields and post their mappings here? [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html) Also, could you try refreshing your Filebeat index pattern in Kibana? [https://www.elastic.co/guide/en/kibana/current/index-patterns.html#\_refresh\_the\_data\_fields](https://www.elastic.co/guide/en/kibana/current/index-patterns.html#_refresh_the_data_fields) Shaunak ---
**Author:** ![nmoham](https://avatars.discourse-cdn.com/v4/letter/n/77aa72/32.png) [@nmoham](https://discuss.elastic.co/u/nmoham)\ **Post date:** [November 24, 2020, 5:47am UTC](https://discuss.elastic.co/t/filebeat-decoded-json-fields-arent-searchable/256410/3 "2020-11-24T05:47:35Z")
I tried refreshing the index pattern in Kibana, but it did not help . I also believe the fields are indexed, because those fields appear in the index pattern (I've attached the snapshot earlier). After much trials, I found, when trying to search the data using the search Bar in Discover (Kibana), the fields are usable/searchable. I could also use them in visualizations, but they still appear to not decoded in the Kibana fields list. ---
**Author:** ![nmoham](https://avatars.discourse-cdn.com/v4/letter/n/77aa72/32.png) [@nmoham](https://discuss.elastic.co/u/nmoham)\ **Post date:** [November 24, 2020, 10:31pm UTC](https://discuss.elastic.co/t/filebeat-decoded-json-fields-arent-searchable/256410/4 "2020-11-24T22:31:20Z")
@shaunak Field index Mappings ```auto { "appd_license-prod-2020.11.23" : { "mappings" : { "properties" : { "@timestamp" : { "type" : "date" }, "@version" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "agent" : { "properties" : { "ephemeral_id" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "hostname" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "id" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "type" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "version" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } }, "apmModuleDetailViewData" : { "properties" : { "licenseModuleType" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "licenseProvisioned" : { "type" : "long" }, "peakUsage" : { "type" : "long" } } }, "apmStackGraphViewData" : { "properties" : { "licenseModuleType" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "licenseProvisioned" : { "type" : "long" }, "peakUsage" : { "type" : "long" } } }, "date" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "ecs" : { "properties" : { "version" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } }, "environment" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "host" : { "properties" : { "name" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } }, "id" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "input" : { "properties" : { "type" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } }, "log" : { "properties" : { "file" : { "properties" : { "path" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } }, "offset" : { "type" : "long" } } }, "name" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "nonApmModuleDetailViewData" : { "properties" : { "licenseModuleType" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "licenseProvisioned" : { "type" : "long" }, "peakUsage" : { "type" : "long" } } }, "tags" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "top" : { "properties" : { "ingest_method" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } } } } } } ``` ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [November 25, 2020, 3:10am UTC](https://discuss.elastic.co/t/filebeat-decoded-json-fields-arent-searchable/256410/5 "2020-11-25T03:10:06Z")
Ah, sorry I missed the index patterns screenshot earlier. Hmm, so it looks like the fields are being mapped and the index pattern in Kibana "sees" them too, e.g. `apmModuleDetailViewData.licenseModuleType`, `apmModuleDetailViewData.licenseModuleType.keyword`, etc. I'd expect these to show up in the Discover fields list on the left but I'm not sure why they're not showing up. I wonder if it has something to do with the field _values_ being arrays (not `nested`). I'm going to transfer this post to the Kibana forums so hopefully the right developers can take a look. ---
**Author:** ![nmoham](https://avatars.discourse-cdn.com/v4/letter/n/77aa72/32.png) [@nmoham](https://discuss.elastic.co/u/nmoham)\ **Post date:** [November 25, 2020, 6:34am UTC](https://discuss.elastic.co/t/filebeat-decoded-json-fields-arent-searchable/256410/6 "2020-11-25T06:34:06Z")
Thanks @shaunak. Hope Someone from the team will be able to help and yes, it looks like Array of values, looking deeper into the data, ```auto "apmStackGraphViewData":[{"licenseModuleType":"JAVA","peakUsage":142,"licenseProvisioned":0},{"licenseModuleType":"DOT_NET","peakUsage":30,"licenseProvisioned":0}] ``` While building the visualizations, I realized the data is getting overwritten for the following (maybe) - apmStackGraphViewData.licenseModuleType apmStackGraphViewData.peakUsage apmStackGraphViewData.licenseProvisioned Not sure, how to process and not lose it. Original Event Example - ```auto {"id":"5f5a0206-c431-4d76-9e8a-06095a2c2317","name":"Mavent-EPPS","date":"2020-11-22 21:55:33 PST","apmModuleDetailViewData":[{"licenseModuleType":"APM","peakUsage":172,"licenseProvisioned":220}],"nonApmModuleDetailViewData":[{"licenseModuleType":"MACHINE_AGENT","peakUsage":31,"licenseProvisioned":220},{"licenseModuleType":"SIM_MACHINE_AGENT","peakUsage":18,"licenseProvisioned":40}],"apmStackGraphViewData":[{"licenseModuleType":"JAVA","peakUsage":142,"licenseProvisioned":0},{"licenseModuleType":"DOT_NET","peakUsage":30,"licenseProvisioned":0}]} ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 23, 2020, 8:34am UTC](https://discuss.elastic.co/t/filebeat-decoded-json-fields-arent-searchable/256410/7 "2020-12-23T08:34:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.