# Not able to view json data in available fields for message json
**URL:** https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932
**Category:** Kibana
**Tags:** docker
**Created:** [June 15, 2021, 5:12am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932 "2021-06-15T05:12:46Z")
**Posts on this page:** 15
**Page:** 1
### Author:  [@Harshad\_Solanki](https://discuss.elastic.co/u/Harshad_Solanki)
#### Post date: [June 15, 2021, 5:12am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/1 "2021-06-15T05:12:46Z")
In my Kibana dashboard I have below json data
"message": "{ "time": "2021-06-15T08:57:20+00:00", "remote\_addr": "1.2.3.4", "the\_real\_ip": "10..2.3.4", "remote\_user": "", "time\_local": "15/Jun/2021:08:57:20 +0000", "body\_bytes\_sent": "1811", "request\_time": "0.117", "upstream\_response\_time": "0.120", "status": "200", "request": "POST /api/HTTP/1.1", "request\_method": "POST", "http\_referer": "", "hostname": "api-controller", "http\_user\_agent": "", "proxy\_protocol\_addr": "", "x-forward-for": "10.4.1.57", "request\_id": "abc04c4a2d","bytes\_sent": 2732,"vhost": "api", "request\_proto": "HTTP/1.1" }",
I want above data of message to be in available fields column which is present at left side of Kibana dashboard . All fields like time , remote\_addr,remote\_user should be in available fields column/tab. Any help how can this be done ?
I have below configured in filbeat config map file :
```auto
- type: log
paths:
- /var/lib/docker/containers/*/*.log
json.message_key: message
json.keys_under_root: true
processors:
- add_kubernetes_metadata:
in_cluster: true
- decode_json_fields:
fields: ["message"]
target: ""
```
But still not able to get all key values of message json in discover .
---
### Author:  [@matw](https://discuss.elastic.co/u/matw)
#### Post date: [June 15, 2021, 8:48am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/2 "2021-06-15T08:48:35Z")
Hi
Could you share the mapping of your Data? Which version of Kibana are you using? Which fields are displayed in the sidebar?
Thx & Best,
Matthias
---
### Author:  [@Harshad\_Solanki](https://discuss.elastic.co/u/Harshad_Solanki)
#### Post date: [June 15, 2021, 9:02am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/3 "2021-06-15T09:02:45Z")
Hi Matthias,
Kibana version is 7.3.2. The rest of fields which are visible are agentid , time etc.
None of fields of message json are visible In the sidebar.
The above data is than passed to ES . Please let me know if you need any more inputs.
Any help would be appreciated.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 15, 2021, 9:13am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/4 "2021-06-15T09:13:06Z")
Can you show a sample document? Does the message field by any chance contain a string that contain a serialised JSON document?
---
### Author:  [@Harshad\_Solanki](https://discuss.elastic.co/u/Harshad_Solanki)
#### Post date: [June 15, 2021, 9:17am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/5 "2021-06-15T09:17:31Z")
Hi Christian,
I have updated my question does that help ? I have other fields as well apart from message like agent\_id , time, stream but they all are visible in discover tab. Only message json key and its values are not visible in discover tab
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 15, 2021, 9:25am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/6 "2021-06-15T09:25:39Z")
As you can see the message field is indeed a string so that means the fields within that string have not been indexed and therefore does not show up in the sidebar.
---
### Author:  [@Harshad\_Solanki](https://discuss.elastic.co/u/Harshad_Solanki)
#### Post date: [June 15, 2021, 9:28am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/7 "2021-06-15T09:28:19Z")
okay so what should be the next step ? can you please guide over here
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 15, 2021, 9:33am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/8 "2021-06-15T09:33:29Z")
You will need to parse that information out before indexing it, e.g. through an index pipeline or Logstash. You will then need to run an update by query with an infest pipeline to update data already indexed.
---
### Author:  [@Harshad\_Solanki](https://discuss.elastic.co/u/Harshad_Solanki)
#### Post date: [June 15, 2021, 9:35am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/9 "2021-06-15T09:35:20Z")
I am new to this but will try any reference if you have please do share.
Thanks for the help.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 15, 2021, 9:36am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/10 "2021-06-15T09:36:54Z")
Where is your data coming from? Filebeat?
---
### Author:  [@Harshad\_Solanki](https://discuss.elastic.co/u/Harshad_Solanki)
#### Post date: [June 15, 2021, 9:40am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/11 "2021-06-15T09:40:35Z")
yes from filbeat and than we pass it to ES . In my question I have shared a snippet which I was trying in filebeat config file but didn't got any output.
---
### Author:  [@matw](https://discuss.elastic.co/u/matw)
#### Post date: [June 16, 2021, 6:32am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/12 "2021-06-16T06:32:29Z")
You could use the grok processor in an ingest pipeline for this case, have a look:
> **[Grok processor | Elasticsearch Guide \[master\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/grok-processor.html)**
---
### Author:  [@matw](https://discuss.elastic.co/u/matw)
#### Post date: [June 16, 2021, 6:34am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/13 "2021-06-16T06:34:55Z")
Here's more about ingest pipelines:
> **[Ingest pipelines | Elasticsearch Guide \[master\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html)**
But of course, like @Christian_Dahlqvist mentioned, Logstash ([Grok filter plugin | Logstash Reference [7.13] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html)) would also be an option, but of course, this would mean to install & use another service. while ingest pipelines are a part of Elasticsearch
---
### Author:  [@Harshad\_Solanki](https://discuss.elastic.co/u/Harshad_Solanki)
#### Post date: [June 18, 2021, 8:43am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/14 "2021-06-18T08:43:32Z")
Hi All,
I have created a local setup of ELK version 6.3.2. can anyone please help to fetch fields present in log json at left side i.e under available fields.
I want the fields present inside log json to be under available field section.
I have tried many things but yet no solution. Will attach all required logstash conf file and Filebeat prospector as well. Also the log file which I am parsing.
Can someone pls check and let me know if anything needs to be updated.
Logstash.conf file
input {
beats {
port =\> "5044"
}
}
filter {
json {
source =\> "message"
}
}
output {
elasticsearch {
hosts =\> ["elasticsearch:9200"]
}
}
File prospectors file
#This prospector captures the docker logs
- type: log
paths:
- /var/lib/docker/containers/test.log
test.log contents
{"log":"{ "time": "2021-06-17T08:40:04+00:00", "remote\_addr": "1.24.1.9", "the\_real\_ip": "0.4.1.9", "remote\_user": "", "time\_local": "17/Jun/2021:08:40:04 +0000", "body\_bytes\_sent": "2809", "request\_time": "0.007", "upstream\_response\_time": "0.008", "status": "200", "request": "GET /api/v1/p/prometheus HTTP/1.1", "request\_method": "GET", "http\_referer": "", "hostname": "api-controller-b7469b79-x9rzm", "http\_user\_agent": "Prometheus/2.13.0", "proxy\_protocol\_addr": "", "x-forward-for": "10.24.4.19, 10.24.1.19", "request\_id": "ttt","bytes\_sent": 3927,"vhost": "[ttu.net](http://ttu.net)", "request\_proto": "HTTP/1.1", "path": "/api/v1/party/actuator/prometheus", "request\_query": "", "request\_length": 597, "duration": 0.007,"proxy\_upstream\_name": "pay-9000", "upstream\_addr": "10.4.1.1:000", "upstream\_response\_length": "13203", "upstream\_response\_time": "0.008", "upstream\_status": "200", "http\_x\_forwarded\_for": "10.244.1.149" }\n","stream":"stdout","time":"2021-06-17T08:40:04.040356146Z"}
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 16, 2021, 8:44am UTC](https://discuss.elastic.co/t/not-able-to-view-json-data-in-available-fields-for-message-json/275932/15 "2021-07-16T08:44:07Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.