"fields" key show date-type fields in JSON view

[talalatpixlee]

This is just an inquiry. In the discover view when i try and look at the raw json for a document, I see the keys _id, _type, _index, _version, _score and _source which are all expected. I see an additional field called "fields" with all the date type fields in the source document. Why is kibana showing these and what is the purpose ?

[flash1293 (Joe Reuter)]

That's a great question @talalatpixlee. These are shown here because they are returned from Elasticsearch for technical reasons - the "raw json" view does just that - showing exactly what got returned from Elasticsearch.

The underlying question here is - why is Kibana even requesting these?

If you check out the request discover is doing to fetch this data (by clicking the "Inspect" button in the top nav), you will see there are all date fields listed out in the docvalue_fields section. This means that Kibana is fetching those values from the indexed version Elasticsearch is using under the hood for searches and aggregations, not directly from the source JSON which got ingested into Elasticsearch originally. The reason for this is simple: Elasticsearch accepts tons of different date formats. If Kibana would rely on the source for dates, it would need to do the same parsing of dates Elasticsearch is doing - a difficult task which is prone to bugs. By using the normalized doc values (they are always formatted the same way), this job becomes much easier and dates can be shown in the user specified format and in the right time zone for the current user.

[talalatpixlee]

Thanks a lot for the response. It helps a lot. For anyone else digging into this I am leaving this link here https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html