# Objects showing in Discover as json, not discrete fields

**URL:** https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178
**Category:** Kibana
**Created:** [May 8, 2019, 1:27pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178 "2019-05-08T13:27:23Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![darkmoon](https://avatars.discourse-cdn.com/v4/letter/d/ecd19e/32.png) [@darkmoon](https://discuss.elastic.co/u/darkmoon)
#### Post date: [May 8, 2019, 1:27pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/1 "2019-05-08T13:27:23Z")

</div>

After deleting and recreating my .kibana index to fix the API/FORBIDDEN issue, all of my structured fields are showing at top level json blobs.

Refreshing the index pattern doesn't help.

Mapping:  
` "meta": {  
"dynamic": "true",  
"properties": {  
"delay": {  
"type": "integer"  
},  
"filter": {  
"type": "keyword"  
},  
"filterhost": {  
"type": "keyword"  
},  
"filtertime": {  
"type": "long"  
},

etc...  
`

In Discover:  
'{ "type": "syslog", "target": { "topic": "logqueue-pprd", "index": "log\_syslog-2019.05.07" }, "queuehost": ["logqueue-prod-03", "logqueue-pprd-01"], "queuetime": 1557256848, "filter": ["20-syslog-base", "78-category"], "input": ["beats", "prodsample"], "size": 107, "filtertime": 1557256850, "delay": 8, "filterhost": "logfilter-pprd-02.cas-1.cls.vt.edu" }'

---

<div class="post-metadata">

### Author: ![Nathan\_Reese](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nathan_reese/32/84829_2.png) [@Nathan\_Reese](https://discuss.elastic.co/u/Nathan_Reese)
#### Post date: [May 9, 2019, 5:33pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/2 "2019-05-09T17:33:39Z")

</div>

Can you provide a screen shot. I am not sure what you mean by "all of my structured fields are showing at top level json blobs". Also, could you provide the complete index mapping.

---

<div class="post-metadata">

### Author: ![darkmoon](https://avatars.discourse-cdn.com/v4/letter/d/ecd19e/32.png) [@darkmoon](https://discuss.elastic.co/u/darkmoon)
#### Post date: [May 9, 2019, 6:21pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/3 "2019-05-09T18:21:23Z")

</div>

I've got things disassembled at the moment, but I'll get the files shortly.

By "all of my structured fields are showing at top level json blobs", I mean my top level fields show a single value containing all their sub fields.

In the above example, my 'meta' field contains this blob: '{ "type": "syslog", "target": { "topic": "logqueue-pprd", "index": "log\_syslog-2019.05.07" }, "queuehost": ["logqueue-prod-03", "logqueue-pprd-01"], "queuetime": 1557256848, "filter": ["20-syslog-base", "78-category"], "input": ["beats", "prodsample"], "size": 107, "filtertime": 1557256850, "delay": 8, "filterhost": "logfilter-pprd-02.cas-1.cls.vt.edu" }'

Rather than having a meta.type, meta.target.topic, meta.target.index, meta.queuehost, meta.queuetime, meta.filter, meta.input, meta.size, meta.filtertime, meta.delay, and meta.filterhost fields, it just displays the 'meta' field.

It displays the 'meta' field as an unknown type, but defaults to string, despite having the proper mappings for meta and it's member fields, as shown above.

This has been working for years.

---

<div class="post-metadata">

### Author: ![darkmoon](https://avatars.discourse-cdn.com/v4/letter/d/ecd19e/32.png) [@darkmoon](https://discuss.elastic.co/u/darkmoon)
#### Post date: [May 9, 2019, 7:30pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/4 "2019-05-09T19:30:09Z")

</div>

Here's the screen shot.

 ![Screenshot_20190509](https://us1.discourse-cdn.com/elastic/original/3X/e/9/e924dcc0b19a33dc6b30d8202ca6bcef7b18f606.png)

The index pattern (just the meta fields - note that the sub fields show properly):

 ![Screenshot_20190509_1941](https://us1.discourse-cdn.com/elastic/original/3X/0/e/0e3a578f40c5edbad100d6e655d4a9627330d609.png)

And the relevant part of the mapping (sorry, the forum won't let me post non-image attachments) mapping:  
indent preformatted text by 4 spaces {  
"mapping": {  
"doc": {  
"dynamic\_templates": [  
{  
"string\_fields": {  
"match": "\*",  
"match\_mapping\_type": "string",  
"mapping": {  
"doc\_values": true,  
"index": "true",  
"type": "keyword"  
}  
}  
}  
],  
"properties": {  
"@timestamp": {  
"type": "date"  
},  
"@version": {  
"type": "integer"  
},  
"beat": {  
"properties": {  
"hostname": {  
"type": "keyword"  
},  
"name": {  
"type": "keyword"  
},  
"version": {  
"type": "keyword"  
}  
}  
},  
"category": {  
"type": "keyword"  
},  
"event": {  
"dynamic": "true",  
"properties": {  
"hostname": {  
"type": "keyword"  
},  
"id": {  
"type": "keyword"  
},  
"level": {  
"type": "keyword"  
},  
"message": {  
"type": "keyword"  
},  
"pid": {  
"type": "keyword"  
},  
"program": {  
"type": "keyword"  
},  
"reason": {  
"type": "keyword"  
}  
}  
},  
"fields": {  
"properties": {  
"group": {  
"type": "keyword"  
},  
"host": {  
"type": "keyword"  
},  
"json": {  
"type": "boolean"  
},  
"name": {  
"type": "keyword"  
},  
"tier": {  
"type": "keyword"  
},  
"type": {  
"type": "keyword"  
}  
}  
},  
"message": {  
"type": "text"  
},  
"meta": {  
"dynamic": "true",  
"properties": {  
"delay": {  
"type": "integer"  
},  
"filter": {  
"type": "keyword"  
},  
"filterhost": {  
"type": "keyword"  
},  
"filtertime": {  
"type": "long"  
},  
"input": {  
"type": "keyword"  
},  
"queuehost": {  
"type": "keyword"  
},  
"queuetime": {  
"type": "long"  
},  
"relayhost": {  
"type": "keyword"  
},  
"size": {  
"type": "long"  
},  
"target": {  
"properties": {  
"index": {  
"type": "keyword"  
},  
"topic": {  
"type": "keyword"  
}  
}  
},  
"type": {  
"type": "keyword"  
}  
}  
},  
"offset": {  
"type": "long"  
},  
"session": {  
"properties": {  
"duration": {  
"type": "keyword"  
},  
"input\_bytes": {  
"type": "keyword"  
},  
"output\_bytes": {  
"type": "keyword"  
},  
"start\_time": {  
"type": "keyword"  
}  
}  
},  
"source": {  
"type": "keyword"  
},  
"tags": {  
"type": "keyword"  
},  
"type": {  
"type": "keyword"  
},  
"user": {  
"properties": {  
"sponsor": {  
"type": "keyword"  
},  
"sponsor\_profile": {  
"type": "keyword"  
}  
}  
},  
"username": {  
"type": "keyword"  
}  
}  
}  
}  
}

---

<div class="post-metadata">

### Author: ![darkmoon](https://avatars.discourse-cdn.com/v4/letter/d/ecd19e/32.png) [@darkmoon](https://discuss.elastic.co/u/darkmoon)
#### Post date: [May 14, 2019, 8:49pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/5 "2019-05-14T20:49:39Z")

</div>

Any thoughts?

All of my indices are doing this.

If I manually edit the index pattern definition to remove the entry for the 'meta' field, all of the sub fields display properly. Of course, I need to do that for all objects.

This is clearly a Kibana bug (incorrect index pattern creation). Can I get a bug number and ETR, please?

---

<div class="post-metadata">

### Author: ![CristianoFerreira](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristianoferreira/32/47529_2.png) [@CristianoFerreira](https://discuss.elastic.co/u/CristianoFerreira)
#### Post date: [May 15, 2019, 9:13am UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/6 "2019-05-15T09:13:05Z")

</div>

Hi,

Have you tried to map the fields "beat", "event", "fields", and "meta" with type: nested like:

```
"properties": {
  "beat": {
    "type": "nested" 
  }
}

```

[https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html)

Also, remove the

```
"dynamic_templates": [ {
    "string_fields": {
        "match": "*",
        "match_mapping_type": "string"
      }
 }]

```

That is matching all fields, thus making all fields of type string.

---

<div class="post-metadata">

### Author: ![darkmoon](https://avatars.discourse-cdn.com/v4/letter/d/ecd19e/32.png) [@darkmoon](https://discuss.elastic.co/u/darkmoon)
#### Post date: [May 15, 2019, 7:19pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/7 "2019-05-15T19:19:52Z")

</div>

Thanks for replying.

No, we aren't looking for nested documents. Just structured fields. The elastic mapping is fine. Kibana just isn't building the correct index pattern.

I'll try removing it, but we are getting dynamic numeric fields. (meta.size for example is of type 'long')

---

<div class="post-metadata">

### Author: ![darkmoon](https://avatars.discourse-cdn.com/v4/letter/d/ecd19e/32.png) [@darkmoon](https://discuss.elastic.co/u/darkmoon)
#### Post date: [May 15, 2019, 7:45pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/8 "2019-05-15T19:45:02Z")

</div>

Sorry, removing the match\_mapping\_type failed to load. Removing the entire dynamic\_templates block put the \*.keyword fields back. We went to some effort to fix that. It also didn't fix the base problem of structured fields showing as json. (rewriting the index pattern does, but I'd rather not have to do that every time someone refreshes a pattern.)

---

<div class="post-metadata">

### Author: ![darkmoon](https://avatars.discourse-cdn.com/v4/letter/d/ecd19e/32.png) [@darkmoon](https://discuss.elastic.co/u/darkmoon)
#### Post date: [May 16, 2019, 8:44pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/9 "2019-05-16T20:44:18Z")

</div>

I tried setting the mapping for those objects to 'nested', rather than 'object' and Kibana didn't even show them anymore.

Anyone have any ideas on how to tell Kibana to treat these as objects, please?

---

<div class="post-metadata">

### Author: ![darkmoon](https://avatars.discourse-cdn.com/v4/letter/d/ecd19e/32.png) [@darkmoon](https://discuss.elastic.co/u/darkmoon)
#### Post date: [May 17, 2019, 3:57pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/10 "2019-05-17T15:57:29Z")

</div>

[https://www.elastic.co/guide/en/kibana/6.7/document-data.html#adding-columns](https://www.elastic.co/guide/en/kibana/6.7/document-data.html#adding-columns)

You can see an example here of what should be happening.

Here's the documentation for the elastic mapping:  
[https://www.elastic.co/guide/en/elasticsearch/reference/6.7/object.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.7/object.html)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 14, 2019, 3:57pm UTC](https://discuss.elastic.co/t/objects-showing-in-discover-as-json-not-discrete-fields/180178/11 "2019-06-14T15:57:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
