Flatten nested dictionaries in index

I'm new to these forums so please let me know if I've posted this in the wrong place.

I have an index that contains a key-value tags consisting of an array of values, which are dictionaries. The dictionaries always have 3 keys, key, type, and `value.

Example document:

  "tags": [
      "key": "http.status_code",
      "type": "string",
      "value": "200"
      "key": "http.protocol",
      "type": "string",
      "value": "HTTP/1.1"

However what I'd really like is to flatten the nested dictionaries so they were top-level key-values in the document using the corresponding key and value, like this:

  "http.status_code": "200",
  "http.protocol": "HTTP/1.1"

Unfortunately I'm having a difficult time understanding how to accomplish this.

Should tags be a nested type in the index? Can I use scripted fields to accomplish this?

"Nest" has a meaning in ES.
By default, all fields are not nested. It just adds a name space to your key.
So in your case, every object under "tags" is searched equally.

In your example, you just need to use "tags.http.status_code".
If you really want top level, you have to store each object as individual document.
Basically removing the layer "tags", which eliminates the arraylist (tags).

Hm. I'm not sure how to use tags.http.status_code as a top level key in the index given that that is the value of key in one of the dictionaries.

But I did find that I can use scripted fields to extract information:

for (def tag : params['_source']['tags']) {
  if (tag['key'] == 'http.status_code') {
    return tag['value']

Unfortunately I have to set this up manually for every possible key-value dictionary.

Sorry, I mean to say "tags.key". I copy and pasted from the wrong source.

if "http.status_code" & "http.protocol" is what you want, you need to store in such structure in your document.

  "http": {
    "status_cude": "200",
    "protocol": "HTTP/1.1"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.