Multi field aggregation with Elasticsearch-kibana

[shethsaam (satyam)]

With kibana devtools, I am trying to create a query to get unique values from multiple fields into a single bucket. I am using cppy_to to create a new field and plan to find the unique values with aggregation. I need to find a way to put the data from the existing fields into the newly created field.
Current Code:

PUT indexme
{
  "mappings": {
    "my_type": {
      "properties": {
        "wlan_da": {
          "type": "keyword",
          "copy_to": "full_namesas" 
        },
        "wlan_sa": {
          "type": "keyword",
          "copy_to": "full_namesas" 
        },
        
        "full_namesas": {
          "type": "keyword"
        }
      }
    }
  }
}

PUT indexme/my_type/1
{
  "wlan_da": { }
  "wlan_sa":  { }
}


GET indexme/_search
{
  "aggs": {
    "testprac_wala2": {
      "terms": {
        "field": "full_namesas"
      }
    }
  }
}

Need help with PUT indexme/my_type/1. Let me know if I am wrong somewhere or missing something. New to elastic community

[tylersmalley]

It should look like this:

POST indexme/my_type
{
  "wlan_da": "foo",
  "wlan_sa": "bar"
}

You're indexing the properties wlan_da and wlan_sa which are available on full_names.

[shethsaam (satyam)]

Hi tylersmalley,
Thanks for your suggestions.
I am unable to understand what you actually mean by "foo" and" bar". More clear explanation of what I am trying: I have an index called abc which has fields wlan_da and wlan_sa. Both this fields have some values. I want to aggregate over this values together in a single bucket to find the unique values in wlan_da and wlan_sa field. So in POST indexme/my_type, I want to insert values from wlan_sa (index=abc) into wlan_sa (index=indexme) and values from wlan_da (index=abc) into wlan_da (index=indexme) so that later by aggregating on full_names, I get unique values from both the fields together in a single bucket.

[tylersmalley]

"foo" and "bar" are just test values. The API call creates a document with those values assigned to wlan_da and wlan_sa. Then, using your search they are aggregated over full_names.

Does that help?

PUT abc
{
  "mappings": {
    "my_type": {
      "properties": {
        "wlan_da": {
          "type": "keyword",
          "copy_to": "full_namesas" 
        },
        "wlan_sa": {
          "type": "keyword",
          "copy_to": "full_namesas" 
        },
        
        "full_namesas": {
          "type": "keyword"
        }
      }
    }
  }
}

POST abc/my_type
{
  "wlan_da": "foo",
  "wlan_sa": "bar"
}


GET abc/_search
{
  "aggs": {
    "testprac_wala2": {
      "terms": {
        "field": "full_namesas"
      }
    }
  }
}

[shethsaam (satyam)]

Hi tylersmalley,
It did helped me. I want to replace test values "foo" and "bar" with existing values in the wlan_da and wlan_sa type of some other index (say XYZ). So I am trying to pull values from the wlan_da and wlan_sa field of some other index ( XYZ) and trying to put them in the full_namesas. So How to get this values in place of "foo" and "bar" ??

[tylersmalley]

If they are already in an index you will need to re-index as copy_to happens at index time.

There is an API to do this: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html

Here I am creating an index containing only wlan_da and wlan_sa and adding a single document.

PUT abc
{
  "mappings": {
    "doc": {
      "properties": {
        "wlan_da": {
          "type": "keyword"
        },
        "wlan_sa": {
          "type": "keyword"
        }
      }
    }
  }
}

POST abc/doc
{
  "wlan_da": "foo",
  "wlan_sa": "bar"
}

Then, I create a new index with the mapping containing copy_to with the desired full_namesas and re-index the data into it.

PUT abc2
{
  "mappings": {
    "doc": {
      "properties": {
        "wlan_da": {
          "type": "keyword",
          "copy_to": "full_namesas" 
        },
        "wlan_sa": {
          "type": "keyword",
          "copy_to": "full_namesas" 
        },
        
        "full_namesas": {
          "type": "keyword"
        }
      }
    }
  }
}

POST _reindex
{
  "source": {
    "index": "abc"
  },
  "dest": {
    "index": "abc2"
  }
}

GET abc2/_search
{
  "aggs": {
    "testprac_wala2": {
      "terms": {
        "field": "full_namesas"
      }
    }
  }
}