Counting Terms In Text Field

Elastic Stack Kibana
1

I'm using Logstash's RSS input to pull data into Elastic. I've configured the field title as a multi-type field, one as text, the other as a keyword. I want to count the frequency of words that appear in titles across searches to identify trends. However, when I go to create a visualization, say a bar chart, I choose terms as the aggregation and then the only field I can select is title.keyword. Obviously, that doesn't help me as it doesn't count the number of times a word shows up but the number of times that whole string of text shows up. What am I doing wrong?

2

The fieldata disabled by default. You should enable it to use text field type in the aggregation or Kibana visualization.

Here is an example for you.

#push example data
PUT collection_test2/_doc/1
{"text_field":"word1 word2 word3"}

PUT collection_test2/_doc/2
{"text_field":"word2 word2 word3 word4"}

#enable the fielddata to true
PUT collection_test2/_mapping
{
  "properties": {
    "text_field": { 
      "type":     "text",
      "fielddata": true
    }
  }
}

#Create visualization or search your text field type data
GET collection_test2/_search
{
  "size": 0,
  "aggs": {
    "NAME": {
      "terms": {
        "field": "text_field"
      }
    }
  }
}

Note: You should update the Kibana data view after update the index mapping and refresh the browser => Stack Management => Data views => collection_test3 => refresh (there is a button for that)

Note2: The result will show the number of docs that contains per wordX.

first image
first image3434×1674 429 KB

3

Adding to @Musab_Dogan suggestion -

From the doc

It usually doesn’t make sense to enable fielddata on text fields. Field data is stored in the heap with the field data cache because it is expensive to calculate. Calculating the field data can cause latency spikes, and increasing heap usage is a cause of cluster performance issues.

Most users who want to do more with text fields use multi-field mappings by having both a text field for full text searches, and an unanalyzed keyword field for aggregations, as follows:

PUT my-index-000001
{
  "mappings": {
    "properties": {
      "my_field": { 
        "type": "text",
        "fields": {
          "keyword": { 
            "type": "keyword"
          }
        }
      }
    }
  }
}
4

This was originally what I did, but I wasn't getting any results. Enabling fielddata solved my issue.