Advises for index

Elastic Stack Elasticsearch
1

Hello guys,

i need advises because i'm a little bit lost

i've a elk with
one elasticsearch server
one logstash server
one kibana server
4 winlogbeat on 4 server(one per server)

so i've many questions,

  • in this case think i don't need replica shards because i idon't have another elastic server right ?

  • in elastic log i've some warnings like this

fielddata%20beat%20name
fielddata beat name.PNG2320×39 17.6 KB

fielddata%20short%20message
fielddata short message.PNG2329×43 18.6 KB

fielddata
fielddata.PNG2325×51 23.7 KB

so i need to add fielddata=true on those field ? my question is how ? i've see this page

https://www.elastic.co/guide/en/elasticsearch/reference/6.x/fielddata.html#_fielddata_is_disabled_on_literal_text_literal_fields_by_default

and

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html

but i'm scared to do something wrong and do a big mistake

thx for help

2

Correct. But don't forget to increase the number of replicas if, in future, you do add another Elasticsearch node.

Please don't post images of text, they're impossible to search, hard for those of us with screenreader to read, and I can't quote them in my responses.

I suspect these fields should be of type keyword instead of text. How are you trying to search them?

3

ok sorry for screenshots next times i'll post as code

yes those field are in keyword type i search them in dashboard with visualisation i've made

4

I think those fields are of type text and not keyword - at least this is what the exception message seems to be saying.

5

in the mapping of the index in management section i've searched those fields and all are the same like this

},
            "TargetUserName": {
              "type": "keyword"
            },
6

Ok, could you share the whole mapping and the whole error message you're interested in understanding?

7

Hello,

sorry the whole mapping and the whole message errors is too long for one post so i've made a folder in my dropbox.

so the error come when i made a search on long period and in kibana i've this message 10 of 97 shards failed.

and yes i those field are in text type sorry for my mistake

Link here

8

No worries :slight_smile: In fact, it looks like they're being indexed as both:

          "beat" : {
            "properties" : {
              "hostname" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "name" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "version" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              }
            }

The field beat.name is a text field, but the same information is also being indexed as a keyword field under the name beat.name.keyword. I think you should be using beat.name.keyword rather than beat.name in the aggregation that your search is trying to perform:

          "aggregations": {
            "5": {
              "terms": {
                "field": "beat.name",
                "size": 10,
                "min_doc_count": 1,
                "shard_min_doc_count": 0,
                "show_term_doc_count_error": false,
                "order": [
                  {
                    "_count": "desc"
                  },
                  {
                    "_key": "asc"
                  }
                ]
              },
9

hello,

i've tried to search with beat.name.keyword and event_data.targetusername.keyword and no result in return but if i search with beat.username not in long period this i don't have the message.

A friend suggest me to add fielddata : true on the mapping and reindex to resolv this message on long period search. it is a good idea ?

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.