Advises for index

[Thibaut603 (Dewel Thibaut)]

Hello guys,

i need advises because i'm a little bit lost

i've a elk with
one elasticsearch server
one logstash server
one kibana server
4 winlogbeat on 4 server(one per server)

so i've many questions,

• in this case think i don't need replica shards because i idon't have another elastic server right ?

• in elastic log i've some warnings like this

so i need to add fielddata=true on those field ? my question is how ? i've see this page

https://www.elastic.co/guide/en/elasticsearch/reference/6.x/fielddata.html#_fielddata_is_disabled_on_literal_text_literal_fields_by_default

and

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html

but i'm scared to do something wrong and do a big mistake

thx for help

[DavidTurner]

@Thibaut603

• in this case think i don't need replica shards because i idon't have another elastic server right ?

Correct. But don't forget to increase the number of replicas if, in future, you do add another Elasticsearch node.

@Thibaut603

• in elastic log i've some warnings like this

Please don't post images of text, they're impossible to search, hard for those of us with screenreader to read, and I can't quote them in my responses.

I suspect these fields should be of type keyword instead of text. How are you trying to search them?

[Thibaut603 (Dewel Thibaut)]

ok sorry for screenshots next times i'll post as code

yes those field are in keyword type i search them in dashboard with visualisation i've made

[DavidTurner]

@Thibaut603

yes those field are in keyword type i search them in dashboard with visualisation i've made

I think those fields are of type text and not keyword - at least this is what the exception message seems to be saying.

[Thibaut603 (Dewel Thibaut)]

in the mapping of the index in management section i've searched those fields and all are the same like this

},
            "TargetUserName": {
              "type": "keyword"
            },

[DavidTurner]

Ok, could you share the whole mapping and the whole error message you're interested in understanding?

[Thibaut603 (Dewel Thibaut)]

Hello,

sorry the whole mapping and the whole message errors is too long for one post so i've made a folder in my dropbox.

so the error come when i made a search on long period and in kibana i've this message 10 of 97 shards failed.

and yes i those field are in text type sorry for my mistake

Link here

Elastic

[DavidTurner]

@Thibaut603

and yes i those field are in text type sorry for my mistake

No worries In fact, it looks like they're being indexed as both:

          "beat" : {
            "properties" : {
              "hostname" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "name" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "version" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              }
            }

The field beat.name is a text field, but the same information is also being indexed as a keyword field under the name beat.name.keyword. I think you should be using beat.name.keyword rather than beat.name in the aggregation that your search is trying to perform:

          "aggregations": {
            "5": {
              "terms": {
                "field": "beat.name",
                "size": 10,
                "min_doc_count": 1,
                "shard_min_doc_count": 0,
                "show_term_doc_count_error": false,
                "order": [
                  {
                    "_count": "desc"
                  },
                  {
                    "_key": "asc"
                  }
                ]
              },

[Thibaut603 (Dewel Thibaut)]

hello,

i've tried to search with beat.name.keyword and event_data.targetusername.keyword and no result in return but if i search with beat.username not in long period this i don't have the message.

A friend suggest me to add fielddata : true on the mapping and reindex to resolv this message on long period search. it is a good idea ?