Fielddata is disabled, but I have added a keyword field

Elastic Stack Elasticsearch
1

I seem to be having an issue with mapping in my template. I have a field with data type "text", when I look into the Kibana SIEM console I receive the following error for several of my fields:
[illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [host.name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.

However when I look up one of these fields I see that I have a keyword field mapped to each "text" type
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256

Exactly where do I need to add the type:keyword to resolve this?

2

It should be:

"name": {
  "type": "keyword"
}
3

I am attempting to update the mapping as you suggested, but it appears, my query is malformed, but I can't spot the error. Would you mind having a look?

PUT auditbeat-7.4.2-2020.03.06/_mapping
{
"properties": {
"host": {
"properties": {
"name": {
"type": "keyword"
}
}
}
}
}

4

I don't know as I can't see the error message. Most likely you are trying to update an existing field, but that's just a guess.

5

Here is the error message:

"type": "illegal_argument_exception",
"reason": "mapper [host.name] of different type, current_type [text], merged_type [keyword]"
},
"status": 400

6

So that's what I thought. You can not update an existing mapping and change the type of an existing field.

7

So is there a way to resolve the error without setting fielddate=true? or do I need to create a new index with the desired mapping?

8

You need to provide the right mapping and reindex your data accordingly.

Note that you should follow Elastic Common Schema as it's used by SIEM.

BTW what is the source of your data? Are you using a custom data source which is not supported out of the box by our beat agents?

9

I am using Auditbeat with default mapping.

10

Which version of Auditbeat?

I mean that when I'm using Auditbeat with all the default values, that I set up auditbeat as written in the documentation, it installs everything for me, including the right templates for auditbeat.

11

Installed version 7.4.2. Installed on Redhat if that matters, pretty much run it straight out of the box. Was looking to see what came through so I could start figuring out what I wanted to see, what I didn't, that sort of thing.

12

Could you check if you have an index template named auditbeat-SOMETHING?

13

I thought I did, but upon checking, that isn't true.

14

So you need to fix that.

15

So I added the template and set the above to mappings, I reindexed everything, but I think I have an issue with getting the correct syntax to update the field host.name. Any thoughts?

16

What is the problem? Sorry I don't understand what is the current status.

17

I am still seeing the same error, I have added the template for audit*. I have added the following in Mappings

{
  "properties": {
    "host": {
      "properties": {
        "name": {
          "type": "keyword"
        }
      }
    }
  }
}
18

What is the output of:

GET auditbeat-7.4.2-2020.03.06/_mapping

May be you need to change the date though.

19

Mappings here:
https://docs.google.com/document/d/1b1-zFB0JAUX5s0N6tpjnArryMyiCIac2nXuUN90H5qI/edit?usp=sharing

20

I don't have access to this document as it's not public. Could you share it on gist.github.com instead?