How to set "enabled":"False" to a field in filebeat

Elastic Stack Beats
1

Add it to setup.template.fields file. But the index template doesn't have the "enabled: false" applied.
Is there a way to disable a field from indexing ?

  - name: message
    level: core
    type: text
    enabled: false
    description: 'For log events the message field contains the log message'
    example: This is test from Openstack Swift

Regards // Hugo

2

Hi Hugo,

Welcome to the community.

can you please reformulate your question as it is not clear?

3

Hi @ibra_013 ,

In the document, there's enabled option for field. There're around 30 fields in out doc. I'd like to disable index for few fields. The doc is sent via Filebeat. I thought there's option for filebeat to create index template with fields disabled.

How to specify enabled: false in Filebeat's fields.yml ?

Thanks // Hugo

4

You're linking to Elasticsearch documentation there, so it's not a Filebeat concept you can configure. You need to create your own template that sets this. Alternatively you can just drop the field with Filebeat.

Also 6.X of the stack id EOL, you need to upgrade.

5

@warkolm

Also 6.X of the stack id EOL, you need to upgrade.
Sorry for confusion, we are using 7.4.

In the Beats document, there're enabled & index mapping parameters.

What's the difference between these two parameters?

Example of my use case :
There're account, account_keyword and additional_info fields in a doc.
We're looking for ways to store all three fields in ES but not index the account and additional_info. The goal is when the user searches the account_keyword, the account and additional_info are visible but not searchable.

        },
        "account" : {
          "type" : "text",
          "index" : false,
          "norms" : false
        },
        "account_keyword" : {
          "type" : "keyword",
          "ignore_above" : 1024
        },
        "additional_info" : {
          "type" : "text",
          "index" : false,
          "norms" : false
        },

I found three different mapping for an index. Not very sure about which is the right one for the use case. I tested the index:false in the filebeat's fields.yml and I can see the result in the above.

index: false
index: no
enabled: false
6

The documentation you linked to is for building your own custom beat, it does not apply to Filebeat sorry.

7

Thanks to point it out. @warkolm I noticed this option in the fields.yaml from package.
It seems commonly used in filebeat.x86_64 7.10.2-1

[root@hkd filebeat]# grep index: fields.yml  -B 5
      - name: text
        type: text
        norms: false
        default_field: false
      description: The stack trace of this error in plain text.
      index: false
--
    - name: x509.public_key_exponent
      level: extended
      type: long
      description: Exponent used to derive the public key. This is algorithm specific.
      example: 65537
      index: false
--
    - name: client.x509.public_key_exponent
      level: extended
      type: long
      description: Exponent used to derive the public key. This is algorithm specific.
      example: 65537
      index: false
--
    - name: server.x509.public_key_exponent
      level: extended
      type: long
      description: Exponent used to derive the public key. This is algorithm specific.
      example: 65537
      index: false
--
    - name: public_key_exponent
      level: extended
      type: long
      description: Exponent used to derive the public key. This is algorithm specific.
      example: 65537
      index: false
--
          description: "Server log file"
          type: group
          fields:
          - name: stacktrace
            description": Stack trace in case of errors
            index: false

[root@hkd filebeat]# yum list installed | grep filebeat
filebeat.x86_64                  7.10.2-1                 @/filebeat-oss-7.10.2-x86_648F6NQU

Regards // Hugo

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.