Searching custom fields does not work with Full Text query mode(?)

[john_eapen (je)]

Hi

My log data contains several fields that Filebeat doesn’t know about in advance. ( eg: myapp.traceId, myapp.host.name etc. ). To avoid conflicts these are not directly root but under a root property "myapp".

Now, when I query explicitly as "myapp.traceId: abc123" , then this works BUT this does not work if
I query simply as "abc123".

I am trying to play around with this setting "setup.template.append_fields [experimental]"
Or do I need to add these properties to fields.yml or something else to get them included into "default_fields"

Appreciate the help
Thx

[shaunak (Shaunak Kashyap)]

Hi,

Yes, you should add these fields under setup.template.append_fields as shown in https://www.elastic.co/guide/en/beats/filebeat/current/configuration-template.html.

Once you've done that, you can verify that Filebeat will export these fields as part of the index template it will send to Elasticsearch by running filebeat export template. For the purposes of the "all fields" query effect that you're trying to achieve, verify that the fields are listed under the settings.index.query.default_field field in the exported template.

Shaunak