Please format logs and configuration files using the </>-Button.
I'm not sure if using type and fields_under_root can be recommended, as the document_type setting also overwrites the type field. That is, given your configuration I'm not sure about the value of type. Running filebeat with debug logging enabled (e.g. -d 'publish') will print the events before publishing. This might help inspecting the actual events contents.
You are using the setting fields_under_root: true. This setting merges configured fields right into the event generated, without creating the namespace fields. Yet you reference to the fields namespace in your output config. Either remove fields_under_root: true from you config or try changing your conditionals from when.<cond>.fields.type to when.<cond>.type: ... .
I have no idea where/when/why you set the type "error" (incomplete config?). If index is empty/fails cause the field referenced is missing, the next configuration will be run. If all failed, the index setting will be used.
I have no idea why you use [@metdata][type] in one of your indices. With filebeat 5.6 this field is hardcoded to "docs" + was only shipped to logstash in the past. The field is not known to the elasticsearch output.
Given you have some application logs and some generic logs you can also configure filebeat like this:
The field should become available in kibana once available. You might have to update the index pattern (check for index patterns in management UI), as an index pattern is kind of a mapping to a set of indices matching.
Depending on fields_under_root setting your custom fields should be available via service_name or fields.service_name.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.