I have a custom beat based on libbeat, that parses log files and adds fields and their value to the beat event to be output to logstash. The log files contain json that get unmarshaled by go and then translated from a go nested map to a dot delimited string key name and associated value.
The problem is the application producing the log files containing the json text often is undisciplined in the key naming and value types. I have my custom beat performing an api request to elastic to read the index mapping and then performing a validation and key naming and type conversions so as to avoid downstream mapping conflicts. This works great, but the beat needs to know the index name before it processes any events.
Currently, I have a custom config defined in the yml config file called IndexWildcard that must be kept in sync with the index name in the yml file output.logstash. I'd rather have my beat just read the index name at startup and not have this additional indexWildacrd setting.
Is there a go api for a libbeat to read the yml content and specifically the output.logstash index setting?
Also, I have a custom processor that I plug into a custom winlogbeat that needs a similar access to yml settings. I hope there is a generalized api I can use for both types of beats.