So, I have recently started playing with the Elastic Stack and have seen some interesting use cases, like ingesting pfSense logs, etc,.
While looking into some logstash and ES configurations I have noticed some interesting patterns, like:
Logstash/Elasticsearch Specific questions:
Some people use numbers at the begining of Logstash configuration file names. Is this to determine loading priority by Logstash, or is it just random choice of the admin?
Some stages of the event pipelines, like filters, occur in more than one file in the logstash/config.d/ directory. Does Logstash loads all those individual files in memory and assuming there is no syntax errrors, applies the applicable ones once there is a match? How does that work exactly?
Can you configure the output phase to create custom indexe names? And if so, does that affect the template which will be attached to the index on Elasticsearch resulting in mapping errors and wrong data types?
What are the best practices when ingesting data from multiple sources? Is it advised on those instaces to have their own separate index created on the output section of the config so they are isolated when they are in Elasticsearch? Are multiple pipelines the best option?
Elasticsearch/Kibana Specific questions:
How exactly does the index name relate to an Elasticsearch template? Can this be chosen manually? How? How does Elasticsearch choses the template to use?
Can corrections to mapping be made/appended to the used mapping post indexing? How?
Thanks in advance.