Is it possible to use the modules that come with filebeat and create your own custom logs as well?
I attempted to follow the file structure of the 'iis' module and create my own, but it doesn't seem to be processing the pipeline I have made (despite it working in the pipeline/_simulation)
I've somewhat got the custom modules figured out on Windows, however I have 2 questions still.
1 - I created a custom module and after running the pipeline update in PS, I can see it in my pipeline on Kibana (GET /_ingest/pipeline?pretty). When I go to 'discover' in Kibana I can see the individual fields, however when I check the filebeat index, I can't see the fields created there. I have one module using the processor "grok" which looks to have updated the indext, however using processor "dissect" for my new module, it is not updating the filebeat index. Am I missing a step?
2 - I looked at the instructions creating a filebeat module in Linux and it looked pretty good, however I want to do this in Windows instead (which did not work out for me). I ended up just coping the 'iis' module flow and creating my own module based on that. My question; how do I add my fields into the index template? In Linux is creates your own fields.yml per module however in windows it looks like I should be exporting the already uploaded template:
PS C:\Program Files\Filebeat>
.\filebeat.exe export template --es.version 7.6.1 | Out-File -Encoding UTF8 filebeat.template.json
Updating that and then importing it back in
PS > Invoke-RestMethod -Method Put -ContentType "application/json" -InFile filebeat.template.json -Uri http://localhost:9200/_template/filebeat-7.6.1
Is that correct?
Once done should I clear out my indexes before restarting the filebeat module?
Clear out all indexes:
Invoke-RestMethod -Method Delete http://localhost:9200/filebeat-*
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.