I've read a hundred pages of docs/forum posts/random blogs and am just as confused as I was to begin with - I took a break from this project for a few months, but nothing magically changed 
Overall Scenario:
- The filebeat osquery module has almost all values saved as strings.
- I want the numeric types to actually be saved as numbers to enable better math and graph capabilities. (See post from July: OSQuery field types)
- Ingest processors are the solution for converting fields/values
- Due to how the osquery modules works, the fields that need to be converted can't be accessed by Processors defined within the filebeat module yml, they must be an elasticsearch pipeline (See Using a processor in a filebeat module may or may not actually find fields)
Problem:
- I can not get a pipeline processor to actually apply to a message, it seems like it is just being totally ignored
Steps performed:
- Ensured filebeat/elasticsearch/osquery were all talking to each other fine, data shows up as expected, default pipelines installed via setup --pipelines --modules osquery etc
- Verified I could suggessfully add a field directly from the filebeat osquery module yml [my-osquery.yml]. This worked.
- Removed that block from the my-osquery.yml
- Created a custom pipeline that would add a field
- Updated the main filebeat config [my-filebeat.config.yml] to point to my pipeline and to use a new index pattern
- Stopped and restarted filebeat with that new config
- Checked new data, did not see the additional field
- Used Kibana console to run a POST _ingest/pipeline/test_osquery_fixer/_simulate with the _source from a real osquery message. Verified that the custom pipeline does work correctly in that mode.
Random things tried:
- updated my pipeline to have a step where it uses the pipeline processor to run the original osquery pipeline too, just in case that was needed, but that didn't work
- added a pipeline config line to the osquery module yml randomly, didn't expect that to work
Here is my pipeline:GET _ingest/pipeline/test_osquery_fixer
{
"test_osquery_fixer" : {
"description" : "testing pipeline",
"processors" : [
{
"pipeline" : {
"name" : "filebeat-7.4.2-osquery-result-pipeline"
}
},
{
"set" : {
"field" : "custompipe",
"value" : "testosqueryfixer",
"ignore_failure" : false
}
}
]
}
}
I added the pipeline line to my-filebeat-config.yml, didn't even try to do conditional-only-for-osquery-docs yet:
output.elasticsearch:
hosts: ["asdf.local:9200"]
pipeline: "test_osquery_fixer"
Even have the pipeline name in the modules.d/my-osquery.yml config...
# Module: osquery
- module: osquery
result:
enabled: true
var.use_namespace: true
input:
pipeline: "test_osquery_fixer"
Final results are:
- Under _simulate the pipeline works
- For messages being received by filebeat the pipeline doesn't work