Disclaimer: As with the OP of the linked thread, I too am learning but the Simulate API is a little confusing to me so apologies for my noobiness. Thank you in advance for your insights and feedback.
In this thread ES 7.15 FileBeats Sophos XG module not separating data into variables, @stephenb tested the parsing of a log message using the Simulate API. When they did this, the message used was not the actual raw log message but rather a message that was already in ES. Perhaps I am mistaken but I thought part of the functionality of Filebeat was to parse log messages so I have these questions:
- Why was the message in ES used instead of the original raw log?
- Can raw log messages also be used with the Simulate API to test the parsing and ingestion of raw logs from a given source?
- When @stepheb said "... I took the sample message you provided and put in a
%{SYSLOG5424PRI}
the<44>
...", what is meant here? When I look at my Sophos ingest pipeline, I do see the%{SYSLOG5424PRI}
but not the<44>
(example below) so I'm not sure to what their statement refers or why this is relevant:
"grok": {
"field": "message",
"patterns": [
"%{SYSLOG5424PRI}%{GREEDYDATA:log.original}$"
]
}