I'm streaming data into SolR and I need to produce a JSON field in the FileBeat payload called event_timestamp. Is there some way I can replicate the @timestamp field, or do I need to edit the FileBeat template? If so how?
Thanks.
If you want to do it directly in filebeat this would required to change the source code. I recommend you to do these modifications in Logstash.
I actually wasn't even aware that Filebeat -> Solr works.
Thanks for your help! I'm using a plugin to stream from logstash to solr, but I think I might ditch solr for elasticsearch since it seems to be a better supported and more well-trodden path.
Yes, Filebeat is specifically target for Logstash and Elasticsearch. Let me know you still need some help.
... So I've been using Filebeat but now I need add a bit more structure to my log messages, so I guess I'll use Logstash... But how can I extract a value from the message itself, for example, if my message looks like:
{
"message" : "reallyUsefulData : heres the information I want"
}
How can I produce a document that look like this:
{
"message" : "reallyUsefulData : heres the information I want"
"reallyUsefulData" : "heres the information I want"
}
Is this what grok is for?
yes, this what logstash with grok filters is used for
...awesome, thanks!