What is the best way to persist data between two messages using the ruby filter? For example to include the previous message parsed I can use global variables like this:
I understand the limitations of message ordering but was wondering what the recommended way of doing this was? I'd rather not use global variables unless absolutely necessary.
You should never need to use global variables. You can do this using an instance variable (@last_event). You can only use a single worker thread, so the solution does not scale, and you have to use the ruby execution engine because with java_execution enabled events get re-ordered.
I'm not too concerned about the ordering, this is to get a fallback of an approximate time of a log line when the log line happens to be missing the timestamp.
I have a file that has a header, then a row with the data I need, then a header which I just drop then subsequent events I need to stick info from the row with the timestamp.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.