Hi All,
I ingest forex data into an index called forex1. The data received looks like this per minute. I do a split on candles:
{
"@timestamp" => 2017-09-29T12:25:00.434Z,
"granularity" => "M1",
"@version" => "1",
"instrument" => "EUR_USD",
"candles" => {
"volume" => 31,
"closeMid" => 1.180655,
"highMid" => 1.18076,
"openMid" => 1.180595,
"lowMid" => 1.18058,
"time" => "2017-09-29T12:24:00.000000Z",
"complete" => false
}
}
{
"@timestamp" => 2017-09-29T12:26:00.335Z,
"granularity" => "M1",
"@version" => "1",
"instrument" => "EUR_USD",
"candles" => {
"volume" => 26,
"closeMid" => 1.180735,
"highMid" => 1.18079,
"openMid" => 1.18063,
"lowMid" => 1.1806,
"time" => "2017-09-29T12:25:00.000000Z",
"complete" => false
}
}
There's a "time" on each doc that I want to use to query the last 5 docs. But for having it simple lets say I just want to fetch the previous doc. That is "time" minus 1 minute.
Im using elasticsearch filter plugin for this but I dont know how to create the query:
elasticsearch {
query => "instrument:%{[instrument]} AND time:[ "TIME-1 MINUTE" to %{[time]}}"
fields => { "closeMid" => "closeMid_t1" }
}
My questions (sorry for the noob questions bear with me):
1 - How do I refer to data in my current event? I want the query to be made on the same "instrument" that I have in the event. Is this the correct syntax? - instrument:%{[instrument]}
(in my case above the query should be done on EUR_USD but I can have different pairs in my data)
2 - How do I subtract 1 one minute from my timestamp called "time"? time:%{[time]} - 1min
or something like that should be better than querying a range.
3 - When the query is done I want to take the field 'closeMid
' from previous event and save it in a new field 'closeMid_t1
' in current event. Is this the correct way?
4 - I want to do this 5 times because a need values from 5 previous docs. Should I create a for loop outside the filter or inside?
5 - How to handle errors? What if I don't have any previous values. How will this be handled by the filter? Should I do an explicit closeMid_t1=0
some how?
Thanks for a suggestions before hand,
Cris