Hello.
I have a working scoring query with decay exponential function calculated from a date. I want to use scheduler in logstash to run the query every 1h, score the results, and export it to json or csv.
My logstash configuration is:
input {
elasticsearch {
hosts => [ "elastic1.lan", "elastic1.lan" ]
index => "ti"
query => '{
"query": {
"function_score": {
"query":{"bool":{"must":[{"exists":{"field":"url"}},{"match":{"tags":"url"}},{"match":{"tags":"valid"}},{"range":{"@timestamp":{"gte":"now-14d","lte":"now"}}}],"must_not":{"match":{"tags":"_grokparsefailure"}}}},
"exp": {
"@timestamp": {
"origin": "now-1h",
"scale": "14d",
"offset": "12h",
"decay" : 0.6
}
},"boost": "28","score_mode": "max","boost_mode":"multiply"
}
}
}
'
size => 1000
scroll => "1m"
docinfo => true
schedule => "47 * * * *"
}
}
filter {}output {
file {
codec => "json"
path => ["/tmp/export.%{+YYYY-MM-dd}_scored_urls.json"]
}
}
Is there any way to run this and get the score in json out from Logstash level?
Or should I not use logstash and stay with other tools like a bash script and cron, or python and crone to run the query I want and export it to whatever format I want?
Thanks for hints.