Using percolate in a Logstash filter?

[es435]

I have an index of percolator queries on Elasticsearch and I'm trying to write a Logstash config file with a filter that will allow me to percolate documents against those queries as I index them, sending an alert if there's a match.
Researching this I've come across suggestions that it's possible to do using the elasticsearch plugin but I'm not sure what the syntax would be to reference both the current document being indexed and the percolate index - if that makes sense?
I'm using ES 5.2 - so percolate as a query type rather than the standalone API. Any advice?
Thanks!

[warkolm (Mark Walkom)]

It should "just work" as it's another query type as you mention.

That said, I have not tried

[es435]

Hi Mark,
Thanks for your reply - glad it sounds feasible as a concept.
I've had a go at the config file and I'm getting an error message - 'logstash.agent fetched an invalid config' at the Elasticsearch bit. The code I'm trying is:

elasticsearch {
    query => "{ percolate { hosts => ["http://localhost:9200"], index => "data-search", type => "queries", id => "%{id}" }"
}

I'm very new to Logstash and Elasticsearch so no idea if I'm thinking along the right lines..

[warkolm (Mark Walkom)]

That syntax doesn't look right, check the docs for a guide - https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html

[es435]

Using the docs I think I've worked out the syntax I need - the problem I'm having is that the document I'm passing through to the percolator hasn't been indexed yet. Given that I'm structuring my query as follows:

elasticsearch {
    hosts => ["http://localhost:9200/data-search/queries"]
    user => "elastic"
    password => ""
    index => "data-search"
    query => '"percolate" : { "field" : "query", "document_type" : "doctype", "document" : { "" }}'
}

Can anybody point me in the right direction for how to refer to the document currently being indexed in the "document" field?
Thanks!

[pemontto]

I tried using the full query DSL like you have and substituted fields into the query using the sprintf format so it looked something like this, but don't know how you could get the whole event into the query without manually defining the fields:

elasticsearch {
  hosts => ["test-es:9204"]
  index => "news_percolator"
  fields => { "_id" => "matches" }
  query => '"percolate" : { "field" : "query", "document_type" : "doctype", "document" : { "title": "%{title}", "summary": "%{summary}" }}'
}