# Logstash query template

**URL:** <https://discuss.elastic.co/t/logstash-query-template/95407>\
**Category:** Logstash\
**Created:** [August 1, 2017, 7:31pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407 "2017-08-01T19:31:43Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![exocore123](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exocore123/32/45130_2.png) [@exocore123](https://discuss.elastic.co/u/exocore123)\
**Post date:** [August 1, 2017, 7:31pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407/1 "2017-08-01T19:31:43Z")

</div>

**Change from original question**

```
{
  "query": {
    "bool": {
      "must": {
        "query_string": {
          "query": "id=%{id}"
        }
      },
      "filter": {
        "range" : {
            "timestamp" : {
              "gt" : "now-1h",
              "lt" : "now"
            }
        }
      }
    }
  }
}

```

I paste this into a `template.json` file and changed my elasticsearch to

```
elasticsearch {
                hosts => ["localhost:9200"]
                index => "${input}"
                query_template => "template.json"
}

```

And this results in an error. `reason=>"Something is wrong with your configuration."`  
Not sure how to work around this? I looked at samples on [elastic.co](http://elastic.co) and looks as simple as this. I tried copying the whole path, but did not solve the problem.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 3, 2017, 12:07pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407/2 "2017-08-03T12:07:45Z")

</div>

Why not just use a date range filter in your query? I believe reasonably recent versions of ES are quite efficient in skipping indexes that obviously don't contain any documents in range (i.e. using logstash-\* isn't significantly slower)

---

<div class="post-metadata">

**Author:** ![exocore123](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exocore123/32/45130_2.png) [@exocore123](https://discuss.elastic.co/u/exocore123)\
**Post date:** [August 3, 2017, 4:37pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407/3 "2017-08-03T16:37:35Z")

</div>

Wasn't sure if applying a date range would have applied to looking for the correct index. Am I suppose to write it within `parameters` filter of `elasticsearch`? I came up with a solution by parsing the timestamp and aggregate it to my index query name instead of using a 2-days date range. Not sure about the solution tho, do you have any opinion/suggestion?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 3, 2017, 5:22pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407/4 "2017-08-03T17:22:10Z")

</div>

> Wasn’t sure if applying a date range would have applied to looking for the correct index.

You don't need to figure out which indexes to query. Just use a wildcard.

> Am I suppose to write it within parameters filter of elasticsearch?

What do you mean by "parameters filter"?

---

<div class="post-metadata">

**Author:** ![exocore123](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exocore123/32/45130_2.png) [@exocore123](https://discuss.elastic.co/u/exocore123)\
**Post date:** [August 3, 2017, 5:53pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407/5 "2017-08-03T17:53:00Z")

</div>

If I wanted to apply a date range in the query, how would I use it within elasticsearch plug in? This [ElasticSearch Parameters Plugin](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-parameters)

Say give or take you have 90 indexes, would wildcard vs specific index contribute to speed? I thought it would since `took`'s average value is less than that of a wildcard search

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 3, 2017, 5:59pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407/6 "2017-08-03T17:59:05Z")

</div>

> If I wanted to apply a date range in the query, how would I use it within elasticsearch plug in? This Elasticsearch Parameters Plugin

That's the elasticsearch _output_ plugin, but you seem to be asking about the elasticsearch _input_ or _filter_ plugin.

> Say give or take you have 90 indexes, would wildcard vs specific index contribute to speed? I thought it would since took's average value is less than that of a wildcard search

Then I'm surprised. I was under the impression that the different would be far less noticeable.

---

<div class="post-metadata">

**Author:** ![exocore123](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exocore123/32/45130_2.png) [@exocore123](https://discuss.elastic.co/u/exocore123)\
**Post date:** [August 3, 2017, 9:57pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407/7 "2017-08-03T21:57:25Z")

</div>

I am trying to test the wildcard with date range, to see if it'll make any difference

```
GET logstash-*/_search
{
  "query": {
    "bool": {
      "must": {
        "query_string": {
          "query": "id=%{id}"
        }
      },
      "filter": {
        "range" : {
            "timestamp" : {
              "gt" : "now-1h",
              "lt" : "now"
            }
        }
      }
    }
  }
}

```

I paste this into a `template.json` and changed my elasticsearch to

```
elasticsearch {
                hosts => ["localhost:9200"]
                index => "${input}"
                query_template => "template.json"
}

```

And this results in an error. `reason=>"Something is wrong with your configuration."` Not sure how to work around this?

Nevermind. Found the error.

`Unknown setting 'query_template' for elasticsearch {:level=>:error}`

Version error?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 4, 2017, 5:49am UTC](https://discuss.elastic.co/t/logstash-query-template/95407/8 "2017-08-04T05:49:42Z")

</div>

Yes, your elasticsearch filter plugin is most likely too old. Try upgrading it, or upgrade Logstash altogether.

---

<div class="post-metadata">

**Author:** ![exocore123](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exocore123/32/45130_2.png) [@exocore123](https://discuss.elastic.co/u/exocore123)\
**Post date:** [August 4, 2017, 6:49am UTC](https://discuss.elastic.co/t/logstash-query-template/95407/9 "2017-08-04T06:49:21Z")

</div>

Ahh... I cant really just upgrade logstash/elasticsearch... do you have any alternative suggestions? Can I somehow squeeze all of the template into query in elasticsearch filter plugin?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 4, 2017, 7:27am UTC](https://discuss.elastic.co/t/logstash-query-template/95407/10 "2017-08-04T07:27:37Z")

</div>

> Can I somehow squeeze all of the template into query in elasticsearch filter plugin?

Sure, just stuff it into a single string like in the examples in the documentation. It won't look nice but it'll work.

---

<div class="post-metadata">

**Author:** ![exocore123](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exocore123/32/45130_2.png) [@exocore123](https://discuss.elastic.co/u/exocore123)\
**Post date:** [August 4, 2017, 4:47pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407/11 "2017-08-04T16:47:21Z")

</div>

Thanks for the response Magnus. I have tried following this documentation [Elasticsearch filter plugin](https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html) and squeeze all of the query on one line, but I am obtaining a `fetched an invalid config` file error

Possibly due to quotes within `query => " "bool" : ... "`?

I tried doing something like `query => " 'bool': ..."` and now the resulting error message is  
`"reason":"Failed to parse query`  
`"reason":{"type":"query_shard_exception","reason":"Failed to parse query`  
`"reason":"Encountered \" <RANGE_GOOP>`

> <https://github.com/logstash-plugins/logstash-filter-elasticsearch/issues/20>

My problem is very similar to the issue v0it has

Nvm. Solved my own question. Had to use lucene format.

`query => '"id":%{id} AND @timestamp:[now-5h/d TO now/d]'`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 1, 2017, 4:47pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407/12 "2017-09-01T16:47:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
