# Elasticsearch Input Plugin - Index field string to match system date

**URL:** https://discuss.elastic.co/t/elasticsearch-input-plugin-index-field-string-to-match-system-date/27304
**Category:** Logstash
**Created:** [August 13, 2015, 7:13am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-field-string-to-match-system-date/27304 "2015-08-13T07:13:30Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Cjtow](https://avatars.discourse-cdn.com/v4/letter/c/7ab992/32.png) [@Cjtow](https://discuss.elastic.co/u/Cjtow)
#### Post date: [August 13, 2015, 7:13am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-field-string-to-match-system-date/27304/1 "2015-08-13T07:13:31Z")

</div>

Good morning/afternoon/evening/hi,

**using logstash version 1.5.3** Sorry if this has been bought up before, I have been searching for a while and can't find anything.

_Life story, you don't need to read_  
I've been working on a pretty unique project where I have a logstash server on Centos which pulls logs from another department's elasticsearch box, which then forwards to syslog to a security box where the logs are stored and correlated.

_may want to read for background_  
I have been having a couple of issues, as I want to run logstash all the time in the background. I have tried running the bin/logstash agent -f x.conf but I can't point it to logstash-\* because there's so much data, I can't even point it to logstash-2015-08-1\* because there's so much data. It times out and throws up an Exception in Thread Unsupported Operation Exception message.

The query that I run only takes a very small amount of data from the server. The only way the above works is if it I point it to today (logstash-2015-08-12 for example). That's fine.

_pls read this bit_  
What I want to do: I want to be able for that script to run in the background all the time against logstash-YYYY-MM-DD is that possible? I get errors whenever I try.

I've seen in older versions of elasticsearch.rb (ex: [https://www.omniref.com/github/elasticsearch/logstash/1.0.9/files/lib/logstash/outputs/elasticsearch.rb#line=18](https://www.omniref.com/github/elasticsearch/logstash/1.0.9/files/lib/logstash/outputs/elasticsearch.rb#line=18)) that there's syntax for it. But when I try, I get an error.

```
  # The index to write events to. This can be dynamic using the %{foo} syntax.
  # The default value will partition your indeces by day so you can more easily
  # delete old data or only search specific date ranges.
  config :index, :validate => :string, :default => **"logstash-%{+YYYY.MM.dd}"**

```

If I was to use that example above I get the error message:  
_Error: [404] {"error":"IndexMissingException[[logstash-%{+YYYY.MM.dd}] missing]","status":404} {:level=\>:error}_

is there a way of doing this, once this is working I can get to work of having this script running all the time. Please could someone help, I've been working on this a while with no previous experience so I just keep poking it until it works.

Thank you for your time,

- Chris

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [August 13, 2015, 8:30am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-field-string-to-match-system-date/27304/2 "2015-08-13T08:30:12Z")

</div>

What does your config with the query look like?

---

<div class="post-metadata">

### Author: ![Cjtow](https://avatars.discourse-cdn.com/v4/letter/c/7ab992/32.png) [@Cjtow](https://discuss.elastic.co/u/Cjtow)
#### Post date: [August 13, 2015, 9:21am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-field-string-to-match-system-date/27304/3 "2015-08-13T09:21:59Z")

</div>

Hi Mark, thanks for the quick response, here's my input. Note I've commented out the index part. It's also worth noting that I can get the same results without the query unless I set it to today's date.

```
input {
  elasticsearch {
    hosts => "servername.x.x"
   #index => "logstash-%{+YYYY.MM.dd}"
    query => '{ "query": { "filtered": { "query": { "bool": { "should": [{ "query_string": { "query": "program:sshd AND NOT @Feature:logstash" } }, { "query_string": { "query": "@LogType:eventlog AND Channel:Security" } }] } } } } }'
    codec => json
  }
}

```

Ideally, it would be good to use the maths from the Elasticsearch output so the config will always get the logs for today. [https://www.elastic.co/guide/en/elasticsearch/reference/master/date-math-index-names.html](https://www.elastic.co/guide/en/elasticsearch/reference/master/date-math-index-names.html)

---

<div class="post-metadata">

### Author: ![Cjtow](https://avatars.discourse-cdn.com/v4/letter/c/7ab992/32.png) [@Cjtow](https://discuss.elastic.co/u/Cjtow)
#### Post date: [August 13, 2015, 10:24am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-field-string-to-match-system-date/27304/4 "2015-08-13T10:24:51Z")

</div>

Not to worry, I've got around it by putting this in the elasticsearch.rb file

```
  # The index or alias to search.
  d = Time.now
  d = d.strftime("%y.%m.%d")

  config :index, :validate => :string, :default => "logstash-*" + d

```

hope that helps someone one day.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:31am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-field-string-to-match-system-date/27304/5 "2017-07-06T05:31:59Z")

</div>


