# Dynamic variable for http poller input plugin

**URL:** <https://discuss.elastic.co/t/dynamic-variable-for-http-poller-input-plugin/223912>\
**Category:** Logstash\
**Created:** [March 17, 2020, 11:19am UTC](https://discuss.elastic.co/t/dynamic-variable-for-http-poller-input-plugin/223912 "2020-03-17T11:19:38Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![elk2](https://avatars.discourse-cdn.com/v4/letter/e/f14d63/32.png) [@elk2](https://discuss.elastic.co/u/elk2)\
**Post date:** [March 17, 2020, 11:19am UTC](https://discuss.elastic.co/t/dynamic-variable-for-http-poller-input-plugin/223912/1 "2020-03-17T11:19:38Z")

</div>

I want to download and parse a list of csv with yearly weather data with logstash.  
"YEAR" would be my variable and I need to increase from 1998 to 2019.  
I tried to use this simple conf file but not even works.

```
input {
  http_poller {
    urls => {
      test1 => "http://weather.uwaterloo.ca/download/Daily_summary_${YEAR:1998}.csv"
     }
    request_timeout => 60
    schedule => { cron => "* * * * * UTC"}
    codec => "plain"
    metadata_target => "http_poller_metadata"
  }
}

filter{

}

output {
  stdout {
    codec => rubydebug
  }
  file {
     path => "/var/log/logstash/wheater-%{YEAR}.log"
  }
}
```

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [March 18, 2020, 8:32am UTC](https://discuss.elastic.co/t/dynamic-variable-for-http-poller-input-plugin/223912/2 "2020-03-18T08:32:56Z")

</div>

Hi there,

what do you mean by "not even works"? What does it do?

Anyway, AFAIK, the only way to pass variables to Logstash is through environment variables.  
If you do not have a env variable YEAR specified, it should take the default one you passed (1998).

Obviously it won't find anything in your output section, so I'd expect it to write something in the `/var/log/logstash/wheater-%{YEAR}.log` literal path.

But what does it write in your standard output?

---

<div class="post-metadata">

**Author:** ![elk2](https://avatars.discourse-cdn.com/v4/letter/e/f14d63/32.png) [@elk2](https://discuss.elastic.co/u/elk2)\
**Post date:** [March 18, 2020, 9:15am UTC](https://discuss.elastic.co/t/dynamic-variable-for-http-poller-input-plugin/223912/3 "2020-03-18T09:15:08Z")

</div>

> [@Fabio-sama](#):
>
> what do you mean by "not even works"? What does it do?

I changed codec in plain. It was wrong.

> [@Fabio-sama](#):
>
> Obviously it won't find anything in your output section, so I'd expect it to write something in the `/var/log/logstash/wheater-%{YEAR}.log` literal path.

Correct .

> [@Fabio-sama](#):
>
> But what does it write in your standard output?

With codec plain logstash save file and data from Daily\_summary\_1998.csv file

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [March 18, 2020, 10:01am UTC](https://discuss.elastic.co/t/dynamic-variable-for-http-poller-input-plugin/223912/4 "2020-03-18T10:01:47Z")

</div>

Ok so, it does what it is expected to be done. I don't really get why you're using Logstash to do something that can easily be done with a script.

I'd use a script (where you can make dynamic variables) to save those csv files and then I'd parse them with Logstash if needed.

It definitely is a better solution rather than storing I don't know how many env variables or launch Logstash so many times.

---

<div class="post-metadata">

**Author:** ![elk2](https://avatars.discourse-cdn.com/v4/letter/e/f14d63/32.png) [@elk2](https://discuss.elastic.co/u/elk2)\
**Post date:** [March 20, 2020, 1:08pm UTC](https://discuss.elastic.co/t/dynamic-variable-for-http-poller-input-plugin/223912/5 "2020-03-20T13:08:15Z")

</div>

Yes. I thought there was a simple solution to do this but it seems that it's not.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2020, 1:08pm UTC](https://discuss.elastic.co/t/dynamic-variable-for-http-poller-input-plugin/223912/6 "2020-04-17T13:08:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.