# Reading the properties file from input section of logstash

**URL:** <https://discuss.elastic.co/t/reading-the-properties-file-from-input-section-of-logstash/108931>\
**Category:** Logstash\
**Created:** [November 23, 2017, 5:58pm UTC](https://discuss.elastic.co/t/reading-the-properties-file-from-input-section-of-logstash/108931 "2017-11-23T17:58:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![kiranilla](https://avatars.discourse-cdn.com/v4/letter/k/8e7dd6/32.png) [@kiranilla](https://discuss.elastic.co/u/kiranilla)\
**Post date:** [November 23, 2017, 5:58pm UTC](https://discuss.elastic.co/t/reading-the-properties-file-from-input-section-of-logstash/108931/1 "2017-11-23T17:58:19Z")

</div>

Hi,

I am able to read the properties file from filter section of logstash but not from input section.  
Could you please help me out in reading the .properties file from input section of logstash.  
Thanks in advance.

Regards,  
Kiran.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [November 23, 2017, 6:55pm UTC](https://discuss.elastic.co/t/reading-the-properties-file-from-input-section-of-logstash/108931/2 "2017-11-23T18:55:37Z")

</div>

You will need to give us more info. Please explain in detail. I don't know what you mean by `properties file from the filter`

---

<div class="post-metadata">

**Author:** ![kiranilla](https://avatars.discourse-cdn.com/v4/letter/k/8e7dd6/32.png) [@kiranilla](https://discuss.elastic.co/u/kiranilla)\
**Post date:** [November 24, 2017, 6:23am UTC](https://discuss.elastic.co/t/reading-the-properties-file-from-input-section-of-logstash/108931/3 "2017-11-24T06:23:05Z")

</div>

HI @guyboertje,

Thanks for your reply.

Here is the example i worked on :  
**sample.properties**

```
    AGSS,URO=http://localhost:8980/akn/
    AGCM,FTP,SMTP=http://localhost:8800/akn/gsk/se

```

1. I read the properties file from filter section of logstash file using ruby code: (which is possible)

2. In the above scenario i am able to read the properties file and getting the output, but the properties file is continuously read every time based on csv rows in a file (eg: csv file contains 100 rows the filter section will execute 100 times).

**3) Instead of i would like to read properties file only once in the input section of the logstash and take the content of the properties file(ie.,URLs) in it and place them in the "URL" plugin in the input section of logstash.**

something like below code of input section of logstash:

> input {  
> http\_poller {  
> urls =\> {  
> EL=\> {  
> method =\> get  
> **url =\> "[http://localhost:8980/akn](http://localhost:8980/akn)"** # This url should be dynamic  
> verify\_cert =\> false  
> headers =\> {  
> Accept =\> "application/json"  
> }  
> }  
> }

**\_url =\> "[http://localhost:8980/akn\_](http://localhost:8980/akn_)**  
**_The "url" should be dynamic (which need to be taken from the properties file)_**

I checked for logstash api where "ruby plugin" is available in the filter section but its not available in the input section.

It would be great if you can you please help me in this scenario,  
Thanks in advance.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [November 24, 2017, 10:52am UTC](https://discuss.elastic.co/t/reading-the-properties-file-from-input-section-of-logstash/108931/4 "2017-11-24T10:52:17Z")

</div>

There is no ruby input - only a ruby filter. Plugins of type `input` are put in the input section of the config, similarly for `filter`.

You don't need to go to your proposed extremes.  
You can use a translate filter and a json filter.  
Write a little ruby/python script to extract and denormalise the `sample.properties` file into an `expanded_properties.csv` file.

```auto
AGSS,http://localhost:8980/akn/
URO,http://localhost:8980/akn/
AGCM,http://localhost:8800/akn/gsk/se
FTP,http://localhost:8800/akn/gsk/se
SMTP,http://localhost:8800/akn/gsk/se

```

I presume that there are a fixed number of URLs in this file so now your script can fetch the JSON from the URLs and replace the URL with the JSON body.

```auto
AGSS,{"foo": "bar"}
URO,{"foo": "bar"}
AGCM,{"baz": "bar"}
FTP,{"baz": "bar"}
SMTP,{"baz": "bar"}

```

Then using a translate filter you can enhance the event with the JSON. After the translate filter you use a JSON filter, it will unpack the JSON field and add the KV pairs.

[https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html)

Once you have this working you can experiment with not denormalising the properties file and writing the LHS as a regex instead see: [https://stackoverflow.com/questions/41522280/how-to-use-regex-in-yaml-file-for-logstash-translate-filter](https://stackoverflow.com/questions/41522280/how-to-use-regex-in-yaml-file-for-logstash-translate-filter).  
This for your translate filter dictionary\_path file contents maybe:

```auto
^(URO|AGSS)$,{"foo": "bar"}
^(AGCM|FTP|SMTP)$,{"baz": "bar"}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 22, 2017, 10:52am UTC](https://discuss.elastic.co/t/reading-the-properties-file-from-input-section-of-logstash/108931/5 "2017-12-22T10:52:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
