# How to push file from github to logstash

**URL:** <https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844>\
**Category:** Logstash\
**Created:** [May 15, 2020, 2:40pm UTC](https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844 "2020-05-15T14:40:52Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![dheeru11](https://avatars.discourse-cdn.com/v4/letter/d/4bbf92/32.png) [@dheeru11](https://discuss.elastic.co/u/dheeru11)\
**Post date:** [May 15, 2020, 2:40pm UTC](https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844/1 "2020-05-15T14:40:53Z")

</div>

I have file which will be uploaded to github daily. I want push the same file to logstash. How can i do that ?

---

<div class="post-metadata">

**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\
**Post date:** [May 16, 2020, 6:59am UTC](https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844/2 "2020-05-16T06:59:04Z")

</div>

logstash can read file from filesystem

---

<div class="post-metadata">

**Author:** ![Luca\_Belluccini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luca_belluccini/32/33239_2.png) [@Luca\_Belluccini](https://discuss.elastic.co/u/Luca_Belluccini)\
**Post date:** [May 16, 2020, 7:08am UTC](https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844/3 "2020-05-16T07:08:13Z")

</div>

You can use the [HTTP Poller input](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html) to download the file periodically.

You can grab the URL of the file from GitHub using the "Raw" button (the url will be something like `https://raw.githubusercontent.com/user/repository/branch/filename`)

---

<div class="post-metadata">

**Author:** ![dheeru11](https://avatars.discourse-cdn.com/v4/letter/d/4bbf92/32.png) [@dheeru11](https://discuss.elastic.co/u/dheeru11)\
**Post date:** [May 18, 2020, 6:42am UTC](https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844/4 "2020-05-18T06:42:45Z")

</div>

Thanks.

But i have file format like filename\_{Date}.csv  
I will get one file everyday in .csv format and i need to upload that to logstash. Kindly share any conf sample file for the same.

---

<div class="post-metadata">

**Author:** ![Luca\_Belluccini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luca_belluccini/32/33239_2.png) [@Luca\_Belluccini](https://discuss.elastic.co/u/Luca_Belluccini)\
**Post date:** [May 18, 2020, 8:24am UTC](https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844/5 "2020-05-18T08:24:03Z")

</div>

If you have to change the URL every day, then I think you can use one of the following:

1. An `exec` input ([doc](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-exec.html)), where you put a script in your preferred language which clones/syncs the Github repository and you sort by latest file and you print the file to stdout

2. An `exec` input which generates the date as string, plus an `http` filter ([doc](https://www.elastic.co/guide/en/logstash/current/plugins-filters-http.html)) to download the file using the date generated in the `exec`.

---

<div class="post-metadata">

**Author:** ![dheeru11](https://avatars.discourse-cdn.com/v4/letter/d/4bbf92/32.png) [@dheeru11](https://discuss.elastic.co/u/dheeru11)\
**Post date:** [May 18, 2020, 9:05am UTC](https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844/6 "2020-05-18T09:05:51Z")

</div>

Hi Luca,

If i want to use the same file how can i modify my below conf file. Getting error as PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

input {  
http\_poller {  
urls =\> {  
urlname =\> "[https://path](https://path) to csv file"  
}  
request\_timeout =\> 60  
schedule =\> {every =\> "30s"}  
codec =\> "line"  
}  
}

filter {  
csv {  
separator =\> ","  
columns =\> ["Transaction Name","Minimum"]  
}  
mutate {convert =\> ["Transaction Name","string"]}  
mutate {convert =\> ["Minimum","float"]}

}

output {  
elasticsearch {  
hosts =\> ["[http://host:9200](http://host:9200)"]  
index =\> "results"  
}  
stdout {codec =\> rubydebug}  
}

---

<div class="post-metadata">

**Author:** ![Luca\_Belluccini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luca_belluccini/32/33239_2.png) [@Luca\_Belluccini](https://discuss.elastic.co/u/Luca_Belluccini)\
**Post date:** [May 18, 2020, 10:59am UTC](https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844/7 "2020-05-18T10:59:06Z")

</div>

Hello @dheeru11

Please follow the [documentation](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html#_example). In particular, there's a section about certificates called `Using the HTTP poller with custom a custom CA or self signed cert`.

In particular:

1. Download and generate the JKS containing the CA Root certificate:

2. Add to the `http_poller`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 15, 2020, 10:59am UTC](https://discuss.elastic.co/t/how-to-push-file-from-github-to-logstash/232844/8 "2020-06-15T10:59:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.