Reading a .csv file from a remote url using filebeat and httpjson

Elastic Stack Beats
1

Hi,

I'm trying to monitor a remote URL that contains a .csv file. I need to retrieve the file and make it available in Logstash.

I'm new to this, and I'm not sure how to get started. Can anyone offer some guidance or suggest some resources I can use to learn more about this process?

A have already tried the solution Filebeat: Configuring input from URL but it does not work for me.

At the moment, I’m using Filebeat and the httpjson plugin. My filebeat.yml looks like:

filebeat.inputs:
- type: httpjson
 interval: 1m
 request.url: http://localhost:8002/avaliation.csv
 request.method: GET
 processors:
   - decode_csv_fields:
     fields: ["message"]
     separator: ","
     ignore_missing: false
     overwrite_keys: true
     trim_leading_space: false
     fail_on_error: true
output.logstash:
 hosts: ["logstash:5045"]

And my logstash conf looks like:

input {
   beats {
       port => "5045"
   }
}
filter {
  csv {}
}
output {
   elasticsearch {
       hosts => "es01:9200"
       index => "beats"
   }
}

Thanks in advance for your help!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.