Csvfile Plugin

Elastic Stack Logstash
1

I need to process CSV files with first line schemas (that can vary from file-to-file) and after reading a bit see that improved CSV handling is on the "to do" list. I got to work and coded up a subclass of logstash-input-file that adds CSV parsing with a first-line-schema mode. The basic CSV parsing is largely borrowed from logstash-filter-csv, extended with a first-line-schema mode.

Code is at https://github.com/jweite/logstash-input-csvfile1. It should be considered strictly alpha at this stage, but I'd be interested in having it considered for submission. Any pre-pull-request review / guidance is appreciated.

PS, while I initially considered enhancing logstash-filter-csv I ultimately concluded that the only 100% reliable way to restart stream processing mid-file was to re-read the file's schema row, something that only the file input plugin can always do.

2

If you'd like to contribute (because we want you to as well!) then check out https://github.com/elastic/logstash/blob/master/CONTRIBUTING.md

3

Thanks Mark. I've been following the https://www.elastic.co/guide/en/logstash/current/contributing-to-logstash.html process (specifically the one on Input Plugins), which is linked by the doc you pointed out. It looks like my next step is to put an issue in GitHub. (I haven't published the plugin on RubyGems yet because I'd rather get at least one other set of eyes on the code before releasing it, given that I'm new at Logstash Plugin creation).