# Autogenerating field names from csv headers **URL:** \ **Category:** Logstash\ **Created:** [December 14, 2020, 8:01pm UTC](https://discuss.elastic.co/t/autogenerating-field-names-from-csv-headers/258638 "2020-12-14T20:01:23Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![toucan](https://avatars.discourse-cdn.com/v4/letter/t/58f4c7/32.png) [@toucan](https://discuss.elastic.co/u/toucan)\ **Post date:** [December 14, 2020, 8:01pm UTC](https://discuss.elastic.co/t/autogenerating-field-names-from-csv-headers/258638/1 "2020-12-14T20:01:23Z")
Hi, I'm trying to parse CSV files with Logstash. Some of the files have a column called _time_, others have two columns called _time\_from_ and _time\_to_ (and they have several other columns). Ideally I'd like to have one pipeline parsing all of these files and autogenerate the field names. I've tried both the CSV filter and the CSV codec and found the following behaviors: With the CSV filter, if Logstash stops in the middle of a file and is restarted, it takes whatever is in the first unread line of the file as the headers and uses those for the field names. This is obviously not intended. The behavior of the CSV codec is problematic in other ways: It uses the header of the first file for the field names and for all the other lines (from all the files), it generates a document (even for the header lines of the other documents). This is also not what I want. What's the best way to solve my problem? Am I using the codec or the filter plugin incorrectly or is there another option I could try? Thanks ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [December 14, 2020, 9:03pm UTC](https://discuss.elastic.co/t/autogenerating-field-names-from-csv-headers/258638/2 "2020-12-14T21:03:05Z")
If you want different columns for different events you are going to have to use a filter and conditionals. See [here](https://discuss.elastic.co/t/how-to-parse-mix-json-logs/167594/28) for an example. Possibly conditional on the filename rather than a pattern. ---
**Author:** ![toucan](https://avatars.discourse-cdn.com/v4/letter/t/58f4c7/32.png) [@toucan](https://discuss.elastic.co/u/toucan)\ **Post date:** [December 15, 2020, 7:04am UTC](https://discuss.elastic.co/t/autogenerating-field-names-from-csv-headers/258638/3 "2020-12-15T07:04:02Z")
Thanks for your answer. So if I understand you correctly, there's no way to detect the field names automatically from the headers (unless they are consistent over all files)? So if I had 100 different headers in my CSV files (fortunately I don't), I would have to write a conditional with 100 splits? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [December 15, 2020, 3:13pm UTC](https://discuss.elastic.co/t/autogenerating-field-names-from-csv-headers/258638/4 "2020-12-15T15:13:55Z")
> [@toucan](#): > > So if I had 100 different headers in my CSV files (fortunately I don't), I would have to write a conditional with 100 splits? Yes, you would. ---
**Author:** ![toucan](https://avatars.discourse-cdn.com/v4/letter/t/58f4c7/32.png) [@toucan](https://discuss.elastic.co/u/toucan)\ **Post date:** [December 17, 2020, 9:27am UTC](https://discuss.elastic.co/t/autogenerating-field-names-from-csv-headers/258638/5 "2020-12-17T09:27:07Z")
Ok, thanks, I got it to work with conditionals and the dissect filter. Of course, had the CSV codec or filter worked as hoped, my configuration would be much shorter 😃 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 14, 2021, 9:27am UTC](https://discuss.elastic.co/t/autogenerating-field-names-from-csv-headers/258638/6 "2021-01-14T09:27:16Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.