# Gsub before CSV parsing

**URL:** <https://discuss.elastic.co/t/gsub-before-csv-parsing/129824>\
**Category:** Logstash\
**Created:** [April 27, 2018, 11:56am UTC](https://discuss.elastic.co/t/gsub-before-csv-parsing/129824 "2018-04-27T11:56:57Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![SHSauler](https://avatars.discourse-cdn.com/v4/letter/s/53a042/32.png) [@SHSauler](https://discuss.elastic.co/u/SHSauler)\
**Post date:** [April 27, 2018, 11:56am UTC](https://discuss.elastic.co/t/gsub-before-csv-parsing/129824/1 "2018-04-27T11:56:57Z")

</div>

Hi guys,

I'm trying to work around a [logstash-filter-csv](https://github.com/logstash-plugins/logstash-filter-csv/issues/64) issue. The line needs to be cleaned from whitespace and quotes before the CSV parser runs.

My current issue is probably that I don't know the "default field name" before it gets parsed? Could you please give me a hint?

Thanks a bunch!

```
input {
  file {
    path => "/home/user/folder/*.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
  mutate {
    gsub => [
     'message', '\"', '',
     'message', ',\s+', ','
    ]
  }

  csv {
    separator => ","
    columns => ["date","time","desc","long"]
  }

  mutate {
    add_field => {
      "timestamp" => "%{date} %{time}"
        }
  }

  date {
    match => ["timestamp", "MM/dd/yyyy HH:mm:ss"]
    remove_field => ["timestamp"]
  }

  mutate {
    remove_field => ["desc"]
  }
}

output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "project"
  }
stdout {}
}
```

---

<div class="post-metadata">

**Author:** ![SHSauler](https://avatars.discourse-cdn.com/v4/letter/s/53a042/32.png) [@SHSauler](https://discuss.elastic.co/u/SHSauler)\
**Post date:** [April 27, 2018, 2:25pm UTC](https://discuss.elastic.co/t/gsub-before-csv-parsing/129824/2 "2018-04-27T14:25:15Z")

</div>

Seems to work like this:

```
input {
  file {
    path => "/home/user/folder/*.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
  mutate {
    gsub => [
     'message', '\\"', '',
     'message', '\"', '',
     'message', ',\s+', ','
    ]
  }

  csv {
    separator => ","
    columns => ["date","time","desc","long"]
  }

  mutate {
    add_field => {
      "timestamp" => "%{date} %{time}"
        }
  }

  date {
    match => ["timestamp", "MM/dd/yyyy HH:mm:ss"]
    remove_field => ["timestamp"]
  }

  mutate {
    remove_field => ["desc"]
  }
}

output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "project"
  }
stdout {}
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 25, 2018, 2:25pm UTC](https://discuss.elastic.co/t/gsub-before-csv-parsing/129824/3 "2018-05-25T14:25:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.