# Trying to avoid manual changes to files pre-logstash **URL:** \ **Category:** Logstash\ **Created:** [January 2, 2018, 7:03pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824 "2018-01-02T19:03:46Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![dorj1234](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dorj1234/32/21339_2.png) [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Post date:** [January 2, 2018, 7:03pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/1 "2018-01-02T19:03:46Z")
Hello, I have csv files that include bad output from SQL output that logstash fails to parse because of quote marks that stay open. Example line: `"1";"2";"3";"N;"N;"N;"N;"N;"N;"N;"N;"N;"N;"N;"N;"N;"N` I am fixing them manually with sed before running logstash. `sed -i 's/"N;/"N";/g' *.csv` `sed -i 's/\"N$/"N"/g' *.csv` Using logstash conf only, how can I run these sed commands or change the file using these sed patterns before logstash tries to ingest them? I saw the "exec" plugin, but need to run those before logstash parses new data, not at a set interval. Thanks in advance! JD ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 2, 2018, 9:20pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/2 "2018-01-02T21:20:24Z")
You can use a mutate filter (its gsub option) to process the input line before the csv filter. ---
**Author:** ![dorj1234](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dorj1234/32/21339_2.png) [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Post date:** [January 3, 2018, 6:30pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/3 "2018-01-03T18:30:58Z")
Thank you, I would really appreciate an understanding of how to escape characters using this method. Right now I fail. Even when trying to set this in logstash.yml: config.support\_escapes: true Using double quotes as an example, how do I escape " inside a conf file gsub clause? Thanks ! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 3, 2018, 6:58pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/4 "2018-01-03T18:58:02Z")
You can use the double quotes without escapes if you make the string itself single-quoted. ---
**Author:** ![dorj1234](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dorj1234/32/21339_2.png) [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Post date:** [January 4, 2018, 4:04pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/5 "2018-01-04T16:04:47Z")
OK that will do it for most of the bad line, but I have to escape the end of line because of the last "N So the equivalent of using $ in sed is also needed for the conf file. Any chance you can think of a way to escape it or an alternative? Thanks ! JD ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 4, 2018, 6:55pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/6 "2018-01-04T18:55:32Z")
Why would there be a need to escape the dollar sign? ---
**Author:** ![dorj1234](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dorj1234/32/21339_2.png) [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Post date:** [January 4, 2018, 6:57pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/7 "2018-01-04T18:57:23Z")
I don't need to escape the dollar sign, I need to escape the "end of line" symbol. In sed that symbol is a dollar sign ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 4, 2018, 9:13pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/8 "2018-01-04T21:13:42Z")
Yes, I know the meaning of a dollar sign in a regular expression. And if you want the dollar sign to mean "end of line" why would you want to escape it? ---
**Author:** ![dorj1234](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dorj1234/32/21339_2.png) [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Post date:** [January 4, 2018, 9:16pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/9 "2018-01-04T21:16:37Z")
Now I understand the question, sorry. I need to replace end-of-line with a double quote inside mutate-\>gsub Instead of "N (--\>end of line) I need to create "N" Inside gsub ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 4, 2018, 9:20pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/10 "2018-01-04T21:20:41Z")
Just use the same expression as with sed apart from your (unnecessary) escaping of the double quote, i.e. `"N$`. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 1, 2018, 9:20pm UTC](https://discuss.elastic.co/t/trying-to-avoid-manual-changes-to-files-pre-logstash/113824/11 "2018-02-01T21:20:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.