# Sending logs to .csv file using logstash? **URL:** \ **Category:** Logstash\ **Created:** [January 16, 2017, 5:14pm UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767 "2017-01-16T17:14:45Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Yaswanth](https://avatars.discourse-cdn.com/v4/letter/y/94ad74/32.png) [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Post date:** [January 16, 2017, 5:14pm UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/1 "2017-01-16T17:14:45Z")
Hi ES 2.4.0 Logstash 5.1.1 I want to send the slowlogs to .csv file using logstash . my config file is like this input { file { path =\> "D:\logstash-5.1.1\logstash-5.1.1\bin\slowlog.log" start\_position =\> "beginning" } } filter { grok { # parses the common bits match =\> ["message", "[%{TIMESTAMP\_ISO8601:TIMESTAMP}][%{LOGLEVEL:LEVEL}%{SPACE}][%{DATA:QUERY}]%{SPACE}[%{DATA:QUERY1}]%{SPACE}[%{DATA:INDEX-NAME}][%{DATA:SHARD}]%{SPACE}took[%{DATA:TOOK}],%{SPACE}took\_millis[%{DATA:TOOKM}], types[%{DATA:types}], stats[%{DATA:stats}], search\_type[%{DATA:search\_type}], total\_shards[%{NUMBER:total\_shards}], source[%{DATA:source\_query}], extra\_source[%{DATA:extra\_source}],"] } } output { csv { fields =\> ["TIMESTAMP","LOGLEVEL","QUERY","QUERY1","INDEX-NAME","SHARD","TOOK","took\_millis","types","stats","search\_type","total\_shards","source\_query","extra\_source"] path =\> "D:\logstash-5.1.1\logstash-5.1.1\bin\final.csv" } stdout { codec =\> rubydebug } } my output is coming in a irrregular format like ![](https://us1.discourse-cdn.com/elastic/original/2X/9/9236f12590484aae88d4692f3a44870042b57609.png)2017-01-17T06:35:26.236Z PC147594 [2017-01-13 12:58:09 843][WARN][index.search.slowlog.query] [Spectra] [testindex-stats][2] took[15.3ms] took\_millis[15] types stats search\_type[QUERY\_THEN\_FETCH] total\_shards[5] source[{"query":{"match":{"text":"ronin"}}}] extra\_source My logs format is [2017-01-13 21:21:03,956][WARN][index.search.slowlog.query] [Yaswanth] [bank][0] took[28.9ms], took\_millis[28], types, stats, search\_type[QUERY\_THEN\_FETCH], total\_shards[5], source, extra\_source, My output should be like every field in logs squarebrackets should in one ,one column like 2017-01-13 21:21:03,956 in one column WARN in one column index.search.slowlog.query in one column so on.. can anyone help me in solving this? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 17, 2017, 6:38am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/2 "2017-01-17T06:38:07Z")
> my output is coming in a irrregular format like mixing with other fields. Please don't attempt to describe what you get. Show us. Use copy/paste. ---
**Author:** ![Yaswanth](https://avatars.discourse-cdn.com/v4/letter/y/94ad74/32.png) [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Post date:** [January 17, 2017, 6:59am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/3 "2017-01-17T06:59:59Z")
hey, I edited the question using the copy paste but i think the picture is not clear. Let me know if you want the screenshot of it so that you will get the clear picture of my output. Thanks ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 17, 2017, 7:23am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/4 "2017-01-17T07:23:05Z")
I don't think the input is correctly parsed. Please show the results of your `stdout { codec => rubydebug }` output for one sample input message. No screenshots. ---
**Author:** ![Yaswanth](https://avatars.discourse-cdn.com/v4/letter/y/94ad74/32.png) [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Post date:** [January 17, 2017, 7:31am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/5 "2017-01-17T07:31:37Z")
sure magnusbaeck.. It is like this { "total\_shards" =\> "5", "LEVEL" =\> "DEBUG", "message" =\> "[2017-01-13 14:37:48,943][DEBUG][index.search.slowlog.query] [Spec tra] [test][2] took[46.1micros], took\_millis[0], types[], stats[], search\_type[QUERY\_THEN \_FETCH], total\_shards[5], source[], extra\_source[], \r", "SHARD" =\> "2", "search\_type" =\> "QUERY\_THEN\_FETCH", "tags" =\> [], "QUERY1" =\> "Spectra", "TOOK" =\> "46.1micros", "path" =\> "D:\logstash-5.1.1\logstash-5.1.1\bin\slowlog.log", "TOOKM" =\> "0", "@timestamp" =\> 2017-01-17T06:35:26.594Z, "INDEX-NAME" =\> "test", "QUERY" =\> "index.search.slowlog.query", "TIMESTAMP" =\> "2017-01-13 14:37:48,943", "@version" =\> "1", "host" =\> "PC147594" ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 17, 2017, 7:57am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/6 "2017-01-17T07:57:10Z")
Okay, that doesn't look unreasonable. And what does the resulting .csv file look like? No screenshots, use copy/paste from the text file. ---
**Author:** ![Yaswanth](https://avatars.discourse-cdn.com/v4/letter/y/94ad74/32.png) [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Post date:** [January 17, 2017, 8:23am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/7 "2017-01-17T08:23:07Z")
OUTPUT .CSV FILE: The below are the columns that are getting in my output .csv file. columns 2017-01-17T06:35:26.236Z PC147594 [2017-01-13 12:58:09 843][WARN][index.search.slowlog.query] [Spectra] [testindex-stats][2] took[15.3ms] took\_millis[15] types[] stats[] search\_type[QUERY\_THEN\_FETCH] total\_shards[5] source[{"query":{"match":{"text":"ronin"}}}] extra\_source[] But what i want is like this,i want the log contents in [] thats it columns 2017-01-17T06:35:26.236Z PC147594 2017-01-13 12:58:09,843 WARN index.search.slowlog.query And so on.. Thanks.. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 17, 2017, 8:29am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/8 "2017-01-17T08:29:29Z")
Okay. There's either a bug in the csv output or there's something going on in your configuration that I don't have time or patience to debug. In the future, please do exactly what's requested. Do not replace commas with newline characters unless asked to. ---
**Author:** ![Yaswanth](https://avatars.discourse-cdn.com/v4/letter/y/94ad74/32.png) [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Post date:** [January 17, 2017, 10:09am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/9 "2017-01-17T10:09:34Z")
Hi , One thing i want to know is that the csv has the default separator as "," how can i change it to "[]" Because that why the output format is not correct . Thanks. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 14, 2017, 10:09am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767/10 "2017-02-14T10:09:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.