# Logstash Issue - export kafka messages into tsv format

**URL:** <https://discuss.elastic.co/t/logstash-issue-export-kafka-messages-into-tsv-format/309321>\
**Category:** Logstash\
**Created:** [July 11, 2022, 1:04pm UTC](https://discuss.elastic.co/t/logstash-issue-export-kafka-messages-into-tsv-format/309321 "2022-07-11T13:04:04Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aniketdatir](https://avatars.discourse-cdn.com/v4/letter/a/8e8cbc/32.png) [@aniketdatir](https://discuss.elastic.co/u/aniketdatir)\
**Post date:** [July 11, 2022, 1:04pm UTC](https://discuss.elastic.co/t/logstash-issue-export-kafka-messages-into-tsv-format/309321/1 "2022-07-11T13:04:04Z")

</div>

Hi Elastic team,

**I am trying to export Kafka messages into tab separated file (tsv) file using below logstash configuration file......But not able to get desired output.**

input {  
kafka {  
bootstrap\_servers =\> "host:port"  
topics =\> ["kafka\_topic"]  
max\_poll\_records =\>500  
}  
}  
output {  
csv {  
fields =\> ["message","@timestamp"]  
csv\_options =\> {"write\_headers" =\> "true" "headers" =\>["message","@timestamp"] "col\_sep" =\> "\t"}  
path =\> "/tmp/abc.tsv"  
}  
}

**Below are the problems facing using above conf file**

1. not able to export in tsv file.' \t' is appending before data in file.  
for ex. a\tb instead a b.
2. headers are repeating after every row  
for ex. a\tb  
1\t2  
a\tb  
3\t4
3. why these many opening and closing happened while running this configuration

[INFO] 2022-07-11 13:28:08.522 [[main]\>worker2] csv - Opening file {:path=\>"/tmp/abc.tsv"}  
[INFO] 2022-07-11 13:28:47.067 [[main]\>worker1] csv - Closing file /tmp/abc.tsv  
[INFO] 2022-07-11 13:29:27.352 [[main]\>worker0] csv - Opening file {:path=\>"/tmp/abc.tsv"}  
[INFO] 2022-07-11 13:29:42.042 [[main]\>worker0] csv - Closing file /tmp/abc.tsv

desired output is  
for ex. a b  
1 2  
3 4

Kindly let me know what I am missing & correct me  
Thanks

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 11, 2022, 3:22pm UTC](https://discuss.elastic.co/t/logstash-issue-export-kafka-messages-into-tsv-format/309321/2 "2022-07-11T15:22:23Z")

</div>

1. Ruby code (at least in some places) would substitute the \t with a tab character. The logstash configuration compiler does not do that by default -- you need a literal tab in your configuration. There is a 7 year old [open issue](https://github.com/logstash-plugins/logstash-output-csv/issues/2) on this. Enabling the config.support\_escapes option in logstash.yml may fix this.

2. Yes, headers are repeated. There is a 6 year old [open issue](https://github.com/logstash-plugins/logstash-output-csv/issues/8) on this.

3. Well, this one I can help on. The csv output borrows the close\_stale\_files method from the file output, and the file output has option to control how often this is called. This is a minimum time, it may take more than 60 seconds for the file to be closed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 8, 2022, 3:23pm UTC](https://discuss.elastic.co/t/logstash-issue-export-kafka-messages-into-tsv-format/309321/3 "2022-08-08T15:23:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
