# Logstash batch: write to a csv file and close it, or exit when done

**URL:** <https://discuss.elastic.co/t/logstash-batch-write-to-a-csv-file-and-close-it-or-exit-when-done/120245>\
**Category:** Logstash\
**Created:** [February 16, 2018, 8:29pm UTC](https://discuss.elastic.co/t/logstash-batch-write-to-a-csv-file-and-close-it-or-exit-when-done/120245 "2018-02-16T20:29:13Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jason\_Brooks](https://avatars.discourse-cdn.com/v4/letter/j/f04885/32.png) [@Jason\_Brooks](https://discuss.elastic.co/u/Jason_Brooks)\
**Post date:** [February 16, 2018, 8:29pm UTC](https://discuss.elastic.co/t/logstash-batch-write-to-a-csv-file-and-close-it-or-exit-when-done/120245/1 "2018-02-16T20:29:13Z")

</div>

Hello,

I have used logstash to build csv files, but I noticed i needed to exit logstash to be certain the output file is closed. While I am ok with this, it seems silly to have to shutdown logstash. The enclosed example is one that succeeded, so the answer isn't time critical.

I am running logstash 6.1 on linux ubuntu 16.04LTS server.

The task: read in a list of ip addresses from a file, and write them out with their locations.

input {  
file {  
path=\>["/tmp/list\_of\_ips.csv"]  
start\_position =\> beginning  
sincedb\_path =\> "/dev/null"  
}  
}  
filter {  
csv {  
columns =\> [  
"IP Address",  
"COUNTRY",  
"STATE",  
"CITY"  
]  
}  
geoip {  
source =\> "IP Address"  
}  
}  
output {  
csv {  
path =\> "/tmp/intel.output.csv"  
fields =\> [  
"IP Address",  
"[geoip][country\_name]",  
"[geoip][region\_name]",  
"[geoip][city\_name]"  
]  
}  
}

my question (multipart?)

Is there a way to have logstash complete (close) a csv file output when the input EOF is reached, or is exiting logstash the only option?  
Would this change if the input was elasticsearch?

--jason

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 19, 2018, 7:34am UTC](https://discuss.elastic.co/t/logstash-batch-write-to-a-csv-file-and-close-it-or-exit-when-done/120245/2 "2018-02-19T07:34:03Z")

</div>

> Is there a way to have logstash complete (close) a csv file output when the input EOF is reached, or is exiting logstash the only option?

Probably, but why does it matter to you whether the file is closed or not?

> Would this change if the input was elasticsearch?

Nope.

---

<div class="post-metadata">

**Author:** ![Jason\_Brooks](https://avatars.discourse-cdn.com/v4/letter/j/f04885/32.png) [@Jason\_Brooks](https://discuss.elastic.co/u/Jason_Brooks)\
**Post date:** [February 19, 2018, 5:59pm UTC](https://discuss.elastic.co/t/logstash-batch-write-to-a-csv-file-and-close-it-or-exit-when-done/120245/3 "2018-02-19T17:59:37Z")

</div>

It matters because I don't know how often buffers in logstash are flushed. If I can be certain the output buffers are empty, it should be fine as-is. Historically (unix/linux-wise) it's considered a BAD THING (tm) to expect a file is complete even if it's not been closed.

--jason

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 19, 2018, 6:51pm UTC](https://discuss.elastic.co/t/logstash-batch-write-to-a-csv-file-and-close-it-or-exit-when-done/120245/4 "2018-02-19T18:51:44Z")

</div>

> It matters because I don't know how often buffers in logstash are flushed.

The flush interval is configurable in the file output. I don't think there's any additional buffering in the JVM layers, but double-checking wouldn't hurt.

> Historically (unix/linux-wise) it's considered a BAD THING (tm) to expect a file is complete even if it's not been closed.

Well, there are a lot of historical things that are no longer true.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 19, 2018, 6:51pm UTC](https://discuss.elastic.co/t/logstash-batch-write-to-a-csv-file-and-close-it-or-exit-when-done/120245/5 "2018-03-19T18:51:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.