# Ending logstash after csv parsing

**URL:** <https://discuss.elastic.co/t/ending-logstash-after-csv-parsing/185232>\
**Category:** Logstash\
**Created:** [June 11, 2019, 3:40pm UTC](https://discuss.elastic.co/t/ending-logstash-after-csv-parsing/185232 "2019-06-11T15:40:33Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![juanc37](https://avatars.discourse-cdn.com/v4/letter/j/9de0a6/32.png) [@juanc37](https://discuss.elastic.co/u/juanc37)\
**Post date:** [June 11, 2019, 3:40pm UTC](https://discuss.elastic.co/t/ending-logstash-after-csv-parsing/185232/1 "2019-06-11T15:40:33Z")

</div>

I have a csv file that I need to index in my ES instance and I only need for logstash to read through to the file end and then stop the service. Is this possible in Elastic Stack 7?  
My config file looks like this:

```
input{
        file {
                path => "C:/Users/.../empData1.csv"
		start_position => "beginning"
		sincedb_path => "NUL"
        }
}

filter{
        csv{
                separator => ","
                columns => ["X,Y,Z"]
        }
	mutate {convert => ["X", "integer"]}
        mutate {convert => ["Y", "integer"]}
}

output {
        elasticsearch{
                hosts => ["localhost:9200"]
                index => "name"
                document_type => "otherName"
        }
        stdout{}
}

```

I turned on debug logging and once the file finished, I kept getting these four lines in a repeating pattern until I manually killed the process:

```
[2019-06-11T08:19:34,895][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu

[2019-06-11T08:19:34,978][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}

[2019-06-11T08:19:34,979][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}

[2019-06-11T08:19:35,537][DEBUG][org.logstash.execution.PeriodicFlush] Pushing flush onto pipeline.

```

My best alternative solution is to throw this debug output into a log file and look for the repeated occurrence of these lines then send a kill signal to the program. Is there a better way I could stop Logstash after it is done parsing through my csv

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 11, 2019, 4:02pm UTC](https://discuss.elastic.co/t/ending-logstash-after-csv-parsing/185232/2 "2019-06-11T16:02:06Z")

</div>

> [@juanc37](#):
>
> I have a csv file that I need to index in my ES instance and I only need for logstash to read through to the file end and then stop the service. Is this possible in Elastic Stack 7?

It can be done if you cat the file into a stdin input. It cannot be done using a file input.

---

<div class="post-metadata">

**Author:** ![juanc37](https://avatars.discourse-cdn.com/v4/letter/j/9de0a6/32.png) [@juanc37](https://discuss.elastic.co/u/juanc37)\
**Post date:** [June 11, 2019, 4:09pm UTC](https://discuss.elastic.co/t/ending-logstash-after-csv-parsing/185232/3 "2019-06-11T16:09:57Z")

</div>

Lets say that I am reading the data from an AWS S3 bucket and am changing the input

```
input{
        s3 {
            bucket => "pdcs-dump-test"
            access_key_id => "xxx"
            secret_access_key => "yyy"
            region => "us-west-2"
    }
}

```

would there be something different I could do?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 11, 2019, 4:12pm UTC](https://discuss.elastic.co/t/ending-logstash-after-csv-parsing/185232/4 "2019-06-11T16:12:37Z")

</div>

No, I think stdin is the only input that will cause logstash to exit when it is finished.

---

<div class="post-metadata">

**Author:** ![juanc37](https://avatars.discourse-cdn.com/v4/letter/j/9de0a6/32.png) [@juanc37](https://discuss.elastic.co/u/juanc37)\
**Post date:** [June 13, 2019, 11:37pm UTC](https://discuss.elastic.co/t/ending-logstash-after-csv-parsing/185232/5 "2019-06-13T23:37:56Z")

</div>

You can actually set watch\_for\_new\_files to false in the s3 plugin and that will terminate the logstash process upon completion of going through data.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 14, 2019, 12:14am UTC](https://discuss.elastic.co/t/ending-logstash-after-csv-parsing/185232/6 "2019-06-14T00:14:25Z")

</div>

Oh, you are right. It [overrides](https://github.com/logstash-plugins/logstash-input-s3/blob/96f7e4a9a36f56681161ea176b37d4913f7fdef7/lib/logstash/inputs/s3.rb#L119) stop rather than calling do\_stop directly as [stdin](https://github.com/logstash-plugins/logstash-input-stdin/blob/42f817af17fb3107cfb6e93e528bccbb893097e9/lib/logstash/inputs/stdin.rb#L67) does, but it does shut logstash down.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 12, 2019, 12:14am UTC](https://discuss.elastic.co/t/ending-logstash-after-csv-parsing/185232/7 "2019-07-12T00:14:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.