# Logstash slow in bash script launch by cronjob

**URL:** https://discuss.elastic.co/t/logstash-slow-in-bash-script-launch-by-cronjob/299103
**Category:** Logstash
**Created:** [March 8, 2022, 2:14pm UTC](https://discuss.elastic.co/t/logstash-slow-in-bash-script-launch-by-cronjob/299103 "2022-03-08T14:14:26Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Henri\_L](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/henri_l/32/102762_2.png) [@Henri\_L](https://discuss.elastic.co/u/Henri_L)
#### Post date: [March 8, 2022, 2:14pm UTC](https://discuss.elastic.co/t/logstash-slow-in-bash-script-launch-by-cronjob/299103/1 "2022-03-08T14:14:26Z")

</div>

Hi I have a bash script like that:

```auto
#!/bin/bash

cat /home/xxx/public_html/datas/zz/file.csv | /usr/share/logstash/bin/logstash -f /home/xxx/public_html/datas/myconf.conf

```

My conf file is basic like that:

```auto
input {
    stdin {}
}
filter {
		csv {
			columns => [
			"id","link","title","description","hashdesc","publicationDate","geo_lat","geo_lng","city","postalCode","department","region","sector","jobtitle","company","contractType","salary","affiliateCpc"
			]
			separator => ","
			skip_empty_columns => false
		}
      mutate {
        add_field => {
          "[pin][location][lat]" => "%{geo_lat}"
				  "[pin][location][lon]" => "%{geo_lng}"
        }
      }
			mutate {
			  convert => {
				"[pin][location][lat]" => "float"
				"[pin][location][lon]" => "float"
        "[cpc]" => "float"
			  }
			}
		mutate {
		  remove_field => ["message", "host", "@version", "path", "geo_lng", "geo_lat", "event", "log"]
		}

}
output {
		elasticsearch {
			hosts => "http://localhost:9200"
			index => "jobs3"
			document_id => "%{guid}"
			timeout => 30
			workers => 1
			doc_as_upsert => true
			action => "update"
		}
	stdout { codec => rubydebug }
}

```

This work perfectly with the shell but when I try to call this script in a cronjob it take a lot of time....

Any idea to speed up the cronjob like in the shell?

Thanks

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [March 8, 2022, 2:47pm UTC](https://discuss.elastic.co/t/logstash-slow-in-bash-script-launch-by-cronjob/299103/2 "2022-03-08T14:47:19Z")

</div>

How much is _a lot of time_ in this case? And what is your cron schedule?

Logstash can take a couple of time to start, this is expected, if this is an issue you should try to run logstash as a service and change your pipeline to read from files.

---

<div class="post-metadata">

### Author: ![Henri\_L](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/henri_l/32/102762_2.png) [@Henri\_L](https://discuss.elastic.co/u/Henri_L)
#### Post date: [March 8, 2022, 2:49pm UTC](https://discuss.elastic.co/t/logstash-slow-in-bash-script-launch-by-cronjob/299103/3 "2022-03-08T14:49:08Z")

</div>

With the shell it take 5 minutes and with cron one hour and 30 min

Cron schedule each 4 hours

My cron is call like that

`bin/bash /home/xxx/public_html/datas/elasticsearch/jobs.sh`

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [March 8, 2022, 3:14pm UTC](https://discuss.elastic.co/t/logstash-slow-in-bash-script-launch-by-cronjob/299103/4 "2022-03-08T15:14:21Z")

</div>

What version are you using and what is the server specs?

A start up time of 5 minutes is too long, but it could happen in some versions depending on the specs, there is an [old issue](https://github.com/elastic/logstash/issues/5507) about it related to jruby and the entropy of the system, but I did not experience this behavior in new versions.

But 1 hour and 30 minutes does not make much sense.

How are you measuring this time? Can you share logstash logs when it starts?

---

<div class="post-metadata">

### Author: ![Henri\_L](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/henri_l/32/102762_2.png) [@Henri\_L](https://discuss.elastic.co/u/Henri_L)
#### Post date: [March 8, 2022, 4:03pm UTC](https://discuss.elastic.co/t/logstash-slow-in-bash-script-launch-by-cronjob/299103/5 "2022-03-08T16:03:41Z")

</div>

No the start up of logstash take 5 seconds only, 5 minutes if for the script with 600000 new entries.  
I monitor the cron time easily I check the start and when the entries are in Elasticsearch... So 1h30 is approximate but significantly more than 5 minutes with the run in console.

Maybe losgstah is depriorize by cron, I have no idea

---

<div class="post-metadata">

### Author: ![Henri\_L](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/henri_l/32/102762_2.png) [@Henri\_L](https://discuss.elastic.co/u/Henri_L)
#### Post date: [March 8, 2022, 4:18pm UTC](https://discuss.elastic.co/t/logstash-slow-in-bash-script-launch-by-cronjob/299103/6 "2022-03-08T16:18:58Z")

</div>

I found the problem. It was the output in cron the bottleneck  
`stdout { codec => rubydebug }`

I simply remove the stdout and cron works like the shell way

Thanks for your help

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 5, 2022, 4:19pm UTC](https://discuss.elastic.co/t/logstash-slow-in-bash-script-launch-by-cronjob/299103/7 "2022-04-05T16:19:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
