# Implement concurrency while inserting data to ES using logstash

**URL:** <https://discuss.elastic.co/t/implement-concurrency-while-inserting-data-to-es-using-logstash/128120>\
**Category:** Logstash\
**Created:** [April 16, 2018, 7:56am UTC](https://discuss.elastic.co/t/implement-concurrency-while-inserting-data-to-es-using-logstash/128120 "2018-04-16T07:56:35Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![manjur](https://avatars.discourse-cdn.com/v4/letter/m/5f8ce5/32.png) [@manjur](https://discuss.elastic.co/u/manjur)\
**Post date:** [April 16, 2018, 7:56am UTC](https://discuss.elastic.co/t/implement-concurrency-while-inserting-data-to-es-using-logstash/128120/1 "2018-04-16T07:56:36Z")

</div>

Hi,

I am using below config file for logstash. I have used schedule= 1 min as jdbc scheduler which keep on inserting data to ES in every 1 min. But when number of records are more i am not able to identify weather it is following transaction or not . it should follow locking of resource data. How to implement locking kind of thing it in this case.

input {  
jdbc {  
jdbc\_driver\_library =\> "demo\sqljdbc\_6.4\enu\mssql-jdbc-6.4.0.jre8.jar"  
jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver"  
jdbc\_connection\_string =\> "jdbc:sqlserver://L-ff;databaseName=Cultivation;integratedSecurity=true;"  
jdbc\_user =\> "manjur.gani"  
statement\_filepath=\>"demo:\SQLQuery12.sql"  
schedule =\> "\* \* \* \* \*"  
tracking\_column =\> "sdate"  
last\_run\_metadata\_path =\> "demo.logstash\_jdbc\_last\_run"  
tracking\_column\_type =\> "timestamp"  
use\_column\_value =\> true  
type =\>farm  
}

}  
output  
{  
//code  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 16, 2018, 8:50am UTC](https://discuss.elastic.co/t/implement-concurrency-while-inserting-data-to-es-using-logstash/128120/2 "2018-04-16T08:50:17Z")

</div>

I'm not sure what you're asking. What transaction? What locking?

> But when number of records are more

More than what?

---

<div class="post-metadata">

**Author:** ![manjur](https://avatars.discourse-cdn.com/v4/letter/m/5f8ce5/32.png) [@manjur](https://discuss.elastic.co/u/manjur)\
**Post date:** [April 16, 2018, 9:12am UTC](https://discuss.elastic.co/t/implement-concurrency-while-inserting-data-to-es-using-logstash/128120/3 "2018-04-16T09:12:47Z")

</div>

okey. lets say schedule time is 1 min. If my first run took more than 1 min because of heavy data and mean while 2nd run starts as we have given schedule time as 1 min.so the question is how we can restrict this behavior.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 16, 2018, 11:42am UTC](https://discuss.elastic.co/t/implement-concurrency-while-inserting-data-to-es-using-logstash/128120/4 "2018-04-16T11:42:47Z")

</div>

Okay. Queries will never run concurrently, i.e. if the first query takes longer than one minute to process the second query won't run until the first one's done.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 14, 2018, 1:43pm UTC](https://discuss.elastic.co/t/implement-concurrency-while-inserting-data-to-es-using-logstash/128120/5 "2018-05-14T13:43:04Z")

</div>

Closed on May 14, 2018, 1:43pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.