# Logistash pipline ended without any error

**URL:** <https://discuss.elastic.co/t/logistash-pipline-ended-without-any-error/102940>\
**Category:** Logstash\
**Created:** [October 6, 2017, 4:32am UTC](https://discuss.elastic.co/t/logistash-pipline-ended-without-any-error/102940 "2017-10-06T04:32:18Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sidra\_89](https://avatars.discourse-cdn.com/v4/letter/s/3d9bf3/32.png) [@sidra\_89](https://discuss.elastic.co/u/sidra_89)\
**Post date:** [October 6, 2017, 4:32am UTC](https://discuss.elastic.co/t/logistash-pipline-ended-without-any-error/102940/1 "2017-10-06T04:32:19Z")

</div>

i am using logistash to transfer data from MYSQL [wamp] to Elasticsearch.

the query i am using is extremely complex will lots of joins and multiple values in select statement

something like this

```
select CONVERT_TZ(so.purchased_at, 'UTC', m.timezone) AS purchased_at, CONVERT_TZ(so.last_updated_at, 'UTC', m.timezone) AS last_updated_at, ...... IF(p.price is not null, p.price * soi.quantity_ordered, 0))) >= IF(p.discount_threshold, p.discount_threshold, IF(u.default_discount_threshold, u.default_discount_threshold, 0.5)) , 1, 0) AS promo from `sale_order_items` as `soi` inner join `sale_orders` as `so` on `soi`.`sale_order_id` = `so`.`id` inner join `products` as `p` on `soi`.`product_id` = `p`.`id` inner join `users` as `u` on `so`.`user_id` = `u`.`id` inner join `marketplaces` as `m` on `u`.`marketplace_id` = `m`.`id` where `u`.`id` in (2) and ((`u`.`id` = 2)) order by CONVERT_TZ(so.purchased_at, 'UTC', m.timezone) desc

```

when i ran it using bin\logistash -f db.conf

logistash pipeline is ended without any error and when i view the newly created index in kibana i can see many documents deleted for that particular index .. i dont know why documents are deleted.

yellow open orders fbAcP-KRTy6lpKZUA 5 1 123 6790 156.7mb 156.7mb  
deleted documents : 6790

this is my db.conf code

input {  
jdbc {  
jdbc\_connection\_string =\> "jdbc:mysql://localhost:3306/db"  
jdbc\_user =\> "user"  
jdbc\_password =\> "password"  
jdbc\_driver\_library =\> "D:\ELK\logstash-5.6.2\mysql-connector-java-5.1.44-bin.jar"  
jdbc\_driver\_class =\> "com.mysql.jdbc.Driver"  
jdbc\_fetch\_size =\> 100  
statement =\> "select \* from sale\_order\_items"  
}  
}  
output {  
elasticsearch {  
index =\> "orders"  
document\_type =\> "order"  
document\_id =\> "%{id}"  
hosts =\> "localhost:9200"  
}  
}

Thank You ..

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 9, 2017, 6:50pm UTC](https://discuss.elastic.co/t/logistash-pipline-ended-without-any-error/102940/2 "2017-10-09T18:50:00Z")

</div>

> logistash pipeline is ended without any error

I think that's normal for the jdbc input unless you've specified a schedule.

> and when i view the newly created index in kibana i can see many documents deleted for that particular index .. i dont know why documents are deleted.

ES implements document updates as a deletion followed by an insertion, i.e. every time you update an existing document the deletion counter is increased.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 6, 2017, 6:50pm UTC](https://discuss.elastic.co/t/logistash-pipline-ended-without-any-error/102940/3 "2017-11-06T18:50:43Z")

</div>

Closed on November 6, 2017, 6:50pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.