# Logstash stopped when received x line and not push to elasticsearch

**URL:** <https://discuss.elastic.co/t/logstash-stopped-when-received-x-line-and-not-push-to-elasticsearch/245387>\
**Category:** Logstash\
**Created:** [August 18, 2020, 10:16am UTC](https://discuss.elastic.co/t/logstash-stopped-when-received-x-line-and-not-push-to-elasticsearch/245387 "2020-08-18T10:16:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![yannice92](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yannice92/32/73951_2.png) [@yannice92](https://discuss.elastic.co/u/yannice92)\
**Post date:** [August 18, 2020, 10:16am UTC](https://discuss.elastic.co/t/logstash-stopped-when-received-x-line-and-not-push-to-elasticsearch/245387/1 "2020-08-18T10:16:13Z")

</div>

I have 2 server. server log and server logstash  
I using only logstash 6.8 and elasticsearch (without filebeat)  
from server log I send the log to server logstash using cron.  
and in server logstash I have multiple pipeline.  
for log I'm using this config

```auto
    input {
      file {
        path => ["/home/xxxxxx/bcp/bcp_usage_home_lte_*.txt"]
        start_position => "beginning"
        sincedb_path => "/etc/logstash/dbpath/usagestatistic.db"
      }
    }

    filter {
      grok {
        match => {"message" => "(?<DATE>[0-9]+)\|(?<MSISDN>(.)+)\|(?<APP>(.)+)\|(?<CATEGORY>(.)+)\|(?<DEVICE>(.)+)\|(?<VENDOR>(.)+)\|(?<APN>(.)+)\|(?<TETHER>[0-9]+)\|(?<QUOTA>[0-9]+)"}
        remove_field => ["message"]
      }
      mutate {
        convert => {
          "QUOTA" => "integer"
          "TETHER" => "integer"
        }
      }
    }

    output {
      elasticsearch {
        hosts => ["1.2.3.4:3306"]
        user => "user"
        password => "xxxxxxxx"
        ssl => true
        ssl_certificate_verification => false
        index => "usagestatistic"
      }
      stdout { codec => rubydebug }
    }

```

and for pipelines.yml

```auto
    # This file is where you define your pipelines. You can define multiple.
    # For more information on multiple pipelines, see the documentation:
    # https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html

    - pipeline.id: usagestatistic
      path.config: "/etc/logstash/conf.d/usagestatistic.conf"
      queue.type: persisted
      pipeline.workers: 2
    - pipeline.id: customer_data-activity
      path.config: "/home/xxxxxx/customer-data/activity.conf"
      pipeline.workers: 1
      #queue.type: persisted
    - pipeline.id: customer_data-delivery
      path.config: "/home/xxxxxx/customer-data/delivery.conf"
      pipeline.workers: 1
      queue.type: persisted
    - pipeline.id: customer_data-profile
      path.config: "/home/xxxxxx/customer-data/profile.conf"
      pipeline.workers: 1
      queue.type: persisted
    - pipeline.id: customer_data-payment
      path.config: "/home/xxxxxx/customer-data/payment.conf"
      pipeline.workers: 1
      queue.type: persisted
    - pipeline.id: customer_data-order_fullfillment
      path.config: "/home/xxxxxx/customer-data/order_fulfillment.conf"
      pipeline.workers: 1
      queue.type: persisted
    - pipeline.id: customer_data-order_fulfillment_history
      path.config: "/home/xxxxxx/customer-data/order_fulfillment_history.conf"
      pipeline.workers: 1
      queue.type: persisted

```

when I check the log I cannot see any error so I think it's not problem with the config files.

```auto
[2020-08-17T14:57:53,751][DEBUG][filewatch.tailmode.handlers.grow] read_to_eof: get chunk
[2020-08-17T14:57:53,752][DEBUG][logstash.inputs.file] Received line {:path=>"/home/xxxxxx/bcp/bcp_usage_home_lte_20200815.txt", :text=>"20200815|628123456789|BranchMetrics|DeviceServices|unclassified|unclassified|home|1|142705"

```

these logstash-plain.txt print end of line log bcp\_usage\_home\_lte\_20200815 but not push to elasticsearch.

when I kill the logstash and start again it running well with output to ES. but tomorrow I check again the next log it received to logstash but stop at x lines. and I tried to restart the logstash and it's happened again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 15, 2020, 10:16am UTC](https://discuss.elastic.co/t/logstash-stopped-when-received-x-line-and-not-push-to-elasticsearch/245387/2 "2020-09-15T10:16:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.