Logstash shutting down

Hello,

When I use a config file to create metrics on Logstash, after all calculations, it shuts down. But I need it keep running. This is my script, can someone say what I'm doing wrong? I 'm new to Logstash and first time using for a job.

input{
   elasticsearch{
    hosts => "localhost"
    index => "teste"
   }
}

filter {

      metrics{
        meter => [ "%{status_contrato}", "events"]
        add_tag => "metric"
        flush_interval => 15
      }

      mutate{
        remove_field =>["message","status_contrato", "[PENDENTE][rate_5m]", "[PENDENTE][rate_1m]", "[PENDENTE][rate_15m]", "[APROVADO][rate_1m]", "[APROVADO][rate_5m]", "[APROVADO][rate_15m]", "[events][rate_15m]", "[events][rate_1m]", "[events][rate_5m]"]
      }

      mutate{
        convert =>{"[APROVADO][count]" => "float"}
        convert =>{"[events][count]" => "float"}
      }

      if "metric" in [tags]{
        ruby{
          code => " aprovados = event.get(%{[APROVADO][count]})/event.get(%{[events][count]})*100;
                    event.set('pct_aprovado', aprovados);

                    pendentes = event.get(%{[PENDENTE][count]})/event.get(%{[events][count]})*100;
                    event.set('pct_pendente', pendentes);
                    "
          add_tag => "metric"
        }
      }


      if "metric" not in [tags]{
        drop{}
      }

      mutate{
        add_field => {"total_aprovados" => "%{[APROVADO][count]}"}
        add_field => {"total_pendentes" => "%{[PENDENTE][count]}"}
        add_field => {"total_contratos" => "%{[events][count]}"}
      }

      mutate{
        convert => {"total_aprovados" => "integer"}
        convert => {"total_pendentes" => "integer"}
        convert => {"total_contratos" => "integer"}
        remove_field => ["[APROVADO]", "[PENDENTE]", "events", "tags", "@version"]
      }

}

output{

  stdout{
    codec => rubydebug
  }


      elasticsearch{
        hosts => "localhost:9200"
        index => "calculos"
      }


}

I'm using on a Windows and the debug log:

{
       "total_aprovados" => 132,
       "total_contratos" => 399,
          "pct_aprovado" => 33.08270676691729,
            "@timestamp" => 2019-06-07T15:59:55.920Z,
          "pct_pendente" => 66.41604010025063,
    "%{status_contrato}" => {
         "rate_5m" => 0.0,
         "rate_1m" => 0.0,
        "rate_15m" => 0.0,
           "count" => 1
    },
       "total_pendentes" => 265
}
{
       "total_aprovados" => 132,
       "total_contratos" => 399,
          "pct_aprovado" => 33.08270676691729,
            "@timestamp" => 2019-06-07T15:59:55.920Z,
          "pct_pendente" => 66.41604010025063,
    "%{status_contrato}" => {
         "rate_5m" => 0.0,
         "rate_1m" => 0.0,
        "rate_15m" => 0.0,
           "count" => 1
    },
       "total_pendentes" => 265
}
{
       "total_aprovados" => 132,
       "total_contratos" => 399,
          "pct_aprovado" => 33.08270676691729,
            "@timestamp" => 2019-06-07T15:59:55.920Z,
          "pct_pendente" => 66.41604010025063,
    "%{status_contrato}" => {
         "rate_5m" => 0.0,
         "rate_1m" => 0.0,
        "rate_15m" => 0.0,
           "count" => 1
    },
       "total_pendentes" => 265
}
[2019-06-07T12:59:56,826][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[2019-06-07T12:59:57,004][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2019-06-07T12:59:57,006][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2019-06-07T12:59:57,371][DEBUG][logstash.javapipeline    ] Shutdown waiting for worker thread {:pipeline_id=>"main", :thread=>"#<Thread:0x731155 dead>"}
[2019-06-07T12:59:57,372][DEBUG][logstash.javapipeline    ] Shutdown waiting for worker thread {:pipeline_id=>"main", :thread=>"#<Thread:0x158a4b4 dead>"}
[2019-06-07T12:59:57,373][DEBUG][logstash.javapipeline    ] Shutdown waiting for worker thread {:pipeline_id=>"main", :thread=>"#<Thread:0x1b90032 dead>"}
[2019-06-07T12:59:57,375][DEBUG][logstash.filters.mutate  ] Closing {:plugin=>"LogStash::Filters::Mutate"}
[2019-06-07T12:59:57,378][DEBUG][logstash.filters.drop    ] Closing {:plugin=>"LogStash::Filters::Drop"}
[2019-06-07T12:59:57,379][DEBUG][logstash.filters.ruby    ] Closing {:plugin=>"LogStash::Filters::Ruby"}
[2019-06-07T12:59:57,379][DEBUG][logstash.filters.mutate  ] Closing {:plugin=>"LogStash::Filters::Mutate"}
[2019-06-07T12:59:57,380][DEBUG][logstash.filters.mutate  ] Closing {:plugin=>"LogStash::Filters::Mutate"}
[2019-06-07T12:59:57,381][DEBUG][logstash.filters.metrics ] Closing {:plugin=>"LogStash::Filters::Metrics"}
[2019-06-07T12:59:57,382][DEBUG][logstash.filters.mutate  ] Closing {:plugin=>"LogStash::Filters::Mutate"}
[2019-06-07T12:59:57,385][DEBUG][logstash.outputs.stdout  ] Closing {:plugin=>"LogStash::Outputs::Stdout"}
[2019-06-07T12:59:57,386][DEBUG][logstash.outputs.elasticsearch] Closing {:plugin=>"LogStash::Outputs::ElasticSearch"}
[2019-06-07T12:59:57,403][DEBUG][logstash.outputs.elasticsearch] Stopping sniffer
[2019-06-07T12:59:57,407][DEBUG][logstash.outputs.elasticsearch] Stopping resurrectionist
[2019-06-07T12:59:57,537][DEBUG][logstash.outputs.elasticsearch] Waiting for in use manticore connections
[2019-06-07T12:59:57,550][DEBUG][logstash.outputs.elasticsearch] Closing adapter #<LogStash::Outputs::ElasticSearch::HttpClient::ManticoreAdapter:0xcf2787>
[2019-06-07T12:59:57,560][DEBUG][logstash.javapipeline    ] Pipeline has been shutdown {:pipeline_id=>"main", :thread=>"#<Thread:0x167805a run>"}
[2019-06-07T12:59:57,698][DEBUG][logstash.instrument.periodicpoller.os] Stopping
[2019-06-07T12:59:57,721][DEBUG][logstash.instrument.periodicpoller.jvm] Stopping
[2019-06-07T12:59:57,723][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Stopping
[2019-06-07T12:59:57,724][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] Stopping
[2019-06-07T12:59:57,800][DEBUG][logstash.agent           ] Shutting down all pipelines {:pipelines_count=>0}
[2019-06-07T12:59:57,804][DEBUG][logstash.agent           ] Converging pipelines state {:actions_count=>0}
[2019-06-07T12:59:57,808][INFO ][logstash.runner          ] Logstash shut down.

Thanks in advance.

Set a schedule for your input plugin otherwise it will just run once.

Check out the scheduling section here:
https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html

1 Like

Thanks, it solved my problem.