Filter Metric Flush_Interval

Elastic Stack Logstash
1

Hello, I'm doing some metrics in a logstash script to generate a new index with some calculations.

Everything works fine for now, but when I change the flush_interval to some value high, it doesn't work, there is no output. But when I use a flush_interval with low value, the output creates 2 or 3 events on index depending on value I use.

I want to know if there is a way to flush_interval be automatized accordingly to the size of input.

This is my script for now:

input{
   elasticsearch{
    hosts => "localhost"
    index => "teste"
   }
}

filter {

      metrics{
        meter => [ "%{status_contrato}", "events"]
        add_tag => "metric"
        flush_interval => 25
      }

      mutate{
        remove_field =>["message","status_contrato", "[PENDENTE][rate_5m]", "[PENDENTE][rate_1m]", "[PENDENTE][rate_15m]", "[APROVADO][rate_1m]", "[APROVADO][rate_5m]", "[APROVADO][rate_15m]", "[events][rate_15m]", "[events][rate_1m]", "[events][rate_5m]"]
      }

      mutate{
        convert =>{"[APROVADO][count]" => "float"}
        convert =>{"[events][count]" => "float"}
      }

      if "metric" in [tags]{
        ruby{
          code => " aprovados = event.get(%{[APROVADO][count]})/event.get(%{[events][count]})*100;
                    event.set('pct_aprovado', aprovados);

                    pendentes = event.get(%{[PENDENTE][count]})/event.get(%{[events][count]})*100;
                    event.set('pct_pendente', pendentes);
                    "
          add_tag => "metric"
        }
      }


      if "metric" not in [tags]{
        drop{}
      }

      mutate{
        add_field => {"total_aprovados" => "%{[APROVADO][count]}"}
        add_field => {"total_pendentes" => "%{[PENDENTE][count]}"}
        add_field => {"total_contratos" => "%{[events][count]}"}
      }

      mutate{
        convert => {"total_aprovados" => "integer"}
        convert => {"total_pendentes" => "integer"}
        convert => {"total_contratos" => "integer"}
        remove_field => ["[APROVADO]", "[PENDENTE]", "events", "tags", "@version"]
      }

}

output{

  stdout{
    codec => rubydebug
  }


      elasticsearch{
        hosts => "localhost:9200"
        index => "calculos"
      }

}

Thanks in advance

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.