Logstash cloudwatch plugin error

Elastic Stack Logstash
1

I'm trying to connect cloudwatch to logstash but get an error.

this is my logstash conf file:

input {
      cloudwatch {
        namespace => "AWS/Transfer"
        metrics => [ "BytesIn", "BytesOut" ]
        filters => { "tag:Monitoring" => "YES" }
        region => "us-east-1"
      }
    }

output {
        elasticsearch {
                hosts => ["10.50.73.138:9200", "10.50.73.139:9200", "10.50.73.140:9200"]
                index => "s3-logs-%{+YYYY.MM.dd}"
        }
}

Getting this error:

2020-11-04T16:59:10,864][INFO ][logstash.inputs.cloudwatch][main] Polling CloudWatch API
[2020-11-04T16:59:10,867][ERROR][logstash.javapipeline    ][main] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::CloudWatch namespace=>"AWS/Transfer", metrics=>["BytesIn", "BytesOut"], filters=>{"tag:Monitoring"=>"YES"}, id=>"595c3d5fea339cb6cae4076c012b260091ac12c24ce9ec69b13755277602c4b7", region=>"us-east-1", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_652be412-8523-4365-abba-5780a269558f", enable_metric=>true, charset=>"UTF-8">, role_session_name=>"logstash", statistics=>["SampleCount", "Average", "Minimum", "Maximum", "Sum"], interval=>900, period=>300, combined=>false>
  Error: No metrics to query
  Exception: RuntimeError
  Stack: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-cloudwatch-2.2.4/lib/logstash/inputs/cloudwatch.rb:154:in `run'
/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:328:in `inputworker'
/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:320:in `block in start_input'
2

The input has decided that the set of metrics it can collect is empty. Are you sure you want that filters option?

3

Filters must be defined for when using AWS/Transfer namespace

I tried to remove the filters part but looks like I would need some sort of filter.

I want filters like:

filter @message like "READ"
| fields @timestamp
| sort @timestamp desc
| stats count(*) as FileRead by bin(10min)

filter @message like "ERROR"
| fields @timestamp
| sort @timestamp desc
| stats count(*) as Errors by bin(24hr)

filter @message like "OPEN"
| parse @message ". OPEN Path=* Mode=*" as user, sessionID, path, mode
| fields @timestamp
| sort @timestamp desc
| stats count_distinct(sessionID) as uniquesessions by bin(10min)

filter @message like "OPEN"
| parse @message ". OPEN Path=* Mode=*" as user, sessionID, path, mode
| fields @timestamp
| filter mode like "READ"
| sort path desc
| stats count(path) as NumberOfRequests by path
| limit 10

How would I add this in the conf file?

4

this is in reference to

go down to "Widget Connected users"

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.