.monitoring index search via http_poller

[jasony (Jason)]

hello,

i tried to search monitoring data with below query.

GET <.monitoring-es-6-{now}>/_search
{
  "query": {
    "match": {
      "type": "node_stats"
    }
  }, 
  "sort": [
    {
      "timestamp": {
        "order": "asc"
      }
    }
  ], 
  "size": 3,
  "_source": {
    "includes": [
     "source_node.*", 
     "node_stats.indices.search.*",
     "node_stats.indices.indexing.*",
     "node_stats.indices.segments.count",
     "node_stats.thread_pool.*",
     "node_stats.jvm.mem.*"
    ]
  }
}

i tested in kibana which worked fine, but same query i tried to use in logstash http_poller, i got below error.

[2019-07-09T03:07:49,103][ERROR][logstash.pipeline        ] Pipeline aborted due to error {:pipeline_id=>"main", :eacktrace=>["/data/app/logstash-6.7.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-4.0.5/lib/logstash/ihttp_poller-4.0.5/lib/logstash/inputs/http_poller.rb:97:in `normalize_request'", "/data/app/logstash-6.7.1/vendor/b'", "org/jruby/RubyHash.java:1419:in `each'", "org/jruby/RubyEnumerable.java:833:in `map'", "/data/app/logstash-6.7s!'", "/data/app/logstash-6.7.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-http_poller-4.0.5/lib/logstash/inputsn'", "/data/app/logstash-6.7.1/logstash-core/lib/logstash/pipeline.rb:270:in `block in register_plugins'", "org/jruns'", "/data/app/logstash-6.7.1/logstash-core/lib/logstash/pipeline.rb:413:in `start_inputs'", "/data/app/logstash-/pipeline.rb:217:in `run'", "/data/app/logstash-6.7.1/logstash-core/lib/logstash/pipeline.rb:176:in `block in start
[2019-07-09T03:07:49,122][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>Logse", :backtrace=>nil}

can you please advise how i can query only to today .monitoring index?

thank you!

[jasony (Jason)]

logstash input part is as below.

input {
  http_poller {
    urls => {
      node_stats => {
        method => get
        user => "elastic"
        password => "elastic123"
        url => "http://els-01:9200/<.monitoring-es-6-{now}>/_search"
        body => '{
  "query": {
    "match": {
      "type": "node_stats"
    }
  },
  "sort": [
    {
      "timestamp": {
        "order": "desc"
      }
    }
  ],
  "size": 3,
  "_source": {
    "includes": [
     "source_node.*",
     "node_stats.node_master",
     "node_stats.indices.search.*",
     "node_stats.indices.indexing.*",
     "node_stats.indices.segments.count",
     "node_stats.thread_pool.*",
     "node_stats.jvm.mem.*"
    ]
  }
}'
        headers => {
          "Content-Type" => "application/json"
        }
      }
  }
  metadata_target => "http_poller_metadata"
  tags => es_monitoring
  request_timeout => 60
  schedule => { cron => "* * * * * UTC"}
  codec => "json"
 }