Kibana logstash not showing data after a period of time


(Lee Weng Sheng) #1

Is there any troubleshoot steps?

[root@m logstash]# curl -XGET 'http://localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open filebeat-2018.09.10 mEt3P2H3Q9OH_KaBuIe3xA 5 1 1723073 0 472.6mb 472.6mb
yellow open logstash-2018.10.02 HogSey73ScOMhDHS_wVTxw 5 1 130818 0 160.3mb 160.3mb
yellow open logstash-2018.09.14 xw0Hw-UrT_OR-LJolyES7Q 5 1 255190843 0 164.3gb 164.3gb
yellow open logstash-2018.09.26 PPWYgVFBQQOUANBULvEvPA 5 1 286092486 0 171.4gb 171.4gb
yellow open logstash-2018.09.17 BN-FeHKcQMCveaFgb9seyA 5 1 255535915 0 164.6gb 164.6gb
yellow open .kibana so9lsWQ_Qfmh21Skt82bOg 1 1 101 53 261.5kb 261.5kb
yellow open logstash-2018.09.29 Jsyg1ABtT2GTEItVkxE4Aw 5 1 244161604 0 153.8gb 153.8gb
yellow open logstash-2018.09.20 YQ7RnnmzRviIf_pX10GNPw 5 1 255842588 0 165.4gb 165.4gb
yellow open logstash-2018.09.05 xXVYWowAQ2S4y1NsT4qx5A 5 1 364033944 0 230.9gb 230.9gb
yellow open logstash-2018.09.21 7zaB8J59Qli3tQidU4O1YA 5 1 265152125 0 172gb 172gb
yellow open logstash-2018.09.16 V1aYBuIqTZi04wDCSlCZsA 5 1 254956986 0 162.7gb 162.7gb
yellow open logstash-2018.10.01 Xfvr2tZ2QuyvoNiS7UPyDQ 5 1 204142103 0 133.1gb 133.1gb
yellow open logstash-2018.09.22 kqs99P_ORQCSacGIywRlUw 5 1 258402326 0 165.9gb 165.9gb
yellow open logstash-2018.09.08 bgzcekJKTU-fc34G2w7H9w 5 1 522375834 0 323.2gb 323.2gb
yellow open logstash-2018.09.13 _zjGxgA_Sc2irA9J141Ryw 5 1 255580280 0 164.7gb 164.7gb
yellow open logstash-2018.09.25 NlJwowU5QMeyjIgJiJ-v0w 5 1 224650592 0 148.8gb 148.8gb
yellow open logstash-2018.09.28 KVCe_nLYSdu-Zj9t8fyMpQ 5 1 327635493 0 186.3gb 186.3gb
yellow open logstash-2018.09.12 VoPeVKuzTy2LVF4woFf0mA 5 1 250364199 0 162.6gb 162.6gb
yellow open logstash-2018.09.06 Rv5xteO3Sd6SRXvgzACZ2Q 5 1 548353142 0 339.9gb 339.9gb
yellow open logstash-2018.09.18 HBg-wUMeQ5msJagonKrgsQ 5 1 255409139 0 164.9gb 164.9gb
yellow open logstash-2018.09.23 Q9s4jgmxSrW4UBUdjXIOVg 5 1 254646512 0 162.9gb 162.9gb
yellow open logstash-2018.09.19 BctiIKLoT72zvl2xxxmhXQ 5 1 256751085 0 165.9gb 165.9gb
yellow open logstash-2018.09.04 Tjdw039qT8-Ux_ir-i0r6A 5 1 326567832 0 214gb 214gb
yellow open logstash-2018.09.07 I7HE2jvmQJWuXln6WMlTGA 5 1 547513652 0 339.7gb 339.7gb
yellow open logstash-2018.09.24 dwGIirkVQYSesiPjpSWlag 5 1 231702587 0 154gb 154gb
yellow open logstash-2018.09.30 ecstfh23Ss63yYa4Xlcryg 5 1 228569338 0 150.4gb 150.4gb
yellow open logstash-2018.09.10 QtcDLnRWTZ6FWaaosFm5uA 5 1 528874194 0 331.1gb 331.1gb
yellow open logstash-2018.09.09 rz5C4l2VS1y_1k-xA9FACw 5 1 520892184 0 321.7gb 321.7gb
yellow open logstash-2018.09.11 QtP3acIPSHWtKphpIjQxtQ 5 1 380653042 0 240.5gb 240.5gb
yellow open logstash-2018.09.15 goVoxPXERDy6IhcYlaXEVg 5 1 258492279 0 165gb 165gb
yellow open logs_apache NdAayrWXRcyQsn55cJWpKA 5 1 2071 0 1mb 1mb
yellow open logstash-2018.09.03 dkx2re7kQxKkwwVyT3zD1w 5 1 327102668 0 151.3gb 151.3gb
yellow open logstash-2018.09.27 gyYJYDohSHuAtoK8Q-VokA 5 1 330744931 0 189.5gb 189.5gb

Here's the latest log:
[2018-10-02T16:45:37,365][ERROR][logstash.shutdownwatcher ] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
[2018-10-02T16:45:48,798][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2018-10-02T16:45:48,802][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[2018-10-02T16:45:49,392][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://127.0.0.1:9200/]}}
[2018-10-02T16:45:49,393][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://127.0.0.1:9200/, :path=>"/"}
[2018-10-02T16:45:49,498][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://127.0.0.1:9200/"}
[2018-10-02T16:45:49,499][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-10-02T16:45:49,574][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"default"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-10-02T16:45:49,580][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//127.0.0.1:9200"]}
[2018-10-02T16:45:49,620][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>40, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>5000}
[2018-10-02T16:45:49,674][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"127.0.0.1:5043"}
[2018-10-02T16:45:49,911][INFO ][logstash.pipeline ] Pipeline main started
[2018-10-02T16:45:49,920][INFO ][org.logstash.beats.Server] Starting server on port: 5043
[2018-10-02T16:45:49,936][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

Is there other stuff i can check?


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.