Elastice Search | Index stop working

Elastic Stack Elasticsearch
1

Hi All,

My ELK was running fine from last two days. Suddenly last night Kibana stop showing any data. Upon investigating look like index show as half (only default files i:e _type). Then I debug logstash and check if the problem was with field grok. But it looks fine no issue. Have look below

Ouputt of Logstash

  "httprequest" => "/stats/new.php",
  "httpmethod" => "POST",
  "type" => "example_main",
  "agentosname" => "Windows 7",
  "timestampnew" => "Mar 10 06:11:15",
  "path" => "/var/log/apache.log",
  "httpreferrer" => "https://www.example.com/apart/
  "httpsize" => "74",
  "@version" => "1",
  "httpuseragent" => "\"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36\"",
  "timestamp" => "10/Mar/2017:06:11:15 +0000",
  "geoip" => {
  "timezone" => "America/Chicago",
  "ip" => "134.132.52.220",
  "latitude" => 29.7702,
  "continent_code" => "NA",
  "city_name" => "Houston",
  "country_code2" => "US",
  "country_name" => "United States",
  "dma_code" => 618,
  "country_code3" => "US",
  "region_name" => "Texas",
  "location" => [
  [0] -95.3628,
  [1] 29.7702
  ],
  "postal_code" => "77002",
  "longitude" => -95.3628,
  "region_code" => "TX"
  },
  "agentdevice" => "Other",
  "tags" => [
  [0] "dated"
  ],
  "ipactual" => "134.132.52.231",
  "httpresponse" => "200",
  "@timestamp" => 2017-03-10T02:11:15.000Z,
  "Akami" => [
  [0] "11.56.11.34",
  [1] "99.00.33.11"
  ],
  "agentname" => "Chrome",
  "ipserver" => "ip-10-10-10-101"
  }

What Kanban show via Elastisearch

  {
  "_index": "example.com-2017.03.10",
  "_type": "example_main",
  "_id": "AVq21xKAIBRE0dl00K10",
  "_score": null,
  "_source": {
  "path": "/var/log/apache.log",
  "@timestamp": "2017-03-10T06:10:32.675Z",
  "geoip": {},
  "@version": "1",
  "type": "example_main",
  "tags": [
  "_grokparsefailure",
  "_geoip_lookup_failure"
  ]
  },
  "fields": {
  "@timestamp": [
  "2017-03-10T06:10:32.675Z",
  1489126232675
  ]
  },
  "sort": [
  1489126232675
  ]
  }

Can any one help me here

2

Please any one can help??

3

Hey,

so the tags field in logstash contains the errors that occured, in this example the line could not be parsed by grok and there was a geo lookup issue.

--Alex

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.