Kibana not showing the data due to time stamp field issue

Elastic Stack Kibana
1

Hi,

I tried to parse old logs and set timestamp what present on logs.

example log : 2015-09-11 20:06:03; - - "GET / HTTP/1.0" 302 311 "-" "-"

logstash-server date config:

..........
date {
    locale => "en"
    match => ["mytimestamp", "YYYY-MM-dd HH:mm:ss"]
    target => "@timestamp"
 }
..........

Here,I'm replacing @timestamp with log file timestamp.

Output :

           "message" => "2015-09-11 20:06:03; - - "GET / HTTP/1.0" 302 311 "-" "-" ",
         "@version" => "1",
       "@timestamp" => "2015-09-11T20:06:03.000Z",
             "host" => "172.17.42.1",
             "type" => "test",
      "mytimestamp" => "2015-09-11 20:06:03",
         ................

Only data shown in Kibana If it unchecked @timestamp field. Otherwise, it doesn't show any data.

2

Could you share your mappings for the resulting documents, as well as a screenshot of how you are trying to configure the index pattern?

3

The problem was due to server timing mismatch of both Logstash and elasticsearch server. Once I sync both timer, data is started to shown on Kibana dashboard.

Though, now I have hit with another problem.

When I set the index name in Kibana Settings, it doesn't show any data [ So I inspect the graph, its not showing the index name in curl url] . But at the same time, When I set it to "_all" it shows the date. Its totally weird. What was the issue here. ?.

Screenshot :
"When Index name set to test-YYYY.MM.DD in Dashboard settings"

mdesk2.png1338×255 7.81 KB 

 **"When Index name set to "_all" "**

mdesk5.png1052×372 34.6 KB

4

Any idea ?.

5

@tbragin could you please share your input on this ?.

6

My guess would be that if you specify a timestamped index pattern , Kibana will only add the indexes (that match the timerange) taht Does actually exist... and does not blindly add all index names that could exist in the timerange. (Which is why you might end up with a :8085//_search query string)

So, can you confirm that the test-YYYY.MM.DD indexes for your timerange does actually exist.
If you indexed the documents with one timestamp and then go and change the timestamp value afterwards - they will not be moved to a new index.

This is all just speculation though.

7

Yes, I have created Index with this pattern [ test-YYY.MM.DD]. Also I have checked that I could see the data when I query the elastic directly.

8

@tbragin If you have any chance, Kindly check this.

9

Any idea ?.