Date Histogram Problem

[Jeremy_Cugmas]

Hello everyone,

I'm a beginner on ELK, Logstash and Kibana so sorry in advance if i make mistakes in my explanation !
So, to explain, I configure Logstash to read my kannel logs and send some datas from them to my ELK, to finally centralize and analyze them on Kibana.
To understand better, there is the pattern of my datas :

The problem is that I want to do a line chart with a date histogram with the sent_date field to see the count of my instances hourly, but when I did it, no point and no line is visible on the chart. However, the data table tells me there are some values as you can see below :

So I'm asking for help to you ! I don't understand why this doesn't work.

Thanks !

PS : Sorry for my english, I'm french

[LeeDr (Lee Drengenberg)]

Hi Jeremy,

I see from your field list that you have 2 date data type fields, @timestamp and sent_date. Which one of those did you select when you created your kannel-logs-test-01 index pattern?

If you go back to Management, Index Patterns, and click on your kannel-logs-test-01 index pattern name near the top left if it's not already selected.
Then you will see the field list for that index pattern. There's a little clock next to the default timestamp;

If it's on @timestamp you might want to delete that index pattern and re-create it and select sent_date instead.

I think what you're doing should work even when you selected @timestamp as your time field name, but there's an open bug; https://github.com/elastic/kibana/issues/5707

Regards,
Lee

[Jeremy_Cugmas]

To do my chart I want to use the sent_date field..
I don't see what you mean when you say which one I choose when I created the index pattern, but there is my logstash configuration to read my logs and store data to my ELK :

filter {
multiline {
pattern => "^%{TIMESTAMP_ISO8601} "
negate => true
what => "previous"
}
grok {
match => {
"message"=>"%{TIMESTAMP_ISO8601:timestamp} %{GREEDYDATA:kind} [SMSC:%{DATA:smsc}] [SVC:%{DATA:svc}] [ACT:%{DATA:account}] [BINF:%{DATA:binf}] [FID:%{DATA:fid}] [META:%{DATA:meta}] [from:%{GREEDYDATA:from}] [to:%{GREEDYDATA:to}] [flags:%{INT:flag1}:%{INT:flag2}:%{INT:flag3}:%{INT:flag4}:%{NUMBER:status:int}] [msg:%{NUMBER:msg_length:int}:%{GREEDYDATA:msg}] [udh:0:]"
}
remove_field => ["flag1","flag2","flag3","flag4","path","message"]
}
date {
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss"]
timezone => "Europe/Paris"
target => "sent_date"
remove_field => "timestamp"
}
}
output {
stdout{ codec => rubydebug}
elasticsearch {
"hosts" => "localhost:9200"
"index" => "kannel-logs-test-01"
}
}

And about the little clock next to the field, there is no one next to my timestamp and my sent_date field. There is no one as you can see below :

EDIT : I find out my error, when I created my index pattern, I have unchecked "Index contains time-based events
" so that's why I wasn't able to do some date histogram !

Thanks for your help !