Grok and date filter

Elastic Stack Logstash
1

Hello

I have some question to ask about date filter how can I try to change type that get from grok to date

(input) :
notBefore=Nov 10 00:00:00 2015 GMT notAfter=Nov 28 23:59:59 2016 GMT issuer= /C=US/O=SSS/OU=STT/CN=Symantec Class 3 EV SSL CA - G3 subject= /1asdasdasdasdasdasd.Com

my logstash conf file is looked like this

input {
file {
path => "/opt/log/*.log"
start_position => beginning
sincedb_path => "/dev/null"
}
}
filter {
grok {
match => { "message" => "(?m)notBefore=%{DATA:Before} notAfter=%{DATA:After} issuer= /%{DATA:issuer}subject= /%{GREEDYDATA:subject}" }
}
date {
locale => en
match => [ "Before", "MMM dd HH:mm:ss yyyy z"]
}
}
output {
elasticsearch { hosts => ["192.168.100.141:9200"]
}
stdout {}
}

But I cant see any data come to elasticsearch and when I try delete data filter it works fine

2

Forget about Elasticsearch for now. Comment out your elasticsearch output and make this your only output: stdout { codec => rubydebug }. Try again. What do you get?

Do note that the date filter can't parse timezone names, i.e. it won't be able to parse "GMT".

3

It's normal result

"message" => "notBefore=Jun 18 00:00:00 2015 GMT notAfter=Sep 15 23:59:59 2016 GMT issuer= /C=US/O=Symantec Corporation/OU=Symantec Trust Network/CN=Symantec Class 3 EV SSL CA - G3 subject= /1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/businessCategory=Private Organization/serialNumber=2927442/C=US/postalCode=60603/ST=Illinois/L=Chicago/street=135 S La Salle St/O=Bank of America Corporation/OU=AIT 59915 - Network Infrastructure/CN=bankofamerica.com",
"@version" => "1",
"@timestamp" => "2015-06-18T00:00:00.000Z",
"path" => "/opt/log/bankofamerica.com.log",
"host" => "logstash01",
"Before" => "Jun 18 00:00:00 2015 GMT",
"After" => "Sep 15 23:59:59 2016 GMT",
"issuer" => "C=US/O=Symantec Corporation/OU=Symantec Trust Network/CN=Symantec Class 3 EV SSL CA - G3 ",
"subject" => "1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/businessCategory=Private Organization/serialNumber=2927442/C=US/postalCode=60603/ST=Illinois/L=Chicago/street=135 S La Salle St/O=Bank of America Corporation/OU=AIT 59915 - Network Infrastructure/CN=bankofamerica.com"

4

I just notice that date filter is not working as I always think so I need to apply target in date filter
now I try use new logstash conf

input {
file {
path => "/opt/log/*.log"
start_position => beginning
sincedb_path => "/dev/null"
}
}
filter {
grok {
match => { "message" => "(?m)notBefore=%{DATA:Before} GMT notAfter=%{DATA:After} GMT issuer= /%{DATA:issuer}subject= /%{GREEDYDATA:subject}" }
}
date {
locale => en
match => [ "Before", "MMM dd HH:mm:ss yyyy"]
target => "Before"
}
}
output {
elasticsearch { hosts => ["192.168.100.141:9200"]
}
stdout { codec => rubydebug }
}

And it's work ! so I want to ask more about how can I apply to multiple target (like Before/After) and why when I go to kibana logstash field I still see Before as String not date

5

how can I apply to multiple target (like Before/After)

Use multiple date filters.

and why when I go to kibana logstash field I still see Before as String not date

Probably because the field already had been mapped as a string. Mappings of fields can't be changed without reindexing.

6

Thank you very much! I am very appreciated for your support!