Add new field in logstash conf file


(Dinesh) #1

Hi,

I am new to logstash and this is my first task with it. I wanted to add two new fields called
"GMTDateTime" and "CSTDateTime" to use them in my kibana reporting. Same work i was doing in SQL query by
GMTDateTime = DATEADD(day, DATEDIFF(day,'19000101',Cast(date as date)), CAST(CAST(time as time) AS DATETIME2(7)));

Here Date is date part from date time field from iis log file and time is time part form date time field from iis log file.

Now I am using the above field to get daylight saving time through a SQL function and storing the values in another field CSTDateTime

CSTDateTime = dbo.fn_GetDaylightSavingsTime(GMTDateTime,-6,-5);

I wanted to perform the same ask in logstash conf file so that these two field will be available on kibana for reporting need.

Below is my conf file code

input {
file {
type => "iis-w3c"
path => "D:/logs/u_ex170102001126.log"
start_position => "beginning"
}
}
filter {

# ignore log comments
if [message] =~ "^#" {
	drop {}
}

mutate {
add_field => {
"GMTDateTime" => ""
"CSTDateTime" => ""
}
}

grok {
	match =>["message", "%{TIMESTAMP_ISO8601:datestamp} %{DATA:s_sitename} %{HOSTNAME:s_computername} %{IP:s_ip} %{WORD:cs_method} %{NOTSPACE:http_request_url} (- %{NUMBER:port} -) %{IPORHOST:real_clientip} (HTTP/%{NUMBER:http_version}) %{NOTSPACE:user_agent} %{NOTSPACE:cookie} %{NOTSPACE:cs_referer} %{URIHOST:cs_host} %{INT:sc_status} %{NUMBER:sc_substatus} %{NUMBER:sc_win32_status} %{NUMBER:sc_bytes} %{NUMBER:cs_bytes} %{NUMBER:time_taken}"]
}

}
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => ["127.0.0.1:9200"]
}
}

Any help will be appreciated.


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.