nitzanm
(nitzan)
June 10, 2019, 7:31am
1
hi
I'm trying to fill a field based on path substring that is coming for filebeat
here's an example for what i tried:
input {
beats {
port => 5044
}
}
filter {
if [fields][logtype] == "log4net" {
grok {
match => { message => "(?m)%{TIMESTAMP_ISO8601:sourceTimestamp}\,%{NUMBER:threadid} %{LOGLEVEL:loglevel} %{GREEDYDATA:sourceMessage}" }
}
if [path] =~ "SQLSERVER" {
add_field => { "EXTRACTOR" => "SQLSERVER" }
}
if [path] =~ "ORACLE" {
add_field => { "EXTRACTOR" => "ORACLE" }
}
}
}
output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
in this case if the log file path is :
E:\OUM82\TI_QA_82\TI_DS_FILES*ORACLE *_LOGS\ORACLE_DEMO-ORACLE-OCTIINFA_107_30_05_2019.log
I want the "extractor" field to be ORACLE
because the path is containing the word ORACLE
TNX.
Badger
June 10, 2019, 12:57pm
2
If your events have a [path] field then I would expect that to work. If it does not work then that suggests your events do not have a [path] field. What do you get for an event from
output { stdout { codec => rubydebug } }
nitzanm
(nitzan)
June 10, 2019, 7:22pm
3
"log" => {
"offset" => 18109,
"file" => {
"path" => "E:\\OUM82\\TI_QA_82\\TI_DS_FILES\\ORACLE_LOGS\\ORACLE_DEMO-ORACLE-OCTIINFA_107_30_05_2019 - Copy.
log"
}
},
"host" => {
"name" => "OctQa",
"os" => {
"family" => "windows",
"kernel" => "10.0.14393.2969 (rs1_release.190503-1820)",
"build" => "14393.2969",
"platform" => "windows",
"name" => "Windows Server 2016 Datacenter",
"version" => "10.0"
},
"hostname" => "OctQa",
"architecture" => "x86_64",
"id" => "67d6abee-cd05-4497-a3b8-eaacbf4403dc"
},
"input" => {
"type" => "log"
},
"loglevel" => "INFO",
"sourceMessage" => "[1] ExtractorLogger Octopai.Desktop.Extractors.OracleExtractor SaveMetaDataQueryResult - Done
executing delete on table: TI.SHD_DB_UI_OBJECT_DETAILS for conectionID: 107 . result: Successful"
}
I can see the path field..
bat i probably missing something, because I can't make this work..
tnx.
Badger
June 10, 2019, 9:32pm
4
You can refer to that as [log][file][path], not [path].
nitzanm
(nitzan)
June 11, 2019, 4:30pm
5
tnx..
it's working now..
you are the man Badger!!
pb:
in my example mutate inside the if is missing
if [fields][logtype] == "log4net" {
grok {
match => { message => "(?m)%{TIMESTAMP_ISO8601:sourceTimestamp}\,%{NUMBER:threadid} %{LOGLEVEL:loglevel} %{GREEDYDATA:sourceMessage}" }
}
if [log][file][path] =~ "SQLSERVER" {
mutate { add_field => { "extractor" => "SQLSERVER" }}
} else if [log][file][path] =~ "ORACLE" {
mutate { add_field => { "extractor" => "ORACLE" }}
} else if [log][file][path] =~ "INFAORA" {
mutate { add_field => { "extractor" => "INFAORA" }}
}else if [log][file][path] =~ "ODI" {
mutate { add_field => { "extractor" => "ODI" }}
}else if [log][file][path] =~ "POWERBI" {
mutate { add_field => { "extractor" => "POWERBI" }}
}else if [log][file][path] =~ "SSAS" {
mutate { add_field => { "extractor" => "SSAS" }}
}else if [log][file][path] =~ "SSIS" {
mutate { add_field => { "extractor" => "SSIS" }}
}else if [log][file][path] =~ "SSISSqlServer" {
mutate { add_field => { "extractor" => "SSISSqlServer" }}
}else if [log][file][path] =~ "SSRS" {
mutate { add_field => { "extractor" => "SSRS" }}
}else if [log][file][path] =~ "TABULAR" {
mutate { add_field => { "extractor" => "TABULAR" }}
}else if [log][file][path] =~ "TERADATA" {
mutate { add_field => { "extractor" => "TERADATA" }}
}
}
tnx again
system
(system)
Closed
July 9, 2019, 4:30pm
6
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.