I am trying to extract contents of the filename (made available by the sourcefield) and would like to add tags/create new fields based on the criterie.
For example if this was the data in the source field:
/app/appname/log/database-server-n5-server1234.domain.local.out
I would like to extract the following bits from the filename and create fields/tags for them: instancenumber: "n5" hostname: "server1234.domain.local.out"
I have tried the following but I keep getting config errors: else if [type] == "AppCacheProxy" { grok { match => { "message" => "\A%{TIMESTAMP_ISO8601}%{NOTSPACE}%{SPACE}%{GREEDYDATA}" } match => { "@source_path", "app/appname/log/database-server-%{NOTSPACE:instancenumber}-%{NOTSPACE:hostname}.log" } break_on_match => false }
I also tried the following line in place of the one above: match => { "source_path" => "app/appname/log/database-server-%{NOTSPACE:instancenumber}-%{NOTSPACE:hostname}.log" }
You should be able to do it using grok. Your pattern does however not seem to match the data. The pattern you provided ends in .log, which is not present in the sample data you provided, meaning it will not match.
I would recommend not using multiple or leading GREEDYDATA in your grok expressions as this can be quite inefficient. Find other patterns that more accurately match your data.
I am trying to extract the contents of a filename from the source field using the expression below. I can't seem to get it to appear on elasticsearch / kibana though. Does this look OK to you?
else if [source] =~ "GDO" {
grok {
match => { "path" => "(?<app>[^_\.]+)_(?<class>[^_\.]+)_(?<member>[^_\.-]+)(?:-[^_]+)?(?:_backen(?<db>d))?\.log" }
}
}
PS: Source is in the format: /app/apparatus/log/logs/APPARATUS_APPARATUSCacheServer_n5-grdserver.domain.local.log
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.