Split path and take certain value by its position [2] and insert it into field

Elastic Stack Logstash
1

hi...
I need to extract from a path certain value by position...

example:
let's say I split this following path into array using '\' as split char
E:\OUM82\APP\Logs\UploadManager_20062019.log
I get:
[0]=E:
[1]=OUM82
[2]=APP
[3]=logs
[4]=UploadManager_20062019.log
so,
I'm always want to take whatever in [2]
how do I implement it?

"log" => {
        "offset" => 18109,
          "file" => {
            "path" => "E:\\OUM82\\TI_QA_82\\TI_DS_FILES\\ORACLE_LOGS\\ORACLE_DEMO-ORACLE-OCTIINFA_107_30_05_2019 - Copy.
log"
        }
    },
               "host" => {
                "name" => "OctQa",
                  "os" => {
              "family" => "windows",
              "kernel" => "10.0.14393.2969 (rs1_release.190503-1820)",
               "build" => "14393.2969",
            "platform" => "windows",
                "name" => "Windows Server 2016 Datacenter",
             "version" => "10.0"
        },
            "hostname" => "OctQa",
        "architecture" => "x86_64",
                  "id" => "67d6abee-cd05-4497-a3b8-eaacbf4403dc"
    },
              "input" => {
        "type" => "log"
    },
           "loglevel" => "INFO",
      "sourceMessage" => "[1] ExtractorLogger Octopai.Desktop.Extractors.OracleExtractor SaveMetaDataQueryResult - Done
executing delete on table: TI.SHD_DB_UI_OBJECT_DETAILS for conectionID: 107 . result: Successful"
}

i need to do something like this with ruby?

 filter {
 ruby {
                code => 
                        filename = event['path'].split('/').last
                  
        }

}

please help.

3

I tried this following grok...
but I still getting _grokparsefailure

  grok {
                  match => { path => "%{GREEDYDATA:pathDriveSign}\\%{GREEDYDATA:RootFolder}\\%{GREEDYDATA:customerFolder}" }
               }

any idea?

here new "stdout"

{
          "tags" => [
        [0] "beats_input_codec_plain_applied",
        [1] "_grokparsefailure"
    ],
         "agent" => {
                "type" => "filebeat",
        "ephemeral_id" => "bd6ace26-79cd-4297-bfb5-5add9f4b4217",
                  "id" => "83fb6261-5872-4d95-853a-44f2cc41d436",
             "version" => "7.0.0",
            "hostname" => "OctUpload"
    },
       "message" => "2019-06-13 17:40:34,591 INFO QueriesParserEngine.Run - GSP queries parser engine end. Total run time duration: 00:02:32.1831164 ",
    "@timestamp" => 2019-06-22T16:25:26.204Z,
         "cloud" => {
        "provider" => "az",
         "machine" => {
            "type" => "Standard_DS13_v2"
        },
          "region" => "westeurope",
        "instance" => {
            "name" => "OctUpload",
              "id" => "768097b1-bfb9-4939-b99c-5337aede39ca"
        }
    },
     "extractor" => "SQLSERVER",
         "input" => {
        "type" => "log"
    },
           "ecs" => {
        "version" => "1.0.0"
    },
      "@version" => "1",
        "fields" => {
        "logtype" => "log4net"
    },
          "host" => {
                  "os" => {
               "build" => "14393.2608",
             "version" => "10.0",
                "name" => "Windows Server 2016 Datacenter",
            "platform" => "windows",
              "kernel" => "10.0.14393.2608 (rs1_release.181024-1742)",
              "family" => "windows"
        },
                  "id" => "d79c20df-4184-41a8-b95d-83669c8e3fbe",
                "name" => "OctUpload",
        "architecture" => "x86_64",
            "hostname" => "OctUpload"
    },
           "log" => {
          "file" => {
            "path" => "E:\\OUM82\\Micron\\TI_DS_FILES\\SQLSERVER_LOGS\\QueriesParser-SQLS-BOMSSPROD66-2_13062019_173801 - Copy.log"
        },
        "offset" => 927068
    }
}
4

You cannot do it with mutate+split (which is what I would normally suggest) due to this issue, which affects regexps, single quoted string, and double quoted strings.

It is possible using grok if you enable config.support_escapes on logstash.yml... Believe it or not

grok { match => { "path" => "^(?<pathDriveSign>\w{1}):\\\\(?<RootFolder>[^\\\\]+)\\\\(?<customerFolder>[^\\\\]+)\\\\." } }

will get you

    "RootFolder" => "OUM82",
 "pathDriveSign" => "E",
"customerFolder" => "APP",

Do not ask me to explain why 4 backslashes are required to represent a single backslash.

There is also a sneaky way to do it in ruby. You cannot have a backslash at the end of a string, so we have a string that contains a backslash and extract the backslash from it.

    ruby {
        code => '
            backslash = "\\Z"[0]
            event.set("components", event.get("path").split(backslash))
        '
    }

results in

"components" => [
    [0] "E:",
    [1] "OUM82",
    [2] "APP",
    [3] "Logs",
    [4] "UploadManager_20062019.log"
]
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.