Parsing an existing field further

[Wilks]

Hi,
Is there way to parse the data from a single filed into 2 different fields? I have a field that contains a username and an ID that I would like to break out. So I want to assign to keep the username in the username field but then remove the numbers and assign it to a say UserID field. I have tried with grok to break it out further but it doesnt get picked up. I dont get an error, it just doent get picked up

so for example
username: Test,User
UserID: 1234567

Right now its all in the username field
"username" : "1234567,TEST,USER"

The log is as follows:

{
  "TestABC" : {
    "dataRec" : {
      "eventName" : "IN",
      "date" : "210721",
      "time" : "07:19:56",
      "userName" : "1234567,TEST,USER              "
    }
  }
}

My filter is as follows:

filter {
  json {
    source => "message"
    remove_field =>   "message"
    remove_field =>   "host"
           }
       
mutate {

    rename => {"[TestABC][dataRec][userName]"            => "sourceUserName"}
    rename => {"[TestABC][dataRec][date]"                => "date"}
    rename => {"[TestABC][dataRec][time]"                => "time"}
    
    add_field => {"start" => "%{date} %{time}"}
    add_field => {"end" => "%{date} %{time}"}
    remove_field => [ "date", "time" ]
     }
     
grok
       {
        match => { "sourceUsername" => [ "%{NUMBER:UserID" ] }
       }

[Badger]

If you use

output { stdout { codec => rubydebug } }

then what does the resulting event look like?

[Wilks]

Hi,
It looks like this

{
          "path" => "/home/log.txt",
       "TestABC" => {
        "dataRec" => {
                 "time" => "07:19:56",
            "eventName" => "IN",
                 "date" => "210721",
             "userName" => "1234567,TEST,USER              "
        }
    },
    "@timestamp" => 2021-08-13T19:49:17.553Z,
          "tags" => [
        [0] "multiline",
        [1] "_grokparsefailure"
    ],
      "@version" => "1"
}

[Wilks]

If I remove the grok code it removes the _grokparsefailure so that's obviously not the proper way forward to break out the userName field further.

    "@timestamp" => 2021-08-13T19:58:12.592Z,
          "path" => "/home/new20.txt",
          "tags" => [
        [0] "multiline"
    ],
       "TestABC" => {
        "dataRec" => {
            "eventName" => "IN",
                 "time" => "07:19:56",
             "userName" => "1234567,TEST,USER              ",
                 "date" => "210721"
        }
    },
      "@version" => "1"
}

[Badger]

The mutate+rename is not happening. Not sure why, but that is why you cannot parse [sourceUsername]

[Wilks]

I got it to work. I forgot to take in account the "nesting". It works with the following

filter {
  json {
    source => "message"
    remove_field =>   "message"
    remove_field =>   "host"
           }
grok
       {
        match => { "[TestABC][dataRec][userName]" => [ "%{BASE16NUM:employeeID}\,%{GREEDYDATA:User}" ] }
       }
}

RESULTS

{
           "TestABC" => {
        "dataRec" => {
            "eventName" => "IN",
             "userName" => "123456,TEST,USER              ",
                 "date" => "210721",
                 "time" => "07:19:56"
        }
    },
        "employeeID" => "123456",
        "@timestamp" => 2021-08-13T23:51:33.679Z,
              "tags" => [
        [0] "multiline"
    ],
              "path" => "/home/test.log",
          "@version" => "1",
    "sourceUserName" => "TEST,USER              "
}