GROK Filter not working as expected

Elastic Stack Beats
1

Hello,

While applying custom GROK filters to my sample data, i get unexpected results.

Sample Data:
hotspot offering lease x.x.x.x for xx:xx:xx:xx:xx:xx without success

Grok Pattern:
%{WORD:network.name} (?<client.message>%{WORD} %{WORD}) %{IP:client.ipaddress} %{WORD} %{MAC:client.mac} (\g'client.message')

Structured Data:

{
  "client": {
    "message": "without success",
    "ipaddress": "x.x.x.x",
    "mac": "xx:xx:xx:xx:xx:xx"
  },
  "network": {
    "name": "hotspot"
  }
}

What i would like to accomplish is the following >>>>

Structured Data:

{
  "client": {
    "message": "offering lease without success",
    "ipaddress": "x.x.x.x",
    "mac": "xx:xx:xx:xx:xx:xx"
  },
  "network": {
    "name": "hotspot"
  }
}

Any help would be welcome.

Thank you very much
Koen

2

I know you posted in the beats section but I used logstash to figure this problem out. Hopefully it will help you out either way.

I got this grok filter to work:

%{WORD:network.name} %{DATA:log_message1} %{IP:client.ipaddress} for %{MAC:client.mac} %{GREEDYDATA:log_message2}

I tested it with this config:

input {
  generator {
    message => 'hotspot offering lease 192.168.1.1 for AA:BB:CC:DD:EE:FF without success'
  count => 1

  }

}

filter {
	
	
	grok {
		match => {"message" => "%{WORD:[network][name]} %{DATA:log_message1} %{IP:[client][ipaddress]} for %{MAC:[client][mac]} %{GREEDYDATA:log_message2}"}
	}
	mutate {
		add_field => {
			"[client][message]" => "%{log_message1} %{log_message2}"
		}
		remove_field => ["log_message1","log_message2","host","sequence"]
	}
}

output {

stdout{}
}

And got this result:


{
    "network" => {
         "name" => "hotspot"
    },
      "@version" => "1",
       "message" => "hotspot offering lease 192.168.1.1 for AA:BB:CC:DD:EE:FF without success",
        "client" => {
              "mac" => "AA:BB:CC:DD:EE:FF",
              "message" => "offering lease without success",
              "ipaddress" => "192.168.1.1"
          },
    "@timestamp" => 2021-12-23T14:55:45.947Z
}
[INFO ] 2021-12-23 09:55:47.579 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9601}
[INFO ] 2021-12-23 09:55:48.193 [LogStash::Runner] runner - Logstash shut down.

Thank you for providing all of the information ahead of time :slight_smile:

3

Did you consider using a full ingest pipeline instead of just grok pattern?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.