# How can ser grok if log have different content from the same equipment? **URL:** \ **Category:** Logstash\ **Created:** [June 10, 2021, 8:49am UTC](https://discuss.elastic.co/t/how-can-ser-grok-if-log-have-different-content-from-the-same-equipment/275541 "2021-06-10T08:49:08Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![111418](https://avatars.discourse-cdn.com/v4/letter/1/258eb7/32.png) [@111418](https://discuss.elastic.co/u/111418)\ **Post date:** [June 10, 2021, 8:49am UTC](https://discuss.elastic.co/t/how-can-ser-grok-if-log-have-different-content-from-the-same-equipment/275541/1 "2021-06-10T08:49:08Z")
```auto Jun 2 16:45:49 tp-id01.test.corp zorp/scb_ssh[1379]: core.alerting(3): (svc/wwVRLvYoUMToHT2Xz9vEbj/ssh_alert:11/ssh): Audit event was recognized; rule='PatternMatcherRule', type='adp.event.command', pattern='user', actions_taken='LOG, NOTIFY, METADB', event_data='ls -l' Jun 8 17:00:11 sc-id01.test.corp zorp/scb_ssh[1416]: ssh.auth(3): (svc/778EuyyZrBFgb3eLsyVsh3/ssh_orig_target:78/ssh): User authentication failure on server; username='Sc', gateway_user='' Jun 9 11:18:43 tp-id01.test.corp restserver[871]: User authentication failed; realm='Balabit', username='admin' ``` The log has different contents when users use different act. In fact ,the log types have more variable contents. How can set grok if log have different content from the same equipment? Do I must use multiple pattern to one on one parse these logs? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 10, 2021, 5:04pm UTC](https://discuss.elastic.co/t/how-can-ser-grok-if-log-have-different-content-from-the-same-equipment/275541/2 "2021-06-10T17:04:49Z")
I would start with something like ``` grok { match => { "message" => "^(?<[@metadata][timestamp]>.{15}) %{HOSTNAME:hostname} %{NOTSPACE:program}\[%{POSINT:pid}\]: (?[^;]+); %{GREEDYDATA:[@metadata][restOfLine]}" } } date { match => ["[@metadata][timestamp]", "MMM d HH:mm:ss", "MMM d HH:mm:ss" ] } kv { source => "[@metadata][restOfLine]" field_split_pattern => ", " } ``` which will get you ``` "pid" => "1416", "hostname" => "sc-id01.test.corp", "msg" => "ssh.auth(3): (svc/778EuyyZrBFgb3eLsyVsh3/ssh_orig_target:78/ssh): User authentication failure on server", "username" => "'Sc',", "program" => "zorp/scb_ssh", "@timestamp" => 2021-06-08T21:00:11.000Z ``` Note that the empty gateway\_user is not added as a field. Or ``` "pid" => "871", "realm" => "'Balabit',", "hostname" => "tp-id01.test.corp", "msg" => "User authentication failed", "username" => "admin", "program" => "restserver", ``` You may then want another grok to parse things out of the msg field. ---
**Author:** ![111418](https://avatars.discourse-cdn.com/v4/letter/1/258eb7/32.png) [@111418](https://discuss.elastic.co/u/111418)\ **Post date:** [June 11, 2021, 2:59am UTC](https://discuss.elastic.co/t/how-can-ser-grok-if-log-have-different-content-from-the-same-equipment/275541/3 "2021-06-11T02:59:17Z")
Hi, Thank you for your responses. I want to test @metadata but some grokdebug webs don't test the complete conf.d. Have any webs can test ? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 11, 2021, 3:12am UTC](https://discuss.elastic.co/t/how-can-ser-grok-if-log-have-different-content-from-the-same-equipment/275541/4 "2021-06-11T03:12:18Z")
Then use logstash to test the config, not a site maintained by a third party that does not work like grok. See [here](https://discuss.elastic.co/t/help-needed-in-grok/213827/2) and [here](https://discuss.elastic.co/t/grokparse-failure-and-after-successful-debug/254324/2). ---
**Author:** ![111418](https://avatars.discourse-cdn.com/v4/letter/1/258eb7/32.png) [@111418](https://discuss.elastic.co/u/111418)\ **Post date:** [June 13, 2021, 9:30am UTC](https://discuss.elastic.co/t/how-can-ser-grok-if-log-have-different-content-from-the-same-equipment/275541/5 "2021-06-13T09:30:06Z")
I use your pattern but have one problem. `event_data` should contain `ls -l` , but the output is `'ls` ```auto { "rule" => "PatternMatcherRule", "host" => "idc-securitu-elk01", "message" => "<284>Jun 2 16:45:49 tp-id01.test.corp zorp/scb_ssh[1379]: core.alerting(3): (svc/wwVRLvYoUMToHT2Xz9vEbj/ssh_alert:11/ssh): Audit event was recognized; rule='PatternMatcherRule', type='adp.event.command', pattern='user', actions_taken='LOG, NOTIFY, METADB', event_data='ls -l'", "id" => "284", "time" => "Jun 2 16:45:49", "event_data" => "'ls", "@timestamp" => 2021-06-13T08:44:14.381Z, "hostname" => "tp-id01.test.corp", "actions_taken" => "LOG, NOTIFY, METADB", "type" => "adp.event.command", "msg" => "core.alerting(3): (svc/wwVRLvYoUMToHT2Xz9vEbj/ssh_alert:11/ssh): Audit event was recognized", "@version" => "1", "pattern" => "user", "program" => "zorp/scb_ssh", "pid" => "1379" } ``` I have been use `mutate` and `gsub` like this: ```auto filter{ grok { match => { "message" => "<(?\d+)>(?