# Why can't this match successfully?

**URL:** https://discuss.elastic.co/t/why-cant-this-match-successfully/369146
**Category:** Logstash
**Created:** [October 21, 2024, 11:24am UTC](https://discuss.elastic.co/t/why-cant-this-match-successfully/369146 "2024-10-21T11:24:28Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![WeirdorPersist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weirdorpersist/32/137046_2.png) [@WeirdorPersist](https://discuss.elastic.co/u/WeirdorPersist)
#### Post date: [October 21, 2024, 11:24am UTC](https://discuss.elastic.co/t/why-cant-this-match-successfully/369146/1 "2024-10-21T11:24:28Z")

</div>

![1](https://us1.discourse-cdn.com/elastic/original/3X/a/7/a78872b37f98789354a00350902135d78b50c7b1.png)  
I am trying to mark missing fields in the logs, but some log entries with missing fields cannot match successfully, which prevents them from being marked. What should I do? Is there something wrong with my configuration file?  
 ![2](https://us1.discourse-cdn.com/elastic/original/3X/9/5/950a3612492484afcc43c3a147eb093ea5688c41.png)  
The original log format is:  
2024-10-15T00:00:03.172528+08:00 yp-VMware-Virtual-Platform systemd[1]: rsyslog.service: Sent signal SIGHUP to main process 1346 (rsyslogd) on client request.  
How can I successfully match and check for missing fields in each log entry?  
Please, everyone, help me！

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 21, 2024, 1:02pm UTC](https://discuss.elastic.co/t/why-cant-this-match-successfully/369146/2 "2024-10-21T13:02:07Z")

</div>

> [@WeirdorPersist](#):
>
> I am trying to mark missing fields in the logs, but some log entries with missing fields cannot match successfully, which prevents them from being marked.

Please show us the log lines you are trying to parse and the pattern you are using to parse them so that we can reproduce the problem. Do not post pictures of text, just post text. Pictures are not searchable, not pasteable, and some folks may not even be able to see them.

---

<div class="post-metadata">

### Author: ![WeirdorPersist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weirdorpersist/32/137046_2.png) [@WeirdorPersist](https://discuss.elastic.co/u/WeirdorPersist)
#### Post date: [October 21, 2024, 1:13pm UTC](https://discuss.elastic.co/t/why-cant-this-match-successfully/369146/3 "2024-10-21T13:13:02Z")

</div>

`(?:%{TIMESTAMP_ISO8601:timestamp})? (?:%{DATA:host})? (?:%{DATA:process_name})?(?:\[%{NUMBER:pid}\])?: (?:%{GREEDYDATA:log_message})?`

The original log format is:  
`2024-10-15T00:00:03.172528+08:00 yp-VMware-Virtual-Platform systemd[1]: rsyslog.service: Sent signal SIGHUP to main process 1346 (rsyslogd) on client request.`

I am trying to parse logs in the following format:

```auto
yp-VMware-Virtual-Platform systemd[1]: rsyslog.service: Sent signal SIGHUP to main process 1346 (rsyslogd) on client request.
2024-10-15T00:00:03.172528+08:00 yp-VMware-Virtual-Platform systemd[1]:
2024-10-15T00:00:03.172528+08:00 systemd[1]: rsyslog.service: Sent signal SIGHUP to main process 1346 (rsyslogd) on client request.

```

I have set each field as optional so that the match can succeed even when fields are missing.Perhaps the error is related to the spaces?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 21, 2024, 1:30pm UTC](https://discuss.elastic.co/t/why-cant-this-match-successfully/369146/4 "2024-10-21T13:30:29Z")

</div>

> [@WeirdorPersist](#):
>
> Perhaps the error is related to the spaces?

Yes, if the space and log message after the process name and pid are optional then they need to be optional in the pattern. You could change

```
(?:%{DATA:process_name})?(?:\[%{NUMBER:pid}\])?: (?:%{GREEDYDATA:log_message})?

```

to

```
(?:%{DATA:process_name})?(?:\[%{NUMBER:pid}\])?:(?: %{GREEDYDATA:log_message})?

```

Then

```
input { generator { count => 1 lines => [
    '2024-10-15T00:00:03.172528+08:00 yp-VMware-Virtual-Platform systemd[1]: rsyslog.service: Sent signal SIGHUP to main process 1346 (rsyslogd) on client request.',
    '2024-10-15T00:00:03.172528+08:00 yp-VMware-Virtual-Platform systemd[1]:',
    '2024-10-15T00:00:03.172528+08:00 systemd[1]: rsyslog.service: Sent signal SIGHUP to main process 1346 (rsyslogd) on client request.'
] } }
output { stdout { codec => rubydebug { metadata => false } } }
filter {
    grok { match => { "message" => "(?:%{TIMESTAMP_ISO8601:timestamp})? (?:%{DATA:host})? (?:%{DATA:process_name})?(?:\[%{NUMBER:pid}\])?:(?: %{GREEDYDATA:log_message})?" } }
}

```

will parse all three messages.

---

<div class="post-metadata">

### Author: ![WeirdorPersist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weirdorpersist/32/137046_2.png) [@WeirdorPersist](https://discuss.elastic.co/u/WeirdorPersist)
#### Post date: [October 22, 2024, 2:34am UTC](https://discuss.elastic.co/t/why-cant-this-match-successfully/369146/5 "2024-10-22T02:34:06Z")

</div>

Thank you so much!
