# Custom filter Grok syntax help **URL:** \ **Category:** Logstash\ **Created:** [September 28, 2018, 8:50pm UTC](https://discuss.elastic.co/t/custom-filter-grok-syntax-help/150388 "2018-09-28T20:50:47Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![hueyg](https://avatars.discourse-cdn.com/v4/letter/h/8c91f0/32.png) [@hueyg](https://discuss.elastic.co/u/hueyg)\ **Post date:** [September 28, 2018, 8:50pm UTC](https://discuss.elastic.co/t/custom-filter-grok-syntax-help/150388/1 "2018-09-28T20:50:47Z")
Most tutorials simply used the built in filters and leave it at that, but I am trying to parse an LDAP log and running into a few issues. The following post almost matches my issue, but need more clarification in general as I am totally new. > [@How filter OpenLDAP Logs?](https://discuss.elastic.co/t/how-filter-openldap-logs/2492): > > Hi, Attempting to parse LDAP access logs (sample below): required to be able to find transactions by 'conn' (to relate the IP Address to a BIND) for example. Jun 11 09:53:08 ldapserver slapd[19329]: conn=1202229 fd=28 ACCEPT from IP=1.2.3.4:57632 (IP=0.0.0.0:389) Jun 11 09:53:08 ldapserver slapd[19329]: conn=1202229 op=0 BIND dn="cn=User,c=corp" method=128 Jun 11 09:53:08 ldapserver slapd[19329]: conn=1202229 op=0 BIND dn="cn=User,c=corp" mech=SIMPLE ssf=0 Jun 11 09:53:08 ldapserver slapd… Here is an example of one line of the log, although there about six other types of events that are logged: > [28/Sep/2018:14:08:43.893180585 -0500] conn=12345678 fd=77 slot=77 SSL connection from 10.10.10.10 to 192.192.192.192 So I initially sent this log over using the Logstash built in syslog filter and it sort of worked. Somehow it derived the timestamp from the first block of text. My filter starts with this: match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} 1. So this appears to work, but I am not sure how. This does not work using the GrokDebug site so not sure. Also do the square brackets not cause problems with the parsing? 2. How does Grok handle delineations? Here they are spaces, some logs are commas, others may be tabs. In this case it seemed to just know that spaces separate the fields. 3. So I can create a field for each section of the line, for example, conn=%{INT:conn} fd=%{INT:fd} slot=%{INT:slot}. That works great. BUT what if I don't want to parse the fd field for example. Can I somehow tell Grok to skip a section and start parsing at the next section? I know this goes into a lot of filtering basics but I can't seem to find this explained in detail anywhere. Thanks again for the help. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 3, 2018, 5:49pm UTC](https://discuss.elastic.co/t/custom-filter-grok-syntax-help/150388/2 "2018-10-03T17:49:28Z")
> So this appears to work, but I am not sure how. Neither am I. SYSLOGTIMESTAMP _shouldn't_ work with the example given. > How does Grok handle delineations? Here they are spaces, some logs are commas, others may be tabs. In this case it seemed to just know that spaces separate the fields. Grok expressions are really just regular expressions, and they need to match exactly. > So I can create a field for each section of the line, for example, conn=%{INT:conn} fd=%{INT:fd} slot=%{INT:slot}. That works great. BUT what if I don't want to parse the fd field for example. Can I somehow tell Grok to skip a section and start parsing at the next section? The `?`operator in regular expressions means "zero or one occurrence of the preceding token", which together with parentheses allows you to specify that a sequence of token is optional. Your example could be modified to ``` conn=%{INT:conn} (fd=%{INT:fd} )?slot=%{INT:slot} ``` to make the `fd` field optional. Note that the kv filter is usually the best option for parsing key/value pairs. But wait. If you actually meant "the fd=X stuff is always there, I just don't care about capturing it into a field" you're actually looking for this: ``` conn=%{INT:conn} fd=%{INT} slot=%{INT:slot} ``` ---
**Author:** ![hueyg](https://avatars.discourse-cdn.com/v4/letter/h/8c91f0/32.png) [@hueyg](https://discuss.elastic.co/u/hueyg)\ **Post date:** [October 5, 2018, 2:59pm UTC](https://discuss.elastic.co/t/custom-filter-grok-syntax-help/150388/3 "2018-10-05T14:59:02Z")
Thanks Magnus, I am slowly getting my mind in the correct context to digest this stuff. Yes, it was the second option of always there but I don't care about capturing. Very helpful stuff. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 2, 2018, 2:59pm UTC](https://discuss.elastic.co/t/custom-filter-grok-syntax-help/150388/4 "2018-11-02T14:59:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.