# FreeIPA Log Configuration for Logstash (Grok Filter) **URL:** \ **Category:** Logstash\ **Created:** [June 19, 2017, 3:39am UTC](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876 "2017-06-19T03:39:41Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![mvelez83](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@mvelez83](https://discuss.elastic.co/u/mvelez83)\ **Post date:** [June 19, 2017, 3:39am UTC](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876/1 "2017-06-19T03:39:41Z")
Hello, I was trying to follow [https://www.freeipa.org/page/Centralized\_Logging](https://www.freeipa.org/page/Centralized_Logging) to configure the following logs from FreeIPA in logstash: /var/log/httpd/error\_log: FreeIPA API call logs (and Apache errors) /var/log/krb5kdc.log: FreeIPA KDC utilization /var/log/dirsrv/slapd-$REALM/access: Directory Server utilization /var/log/dirsrv/slapd-$REALM/errors: Directory Server errors (including mentioned replication errors) The issue is the page is for rsyslog but I'm using the following setup: Filebeat \> Redis \> Logstash \> Elasticsearch Would anyone have any recommendations on how to create a grok filter for the above logs using the above setup? Thank you for any and all help! Thank You ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 19, 2017, 5:31am UTC](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876/2 "2017-06-19T05:31:46Z")
What do the logs look like? ---
**Author:** ![mvelez83](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@mvelez83](https://discuss.elastic.co/u/mvelez83)\ **Post date:** [June 19, 2017, 8:25am UTC](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876/3 "2017-06-19T08:25:37Z")
Hello, Here is a sample of data for each log. /var/log/httpd/error\_log: > [Sun Jun 18 03:41:01.802587 2017] [auth\_digest:notice] [pid 17530] AH01757: generating secret for digest authentication ... > [Sun Jun 18 03:41:01.803553 2017] [lbmethod\_heartbeat:notice] [pid 17530] AH02282: No slotmem from mod\_heartmonitor > [Sun Jun 18 03:41:01.803571 2017] [:warn] [pid 17530] NSSSessionCacheTimeout is deprecated. Ignoring. > [Sun Jun 18 03:41:01.807129 2017] [mpm\_prefork:notice] [pid 17530] AH00163: Apache/2.4.6 (Red Hat Enterprise Linux) mod\_auth\_gssapi/1.4.0 mod\_nss/1.0.14 NSS/3.21 Basic ECC mod\_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations > [Sun Jun 18 03:41:01.807148 2017] [core:notice] [pid 17530] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND' > [Sun Jun 18 03:41:06.643044 2017] [:error] [pid 30351] ipa: INFO: \*\*\* PROCESS START \*\*\* > [Sun Jun 18 03:41:06.646679 2017] [:error] [pid 30350] ipa: INFO: \*\*\* PROCESS START \*\*\* /var/log/krb5kdc.log: Jun 19 08:19:23 [sipa01.c01.aaa.gen.nyc.aevtech.net](http://sipa01.c01.aaa.gen.nyc.aevtech.net) krb5kdc17497: closing down fd 4 Jun 19 08:19:23 [sipa01.c01.aaa.gen.nyc.aevtech.net](http://sipa01.c01.aaa.gen.nyc.aevtech.net) krb5kdc17497: TGS\_REQ (6 etypes {18 17 16 23 25 26}) 10.34.80.90: ISSUE: authtime 1497860363, etypes {rep=18 tkt=18 ses=18}, host/seg04.c20.streaming.nyc.aevtech.net@AEVTECH.NET for ldap/sipa01.c01.aaa.gen.nyc.aevtech.net@AEVTECH.NET Jun 19 08:19:23 [sipa01.c01.aaa.gen.nyc.aevtech.net](http://sipa01.c01.aaa.gen.nyc.aevtech.net) krb5kdc17497: closing down fd 4 /var/log/dirsrv/slapd-MLBAM-NET/access: > [19/Jun/2017:08:22:10.256710575 +0000] conn=146207 op=536114 RESULT err=0 tag=101 nentries=0 etime=0 > [19/Jun/2017:08:22:10.258331605 +0000] conn=252725 op=1 UNBIND > [19/Jun/2017:08:22:10.258345573 +0000] conn=252725 op=1 fd=305 closed - U1 > [19/Jun/2017:08:22:10.259720144 +0000] conn=252726 fd=305 slot=305 connection from 10.34.116.95 to 10.34.192.40 > [19/Jun/2017:08:22:10.260070526 +0000] conn=252726 op=0 SRCH base="" scope=0 filter="(objectClass=_)" attrs="_ altServer namingContexts supportedControl supportedExtension supportedFeatures supportedLDAPVersion supportedSASLMechanisms domaincontrollerfunctionality defaultnamingcontext lastusn highestcommittedusn aci" > [19/Jun/2017:08:22:10.260958407 +0000] conn=252726 op=0 RESULT err=0 tag=101 nentries=1 etime=0 > [19/Jun/2017:08:22:10.268216523 +0000] conn=146207 op=536115 SRCH base="dc=aevtech,dc=net" scope=2 filter="(&(|(objectClass=krbprincipalaux)(objectClass=krbprincipal)(objectClass=ipakrbprincipal))(|(ipaKrbPrincipalAlias=host/java05.ad01.hls.aevtech.net@AEVTECH.NET)(krbPrincipalName:caseIgnoreIA5Match:=host/java05.ad01.hls.nyc.aevtech.net@AEVTECH.NET)))" attrs="krbPrincipalName krbCanonicalName krbUPEnabled krbPrincipalKey krbTicketPolicyReference krbPrincipalExpiration krbPasswordExpiration krbPwdPolicyReference krbPrincipalType krbPwdHistory krbLastPwdChange krbPrincipalAliases krbLastSuccessfulAuth krbLastFailedAuth krbLoginFailedCount krbPrincipalAuthInd krbExtraData krbLastAdminUnlock krbObjectReferences krbTicketFlags krbMaxTicketLife krbMaxRenewableAge nsAccountLock passwordHistory ipaKrbAuthzData ipaUserAuthType ipatokenRadiusConfigLink objectClass" /var/log/dirsrv/slapd-MLBAM-NET/errors: > [15/Jun/2017:19:28:34.149592432 +0000] Retry count exceeded in delete > [15/Jun/2017:19:28:34.172819252 +0000] DSRetroclPlugin - delete\_changerecord: could not delete change record 277064 (rc: 51) > [15/Jun/2017:19:28:46.443457661 +0000] schema-compat-plugin - warning: no entries set up under cn=computers, cn=compat,dc=mlbam,dc=net > [15/Jun/2017:19:28:46.447371413 +0000] schema-compat-plugin - Finished plugin initialization. > [16/Jun/2017:18:28:20.827952664 +0000] is\_allowed\_to\_access\_attr - [file ipa\_pwd\_extop.c, line 777]: slapi\_access\_allowed does not allow WRITE to ipaProtectedOperation;write\_keys! > [16/Jun/2017:18:28:20.830961454 +0000] ipapwd\_getkeytab - [file ipa\_pwd\_extop.c, line 1651]: Not allowed to set keytab on [host/hyp189..nyc.aevtech.net@AEVTECH.NET]! > [16/Jun/2017:18:28:22.701435694 +0000] is\_allowed\_to\_access\_attr - [file ipa\_pwd\_extop.c, line 777]: slapi\_access\_allowed does not allow WRITE to ipaProtectedOperation;write\_keys! > [16/Jun/2017:18:28:22.704530887 +0000] ipapwd\_getkeytab - [file ipa\_pwd\_extop.c, line 1651]: Not allowed to set keytab on [host/hyp191.nyc.aevtech.net@AEVTECH.NET]! Let me know if you need anything else so I can send it to you as soon as possible. Thank you! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 19, 2017, 11:35am UTC](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876/4 "2017-06-19T11:35:26Z")
Ok, so what groks have you tried for each? ---
**Author:** ![mvelez83](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@mvelez83](https://discuss.elastic.co/u/mvelez83)\ **Post date:** [June 19, 2017, 12:39pm UTC](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876/5 "2017-06-19T12:39:22Z")
Hello, I was going to try the following: grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" } add\_field =\> ["received\_at", "%{@timestamp}"] add\_field =\> ["received\_from", "%{host}"] } I wasn't sure if that would work with these logs. I'm also new to using logstash and grok filters so I wasn't 100% sure if this would work from what I read since the article was based on rsyslog. Thank You ---
**Author:** ![mvelez83](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@mvelez83](https://discuss.elastic.co/u/mvelez83)\ **Post date:** [June 20, 2017, 1:11am UTC](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876/6 "2017-06-20T01:11:03Z")
Hello, Would the above grok filter work? I am new to grok filters and I'm trying to understand how to structure them depending on the log output. My confusion is when it comes to the "match" field in the grok filter, how do we know which options to select for the match field? Thank you ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 18, 2017, 1:25am UTC](https://discuss.elastic.co/t/freeipa-log-configuration-for-logstash-grok-filter/89876/7 "2017-07-18T01:25:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.