# Logstash to Kafka with SASL

**URL:** https://discuss.elastic.co/t/logstash-to-kafka-with-sasl/78918
**Category:** Logstash
**Created:** [March 16, 2017, 5:58pm UTC](https://discuss.elastic.co/t/logstash-to-kafka-with-sasl/78918 "2017-03-16T17:58:38Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Markovich](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Markovich](https://discuss.elastic.co/u/Markovich)
#### Post date: [March 16, 2017, 5:58pm UTC](https://discuss.elastic.co/t/logstash-to-kafka-with-sasl/78918/1 "2017-03-16T17:58:38Z")

</div>

Hello everyone!

I'm new to logstash!  
I need to send logs from hdfs-namenode to kafka broker with kerberos.  
Here is my configs and full log of logstash.

Please tell me what's wrong?

I think logstash is not using jaas.conf because if I comment sasl\_kerberos\_service\_name it will say that this property is needed but this can be read from jaas.conf...

hdfs-nn.conf  
++++++++++++++++++++++  
input {  
file {  
type =\> "hdp-nn-audit"  
path =\> "/u02/var/log/hadoop/hdfs/hdfs-audit.log"  
start\_position =\> end  
sincedb\_path =\> "/var/log/logstash/since.db"  
}  
}

filter{  
if [type] == "hdp-nn-audit" {  
grok {  
match =\> ["message", "ugi=(?([\w\d-]+))@|ugi=(?([\w\d-]+))/[\w\d-.]+@|ugi=(?([\w\d.-\_]+))[\s(]+"]  
}  
}  
}

output {  
if [type] == "hdp-nn-audit" {  
kafka {  
codec =\> plain {  
format =\> "%{message}"  
}  
bootstrap\_servers =\> "demo4:6667"  
topic\_id =\> "hdfs\_audit\_log"  
security\_protocol =\> "SASL\_PLAINTEXT"  
sasl\_kerberos\_service\_name =\> "kafka"  
jaas\_path =\> "/opt/logstash-5.2.2/config/kafka\_client\_jaas.conf"  
kerberos\_config =\> "/etc/krb5.conf"  
#request\_required\_acks =\> 0  
#request\_timeout\_ms =\> 10000  
#producer\_type =\> "async"  
#message\_send\_max\_retries =\> 3  
#retry\_backoff\_ms =\> 100  
#queue\_buffering\_max\_ms =\> 5000  
#queue\_enqueue\_timeout\_ms =\> 5000  
#batch\_num\_messages =\> 200  
#send\_buffer\_bytes =\> 102400  
client\_id =\> "hdp-nn-audit"  
#partition\_key\_format =\> "%{user}"  
}  
# stdout { codec =\> rubydebug }  
}  
}

kafka\_client\_jaas.conf  
++++++++++++++++

KafkaClient {  
com.sun.security.auth.module.Krb5LoginModule required  
useTicketCache=true  
renewTicket=true  
serviceName="kafka";  
};

bin/logstash -f config/hdfs-nn.conf

is sending somwhere logs, but I can't see them using:

bin/kafka-console-consumer.sh --topic hdfs\_audit\_log --zookeeper [demo4.pro.ru:2181](http://demo4.pro.ru:2181) --security-protocol PLAINTEXTSASL

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 13, 2017, 5:58pm UTC](https://discuss.elastic.co/t/logstash-to-kafka-with-sasl/78918/2 "2017-04-13T17:58:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.