# Cluster configuarton for ELK stack with Kafka

**URL:** <https://discuss.elastic.co/t/cluster-configuarton-for-elk-stack-with-kafka/112823>\
**Category:** Logstash\
**Created:** [December 21, 2017, 12:50pm UTC](https://discuss.elastic.co/t/cluster-configuarton-for-elk-stack-with-kafka/112823 "2017-12-21T12:50:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\
**Post date:** [December 21, 2017, 12:50pm UTC](https://discuss.elastic.co/t/cluster-configuarton-for-elk-stack-with-kafka/112823/1 "2017-12-21T12:50:24Z")

</div>

Hi All,

I am done with standalone testing of ELK with Kafka now i want to implement Full Clustered ELK + Kafka .

Please share docs and imp points for the same .

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 21, 2017, 1:28pm UTC](https://discuss.elastic.co/t/cluster-configuarton-for-elk-stack-with-kafka/112823/2 "2017-12-21T13:28:08Z")

</div>

You'll get more useful answers if you ask a more concrete question.

---

<div class="post-metadata">

**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\
**Post date:** [December 22, 2017, 7:45am UTC](https://discuss.elastic.co/t/cluster-configuarton-for-elk-stack-with-kafka/112823/3 "2017-12-22T07:45:04Z")

</div>

Below is my zookeeper config file ,

Zoo.cfg

############################################

# The number of milliseconds of each tick

tickTime=2000

# The number of ticks that the initial

# synchronization phase can take

initLimit=10

# The number of ticks that can pass between

# sending a request and getting an acknowledgement

syncLimit=5

# the directory where the snapshot is stored.

# do not use /tmp for storage, /tmp here is just

# example sakes.

dataDir=/zookeeper/data

# dataLogDir ======= \>\>\>\>\> where you would like ZooKeeper to log

dataLogDir=/zookeeper/logs

# the port at which the clients will connect

clientPort=2181

# the maximum number of client connections.

# increase this if you need to handle more clients

maxClientCnxns=60

# Be sure to read the maintenance section of the

# administrator guide before turning on autopurge.

# 

# [http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc\_maintenance](http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance)

# The number of snapshots to retain in dataDir

autopurge.snapRetainCount=5

# Purge task interval in hours

# Set to "0" to disable auto purge feature

autopurge.purgeInterval=0

# 3DS Zookeeper cluster vkafka0[1-3]dsy

server.1=PQR:2888:3888  
server.2=XYZ:2888:3888  
server.3=ABC:2888:3888

######################################################

I am trying to start my zookeeper cluster but getting below exception , on all 3 servers . what is wrong with config files .

2017-12-22 08:36:56,993 [myid:] - INFO [main:QuorumPeerConfig@136] - Reading configuration from: /usr/local/zookeeper/bin/../conf/zoo.cfg  
2017-12-22 08:36:57,003 [myid:] - INFO [main:QuorumPeer$QuorumServer@184] - Resolved hostname: ABC to address: /ABC  
2017-12-22 08:36:57,003 [myid:] - INFO [main:QuorumPeer$QuorumServer@184] - Resolved hostname: XYZ to address: /XYZ  
2017-12-22 08:36:57,004 [myid:] - INFO [main:QuorumPeer$QuorumServer@184] - Resolved hostname: PQR to address: /PQR  
2017-12-22 08:36:57,004 [myid:] - INFO [main:QuorumPeerConfig@398] - Defaulting to majority quorums  
2017-12-22 08:36:57,005 [myid:] - ERROR [main:QuorumPeerMain@88] - Invalid config, exiting abnormally  
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing /usr/local/zookeeper/bin/../conf/zoo.cfg  
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:156)  
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:104)  
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)  
Caused by: java.lang.IllegalArgumentException: /zookeeper/data/myid file is missing  
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:408)  
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:152)  
... 2 more  
Invalid config, exiting abnormally

please help on the same

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 22, 2017, 8:12am UTC](https://discuss.elastic.co/t/cluster-configuarton-for-elk-stack-with-kafka/112823/4 "2017-12-22T08:12:51Z")

</div>

This forum is for Logstash questions. Zookeeper questions belong elsewhere.

> /zookeeper/data/myid file is missing

Isn't this error message pretty clear?

---

<div class="post-metadata">

**Author:** ![SJN8](https://avatars.discourse-cdn.com/v4/letter/s/f08c70/32.png) [@SJN8](https://discuss.elastic.co/u/SJN8)\
**Post date:** [December 22, 2017, 9:29am UTC](https://discuss.elastic.co/t/cluster-configuarton-for-elk-stack-with-kafka/112823/5 "2017-12-22T09:29:54Z")

</div>

yes i have created file myid .

But i am getting some other exception now .

Can you help me where i can ask for kafka related queries .

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 19, 2018, 9:29am UTC](https://discuss.elastic.co/t/cluster-configuarton-for-elk-stack-with-kafka/112823/6 "2018-01-19T09:29:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.