File Based Discovery plugin not working

Elastic Stack Elasticsearch
#1

I'm trying to set up an elasticsearch cluster with docker containers with marathon on mesos and I'm trying to use the file based discovery. I have a cronjob that runs every 1 minute and updates the file with the list of hosts. The problem is this only works once, it doesn't update dynamically as the documentation here says: https://www.elastic.co/guide/en/elasticsearch/plugins/current/discovery-file.html

"Anytime a change is made to the unicast_hosts.txt file, even as Elasticsearch continues to run, the new changes will be picked up by the plugin and the new hosts list will be used for the next pinging round for master election."

^ This is the part that isn't working at all.

File based discovery does not work
#2

How do you know that it's not working?
What should happen on your nodes when this file is updated?

I mean: you started some nodes, then update the file, starting new ones then kill all the old ones or something like this?

Or you called an API to check that?

#3

I have a marathon job that has 3 nodes. Each node has a cron job that updates the file dynamically. First, none of the nodes form a cluster. I scale down to 2 nodes, the file gets updated, no cluster still.

Even with scaling up again to 5 nodes, the file gets updated with all of the new IPs but still elasticsearch doesn't try to form a cluster with them. I know my connectivity works because if I use the regular unicast host list they work so I'm almost 100% sure it's the plugin not working.

Not sure if this is relevant or not but I'm running elasticsearch on the alpine based docker container from the official docker hub.

#4

That's the problem you need to solve I think.

It's unrelated to the file based discovery plugin IMO.

Can you start a first node, then add a second one and share the logs?
Also sharing your elasticsearch.yml file could help.

#5

Here's the logs from the both of the nodes and the config file. http://pastebin.com/H9NPqJC0

I turned one on, waited a bit, turned another on, the list got updated with the new node yet they still don't form a cluster.

#6

Can you change the log level to DEBUG for org.elasticsearch.discovery.file package?

#7

How can I do that specifically for the discovery plugin? I know how to change it to debug using config/log4j2.properties but I'm not sure if it's what you're asking for.

rootLogger.level = info to debug?

#8

I "think" you can run:

PUT /_cluster/settings
{
    "transient" : {
        "logger.org.elasticsearch.discovery.file" : "DEBUG"
    }
}
#9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.