[ES 5.3] How do I set plugin settings?

[ddmichael]

I am writing an integration test for elasticsearch 5.3. (note: placing and overriding the refresh interval in the node settings was working on elasticsearch 1.7)

public class ProtectedWordsIndexTests extends ESIntegTestCase {
  private final WordDelimiterActionListener wordsListener = 
  WordDelimiterActionListener.getInstance();
  private final static String INDEX_NAME = "protected_words";
  private final static String TYPE_NAME = "word";
  private final static String FILTER_NAME = "my_word_delimiter";

  @Override
  protected Collection<Class<? extends Plugin>> nodePlugins() {
     return Collections.singleton(WordDelimiterPlugin.class);
  }

  @Override
  protected Settings nodeSettings(int nodeOrdinal) {
    return builder()
       .put("plugin.types", TYPE_NAME)
       .put("plugin.dynamic_word_delimiter.refresh_interval", "500ms")
       .put(super.nodeSettings(nodeOrdinal))
       .build();
  }

  public void testAddWordToIndex() throws Exception {
     Settings indexSettings = builder()
       .put(IndexMetaData.SETTING_VERSION_CREATED, Version.CURRENT)
       .put("index.analysis.filter.my_word_delimiter.type", "dynamic_word_delimiter")
       .build();
     TokenFilterFactory filterFactory = filterFactory(indexSettings, FILTER_NAME);

     createIndex(INDEX_NAME);
     ensureGreen();
     client().prepareIndex(INDEX_NAME, TYPE_NAME, "1")
        .setSource("word", "1tb")
        .execute();

     Thread.sleep(TimeValue.timeValueSeconds(1).getMillis());

     Set<String> protectedWords = wordsListener.getProtectedWords();
     assertTrue(protectedWords.size() == 1);
  }
}

When I am running testAddWordToIndex() I am getting the following error:

"java.lang.IllegalArgumentException: unknown setting [plugin.dynamic_word_delimiter.refresh_interval] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"
If I remove the following part and increase the refresh interval to be more than the default, the test passes. So I just can't override this.

.put("plugin.dynamic_word_delimiter.refresh_interval", "500ms")
The default refresh interval is declared here:

public class WordDelimiterRunnable extends AbstractRunnable {
   public static final TimeValue REFRESH_INTERVAL = TimeValue.timeValueSeconds(20);
   public static final String INDEX_NAME = "protected_words";
   public static final String INDEX_TYPE = "word";
   public static final int RESULTS_SIZE = 10000;

   private volatile boolean running;
   private final Client client;
   private final String index;
   private final long interval;
   private final String type;

   public WordDelimiterRunnable(Client client, Settings settings) {
     this.client = client;
     this.index = settings.get("plugin.dynamic_word_delimiter.protected_words_index", INDEX_NAME);
     this.type = settings.get("plugin.dynamic_word_delimiter.protected_words_type", INDEX_TYPE);
     this.interval = settings.getAsTime("plugin.dynamic_word_delimiter.refresh_interval", REFRESH_INTERVAL).getMillis();
   }
   // more code here
}

[polyfractal (Zachary Tong)]

I'm not positive this is the solution, but I think you need to also override transportClientPlugins:

@Override
protected final Collection<Class<? extends Plugin>> transportClientPlugins() {
     return nodePlugins();
}

[ddmichael]

Thanks. I finally found the missing link. I had to prefix the settings' strings with "node.attr". Hence,
"node.attr.plugin.dynamic_word_delimiter.protected_words_index" etc..

https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_50_settings_changes.html#_node_attribute_settings

[rjernst (Ryan Ernst)]

node.attr.* is for something completely different. In this case you will be able to set your settings with that prefix, but reading them as you are in your plugin will not work.

Each plugin needs to register the settings it expects to read. This is done by overriding the getSettings() method on the Plugin class. That method returns a collection of Setting objects.

For example, for your plugin.dynamic_word_delimiter.refresh_interval setting, you can create a Setting object with:

static final Setting REFRESH_INTERVAL_SETTING = Setting.timeValue("plugin.dynamic_word_delimiter.refresh_interval", TimeValue.timeValueSeconds(10), Setting.Property.NodeScope);

Then when reading your setting in the WordDelimiterRunnable constructor, use:

this.interval = REFRESH_INTERVAL_SETTING.get(settings);

One last thing: if you plan to release this plugin, I would suggest using a more descriptive prefix than "plugin", since there may be many plugins installed.

[ddmichael]

Thanks for the enlightening answer! The thing is it works though. That is:

1. WordDelimiterRunnable correctly reads and sets the settings (using the node.attr.* prefix) which I defined inside the class.
2. Overriding the settings in elasticsearch.yml (using the node.attr.* prefix) indeed does what it's supposed to do.

I know that the above both work as I am setting the interval to an expected and measurable value, like 5 seconds and 30 seconds.

@rjernst any idea why it's working and whether it's really necessary to override Plugin's getSettings?

[rjernst (Ryan Ernst)]

When you are setting node.attr.plugin.dynamic_word_delimiter.protected_words_index, do you then read that exact setting name, or just plugin.dynamic_word_delimiter.protected_words_index? The code you had above did the latter, which should not work. But if you did the following, it would probably work (but still be wrong, these attrs are meant for allocator attributes, ie rack awareness):

settings.get("node.attr.plugin.dynamic_word_delimiter.protected_words_index", INDEX_NAME);

Overriding getSettings() is necessary; it is what that method was designed for.