Store Indexes on multiple drives

Elastic Stack Elasticsearch
1

I am new to ES ,I have single node ES instance on a Windows machine, i was trying to find a way to where i can point my new index on a new drive but i could not find any way of handling that.

Even adding multiple path in elasticsearch.yml file in path.data is failing
example: path.data : [C:\ES,D:\ES] does not work.
Is there a way of handling that? any help would be much appreciated.

2

If you have a simple one-node setup then it's probably best to keep all your data in one place. If you want to move the Elasticsearch data from one drive to another then you can shut the node down, move the data path to the new drive, adjust your node's configuration to point to the new location and then restart the node.

3

Thank you for your response @DavidTurner . I would still want to understand the process of adding multiple disks to path.data to see if it gives me any performance gain. Could you suggest me a way to achieve that.

4

Ok, yes, you can try to set path.data: ["C:\ES", "D:\ES"], but note that Elasticsearch doesn't move existing data between the two paths to do any kind of balancing, so you'll only see an effect with new indices.

5

@DavidTurner I tried updating the path as said but ES fails to start, the idea is to use the available storage to split the data across.

6

ES rarely fails to start without emitting helpful log messages. Can you share the messages you're seeing?

7

@DavidTurner
image

this is the message

8

A screenshot of half of an error message is basically useless.

9

Exception in thread "main" SettingsException[Failed to load settings from [elasticsearch.yml]]; nested: MarkedYAMLException[while scanning a double-quoted scalar
in 'reader', line 9, column 13:
path.data: ["I:\ES","H:\ES"]
^
found unknown escape character E(69)
in 'reader', line 9, column 17:
path.data: ["I:\ES","H:\ES"]
^

at [Source: sun.nio.ch.ChannelInputStream@2ab4bc72; line: 9, column: 13]]; nested: ScannerException[while scanning a double-quoted scalar
in 'reader', line 9, column 13:
path.data: ["I:\ES","H:\ES"]
^
found unknown escape character E(69)
in 'reader', line 9, column 17:
path.data: ["I:\ES","H:\ES"]
^
];
at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1216)
at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1189)
at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:100)
at org.elasticsearch.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:95)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
Caused by: com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException: while scanning a double-quoted scalar
in 'reader', line 9, column 13:
path.data: ["I:\ES","H:\ES"]
^
found unknown escape character E(69)
in 'reader', line 9, column 17:
path.data: ["I:\ES","H:\ES"]
^

at [Source: sun.nio.ch.ChannelInputStream@2ab4bc72; line: 9, column: 13]
at com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLException.from(MarkedYAMLException.java:27)
at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:343)
at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:52)
at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:733)
at org.elasticsearch.common.settings.Settings.fromXContent(Settings.java:699)
at org.elasticsearch.common.settings.Settings.access$500(Settings.java:84)
at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1212)
... 8 more
Caused by: while scanning a double-quoted scalar
in 'reader', line 9, column 13:
path.data: ["I:\ES","H:\ES"]
^
found unknown escape character E(69)
in 'reader', line 9, column 17:
path.data: ["I:\ES","H:\ES"]

                ^

    at org.yaml.snakeyaml.scanner.ScannerImpl.scanFlowScalarNonSpaces(ScannerImpl.java:1906)
    at org.yaml.snakeyaml.scanner.ScannerImpl.scanFlowScalar(ScannerImpl.java:1843)
    at org.yaml.snakeyaml.scanner.ScannerImpl.fetchFlowScalar(ScannerImpl.java:1029)
    at org.yaml.snakeyaml.scanner.ScannerImpl.fetchDouble(ScannerImpl.java:1011)
    at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:396)
    at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:226)
    at org.yaml.snakeyaml.parser.ParserImpl$ParseFlowSequenceEntry.produce(ParserImpl.java:628)
    at org.yaml.snakeyaml.parser.ParserImpl$ParseFlowSequenceFirstEntry.produce(ParserImpl.java:616)
    at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)
    at org.yaml.snakeyaml.parser.ParserImpl.getEvent(ParserImpl.java:167)
    at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:340)
    ... 13 more
10

Thanks. Looks like a Windows thing, I think you have to double all the backslashes:

path.data: ["I:\\ES","H:\\ES"]
11

@DavidTurner Thank you so much, it solved my issue.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.