Whenever I try to take a snapshot of an index which contains a ':' colon in
its name,I end up with the following trace:
{
"error":"IllegalArgumentException[java.net.URISyntaxException: Relative
path in absolute URI: crawl:1]; nested: URISyntaxException[Relative path in
absolute URI: crawl:1]; ",
"status":500
}
It does not matter if the 'indices' argument is provided or not, the
snapshot name is set to 'snapshot'. I have tried to specify the index
escaping the sign with '%3a' but in this case the name does not fit any
available indices.
The question is, is there a way to somehow escape the character (if not
within the request then perhaps code itself?) and if so, would creating a
feature request make sense?
':' has a special meaning in an URI, which is what HDFS uses. You basically have to either escape the character (%3A) or
use a different character.
Potentially you can rename the file to the desired name by running a separate command after the job has been completed.
However, no URI/URL can be constructed from your file-name, in the format you desire, so you'll have to encode/decode
the location every-time.
On 9/1/14 8:10 PM, Mateusz Kaczynski wrote:
Whenever I try to take a snapshot of an index which contains a ':' colon in its name,I end up with the following trace:
{
"error":"IllegalArgumentException[java.net.URISyntaxException: Relative path in absolute URI: crawl:1]; nested:
URISyntaxException[Relative path in absolute URI: crawl:1]; ",
"status":500
}
It does not matter if the 'indices' argument is provided or not, the snapshot name is set to 'snapshot'. I have tried to
specify the index escaping the sign with '%3a' but in this case the name does not fit any available indices.
I assume this is related to [HDFS-13] filenames with ':' colon throws java.lang.IllegalArgumentException - ASF JIRA filenames with ':' colon throws
java.lang.IllegalArgumentException?
The question is, is there a way to somehow escape the character (if not within the request then perhaps code itself?)
and if so, would creating a feature request make sense?
Yes, I understand that this is a restricted character and would require
escaping.
There are perhaps 2 separate issues here:
If 'indices' is not specified, i.e.
curl -XPUT "localhost:9200/_snapshot/hdfs-cluster/snapshot_1"
elasticsearch is going to complain with the mentioned error when it finds
any index with colon in its name and stop.
If 'indices' is in use, I don't see a way(would be great to be proved
wrong) to escape the name, i.e. doing something like this
curl -XPUT "localhost:9200/_snapshot/hdfs-cluster/snapshot_2" -d '{"indices":
"crawl%3A1"}'
finishes empty as the index with such a name does not exist.
That's why I though escaping might be done somewhere in the plugin?
If it's only the index that you are interested, why not use aliases instead?
Have the index snapshot/restored through a legal fs name (crawl-1) and have an alias
that uses the special character (crawl:1) pointing to it?
On 9/2/14 2:20 PM, Mateusz Kaczynski wrote:
Hi Costin, thanks for the response.
Yes, I understand that this is a restricted character and would require escaping.
There are perhaps 2 separate issues here:
If 'indices' is not specified, i.e.
|
curl -XPUT "localhost:9200/_snapshot/hdfs-cluster/snapshot_1"
|
elasticsearch is going to complain with the mentioned error when it finds any index with colon in its name and stop.
If 'indices' is in use, I don't see a way(would be great to be proved wrong) to escape the name, i.e. doing something
like this
|
curl -XPUT "localhost:9200/_snapshot/hdfs-cluster/snapshot_2"-d '{"indices": "crawl%3A1"}'
|
finishes empty as the index with such a name does not exist.
That's why I though escaping might be done somewhere in the plugin?
If the alias method doesn't work for you then escaping might be the only way (assuming there's going to be a reliable
way to apply it) - I suggest raising an issue since this sounds like a generic functionality that could work across all
snapshot/restore plugins.
Cheers,
On 9/2/14 4:03 PM, Mateusz Kaczynski wrote:
I'm afraid it's the other way around, i.e. we already have 50 indices or so (+tools) using colon in the base name.
And just to check that as well, specifying alias in 'indices' instead of the base name still results in the same error.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.