How do I index an _id with + symbol?

Elastic Stack Elasticsearch
1

Making the switch from ES2 to ES6 and having a serious problem. Whenever I index a record with an id that contains a + symbol, Elasticsearch replaces the + with a space. The “id” field is correct, but the “_id” field contains a space.

Later, when I search for data and retrieve the key, I have a mismatch in my document store (dynamo) because the + have been replaced. In other words, by modifying the primary key used across databases, it makes it difficult to link the records.

Is it possible to escape the + so that _id is stored with the value I want? I’ve tried \ + or %2B for the +, and other variations but it does not work. I also tried changing the mapping on the index but nothing seems to work and I got an error trying to change _id field to a keyword.

Any tips for this? It seems like such a simple thing to store a symbol as part of the string, but ES6 is making it painful.

Not sure if it matters, but I’m using the Java Rest 6 client for all operations. Below is simple sample code to reproduce this:

RestHighLevelClient client = new RestHighLevelClient(...);
IndexRequest request = new IndexRequest("test", "test", "+1234");
request.source("{}", XContentType.JSON);
client.index(request,RequestOptions.DEFAULT);

And the data in ES:

GET test/_search
{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
	"_index": "test",
	"_type": "test",
	"_id": " 1234",
	"_score": 1,
	"_source": {}
      }
    ]
  }
}

Note the space and lack of + in the "_id" field.

2

Which version of Elasticsearch 6 are you using specifially, I'll try to see if this still happens in the latest 6.4 version.

3

Hah, the "+" sign seems to be a special snowflake for a while: https://github.com/elastic/elasticsearch/issues/5341

I tried the %2b encoding in the Kibana Console and it works as a workaround at least there:

PUT /test/_doc/%2b1234
{
  "foo" : "bar"
}

GET /test/_search

GET /test/_search
{
  "query": {
    "ids": {
      "values": ["+1234"]
    }
  }
}

Still need to try this with the Java HL Rest client though.

4

There already seems to be an open issue for this in relation with the HL Rest client. Doesn't seem to be merged yet though unfortunately:

5

I'm running ES 6.3.2 and I tried using the DevTools console to reproduce your work. Unfortunately, I don't get the results you posted with using %2b instead of +. Instead, it shows the _id having a space. Is this something they fixed after 6.3.2?

6

I installed a 6.4.2 version and can confirm the following:

  1. There is a server-side issue in 6.3.2 that automatically sets + to a space using Dev Tools. This is fixed in 6.4.2.

  2. In 6.4.2, you still can't actually pass a + even using Dev Tools. You still have to pass %2b instead of +.

  3. Using 6.4.2, there is a bug in the High-Level Rest Client that does not +'s to be used, whether they be passed as %2b or + that has not been merged into the repo.

In other words, to fix this using a low-level client, you need to be running at least 6.4.2. To fix this with the high-level client, have to wait for at least 6.5.0 or future when the issue is closed.

Sigh... I hate work-arounds but I guess I'll to do something stupid until this is fixed in the appropriate places.

1 Like
7

Sorry for the inconvenience and for hanging in there.

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.