A problem with _id in the mapping

Zhoutuo_Yang · July 9, 2013, 8:43pm

Hi folks,
I'm having trouble with _id field.
I use path to referring another field in the document for the id. Like
"_id": {
"path": "myId"
}.
The problem is the elasticsearch seems to reject the document with the same
"myId" which will become the "_id" after indexing.
Isn't the overwriting a more nature way instead of rejecting?
BTW, I'm feeding docs using logstash with elasticsearch_http
Thank you.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

radu_gheorghe · July 10, 2013, 8:16am

Hello,

I can't reproduce this issue, unless I specifically say to "create"
documents instead of "index". Here's a gist:

gist.github.com

https://gist.github.com/radu-gheorghe/5964376

id_path_overwrite.sh

#!/bin/bash

curl -XDELETE localhost:9200/test
curl -XPUT localhost:9200/test
curl -XPUT localhost:9200/test/test/_mapping -d '{
  "test": {
    "_id": {
      "path": "foo"
    }
  }

This file has been truncated. show original

Can you provide a gist where the problem is recreated? Also, what versions
of ES and logstash are you using?

Best regards,
Radu

On Tue, Jul 9, 2013 at 11:43 PM, Zhoutuo Yang zhoutuoy@gmail.com wrote:

Hi folks,
I'm having trouble with _id field.
I use path to referring another field in the document for the id. Like
"_id": {
"path": "myId"
}.
The problem is the elasticsearch seems to reject the document with the
same "myId" which will become the "_id" after indexing.
Isn't the overwriting a more nature way instead of rejecting?
BTW, I'm feeding docs using logstash with elasticsearch_http
Thank you.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Zhoutuo_Yang · July 10, 2013, 5:16pm

Hi,
Thank you for replying, I found that it is a logstash issue. The plugin
elasticsearch_http is unstable somehow.
I pulled it out, made it my own plugin with the same source code, it all
worked.
Version of logstash: 1.1.13, ES: 0.90.1
BTW, the exception was also thrown by logstash not ES

On Wed, Jul 10, 2013 at 1:16 AM, Radu Gheorghe
radu.gheorghe@sematext.comwrote:

Hello,

I can't reproduce this issue, unless I specifically say to "create"
documents instead of "index". Here's a gist:
Elasticsearch: If you create a mapping with the ID field pointing to a certain path, inserting two docs with the same ID should overwrite, instead of throwing an error. Unless you specify to "create" documents · GitHub

Can you provide a gist where the problem is recreated? Also, what versions
of ES and logstash are you using?

Best regards,
Radu

On Tue, Jul 9, 2013 at 11:43 PM, Zhoutuo Yang zhoutuoy@gmail.com wrote:

Hi folks,
I'm having trouble with _id field.
I use path to referring another field in the document for the id. Like
"_id": {
"path": "myId"
}.
The problem is the elasticsearch seems to reject the document with the
same "myId" which will become the "_id" after indexing.
Isn't the overwriting a more nature way instead of rejecting?
BTW, I'm feeding docs using logstash with elasticsearch_http
Thank you.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/LUXCx4i6W5g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.