opType create does not seem to create duplicates


(Paul Loy) #1

Hi,

in the documentationhttp://www.elasticsearch.com/docs/elasticsearch/rest_api/index/#Operation_Typeit
states that:

"If the op_type is set to create, and there is already an existing document

with the same type and id within the index, the index will now hold twodocuments with the same id and type."

I do not see this in practice. If I do a _status on the index after creating
two documents with the same id and type it shows something like this:

"docs":{"num_docs":1,"max_doc":2,"deleted_docs":1}

is the ?op_type=create being honoured? or is there some cleanup task?

Thanks,

Paul

PS, here are my test REST calls, using 0.7.1:

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/'
{"ok":true,"acknowledged":true}

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/post/_mapping' -d '
{
post : {
properties : {
user : { type : "string", index : "not_analyzed"},
content : { type : "string", index : "analyzed"},
postDate : { type : "date", format : "YYYYMMddHHmmSS", index :
"analyzed" },
tags : { type : "string", indexName : "tag", index : "not_analyzed" }
}
}
}
'
{"ok":true,"acknowledged":true}

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/post/3?optype=create'
-d '
{
post : {
user : "keteracel",
content : "This is my first blog post using elastic search as the
backing data store!

I hope you like it!

Paul.",
postDate : "20100315193410",
tags : ["blog","first","elastic search"]
}
}
'
{"ok":true,"_index":"blogs","_type":"post","_id":"3"}

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/post/3?optype=create'
-d '
{
post : {
user : "keteracel"
}
}
'
{"ok":true,"_index":"blogs","_type":"post","_id":"3"}

-bash-3.2$ curl -XGET 'http://localhost:9200/blogs/_status'
{"ok":true,"_shards":{"total":10,"successful":5,"failed":0},"indices":{"blogs":{"aliases":[],"settings":{"index.number_of_shards":"5","index.number_of_replicas":"1"},"store_size":"1.9k","store_size_in_bytes":1949,"estimated_flushable_memory_size":"382","estimated_flushable_memory_size_in_bytes":382,"translog_operations":2,
"docs":{"num_docs":1,"max_doc":2,"deleted_docs":1}
,"shards":{"0":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":0,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}],"1":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":1,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}],"2":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":2,"index":"blogs"},"state":"STARTED","store_size":"1.7k","store_size_in_bytes":1741,"estimated_flushable_memory_size":"382","estimated_flushable_memory_size_in_bytes":382,"translog_id":0,"translog_operations":2,"docs":{"num_docs":1,"max_doc":2,"deleted_docs":1}}],"3":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":3,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}],"4":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":4,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}]}}}}

--

Paul Loy
paul@keteracel.com
http://www.keteracel.com/


(Shay Banon) #2

The parameter should be op_type, and no optype. Another option is to add
_create, for example: 'localhost:9200/index/type/id/_create'.

On Wed, May 26, 2010 at 12:53 PM, Paul Loy keteracel@gmail.com wrote:

Hi,

in the documentationhttp://www.elasticsearch.com/docs/elasticsearch/rest_api/index/#Operation_Typeit states that:

"If the op_type is set to create, and there is already an existing

document with the same type and id within the index, the index will now hold
two documents with the same id and type."

I do not see this in practice. If I do a _status on the index after
creating two documents with the same id and type it shows something like
this:

"docs":{"num_docs":1,"max_doc":2,"deleted_docs":1}

is the ?op_type=create being honoured? or is there some cleanup task?

Thanks,

Paul

PS, here are my test REST calls, using 0.7.1:

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/'
{"ok":true,"acknowledged":true}

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/post/_mapping' -d '
{
post : {
properties : {
user : { type : "string", index : "not_analyzed"},
content : { type : "string", index : "analyzed"},
postDate : { type : "date", format : "YYYYMMddHHmmSS", index :
"analyzed" },
tags : { type : "string", indexName : "tag", index : "not_analyzed" }
}
}
}
'
{"ok":true,"acknowledged":true}

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/post/3?*optype=create
*' -d '
{
post : {
user : "keteracel",
content : "This is my first blog post using elastic search as the
backing data store!

I hope you like it!

Paul.",
postDate : "20100315193410",
tags : ["blog","first","elastic search"]
}
}
'
{"ok":true,"_index":"blogs","_type":"post","_id":"3"}

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/post/3?*optype=create
*' -d '
{
post : {
user : "keteracel"
}
}
'
{"ok":true,"_index":"blogs","_type":"post","_id":"3"}

-bash-3.2$ curl -XGET 'http://localhost:9200/blogs/_status'

{"ok":true,"_shards":{"total":10,"successful":5,"failed":0},"indices":{"blogs":{"aliases":[],"settings":{"index.number_of_shards":"5","index.number_of_replicas":"1"},"store_size":"1.9k","store_size_in_bytes":1949,"estimated_flushable_memory_size":"382","estimated_flushable_memory_size_in_bytes":382,"translog_operations":2,
"docs":{"num_docs":1,"max_doc":2,"deleted_docs":1}
,"shards":{"0":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":0,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}],"1":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":1,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}],"2":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":2,"index":"blogs"},"state":"STARTED","store_size":"1.7k","store_size_in_bytes":1741,"estimated_flushable_memory_size":"382","estimated_flushable_memory_size_in_bytes":382,"translog_id":0,"translog_operations":2,"docs":{"num_docs":1,"max_doc":2,"deleted_docs":1}}],"3":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":3,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}],"4":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":4,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}]}}}}

--

Paul Loy
paul@keteracel.com
http://www.keteracel.com/


(Paul Loy) #3

Haha, how embarassing!

:smiley:

Thanks!

On Wed, May 26, 2010 at 11:05 AM, Shay Banon
shay.banon@elasticsearch.comwrote:

The parameter should be op_type, and no optype. Another option is to add
_create, for example: 'localhost:9200/index/type/id/_create'.

On Wed, May 26, 2010 at 12:53 PM, Paul Loy keteracel@gmail.com wrote:

Hi,

in the documentationhttp://www.elasticsearch.com/docs/elasticsearch/rest_api/index/#Operation_Typeit states that:

"If the op_type is set to create, and there is already an existing

document with the same type and id within the index, the index will now hold
two documents with the same id and type."

I do not see this in practice. If I do a _status on the index after
creating two documents with the same id and type it shows something like
this:

"docs":{"num_docs":1,"max_doc":2,"deleted_docs":1}

is the ?op_type=create being honoured? or is there some cleanup task?

Thanks,

Paul

PS, here are my test REST calls, using 0.7.1:

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/'
{"ok":true,"acknowledged":true}

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/post/_mapping' -d '
{
post : {
properties : {
user : { type : "string", index : "not_analyzed"},
content : { type : "string", index : "analyzed"},
postDate : { type : "date", format : "YYYYMMddHHmmSS", index :
"analyzed" },
tags : { type : "string", indexName : "tag", index : "not_analyzed" }
}
}
}
'
{"ok":true,"acknowledged":true}

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/post/3?*
optype=create*' -d '
{
post : {
user : "keteracel",
content : "This is my first blog post using elastic search as the
backing data store!

I hope you like it!

Paul.",
postDate : "20100315193410",
tags : ["blog","first","elastic search"]
}
}
'
{"ok":true,"_index":"blogs","_type":"post","_id":"3"}

-bash-3.2$ curl -XPUT 'http://127.0.0.1:9200/blogs/post/3?*
optype=create*' -d '
{
post : {
user : "keteracel"
}
}
'
{"ok":true,"_index":"blogs","_type":"post","_id":"3"}

-bash-3.2$ curl -XGET 'http://localhost:9200/blogs/_status'

{"ok":true,"_shards":{"total":10,"successful":5,"failed":0},"indices":{"blogs":{"aliases":[],"settings":{"index.number_of_shards":"5","index.number_of_replicas":"1"},"store_size":"1.9k","store_size_in_bytes":1949,"estimated_flushable_memory_size":"382","estimated_flushable_memory_size_in_bytes":382,"translog_operations":2,
"docs":{"num_docs":1,"max_doc":2,"deleted_docs":1}
,"shards":{"0":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":0,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}],"1":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":1,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}],"2":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":2,"index":"blogs"},"state":"STARTED","store_size":"1.7k","store_size_in_bytes":1741,"estimated_flushable_memory_size":"382","estimated_flushable_memory_size_in_bytes":382,"translog_id":0,"translog_operations":2,"docs":{"num_docs":1,"max_doc":2,"deleted_docs":1}}],"3":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":3,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}],"4":[{"routing":{"state":"STARTED","primary":true,"node":"f32e18a3-6ad1-4b33-bac1-b436d7f3b5a3","relocating_node":null,"shard":4,"index":"blogs"},"state":"STARTED","store_size":"52","store_size_in_bytes":52,"estimated_flushable_memory_size":"0","estimated_flushable_memory_size_in_bytes":0,"translog_id":0,"translog_operations":0,"docs":{"num_docs":0,"max_doc":0,"deleted_docs":0}}]}}}}

--

Paul Loy
paul@keteracel.com
http://www.keteracel.com/

--

Paul Loy
paul@keteracel.com
http://www.keteracel.com/paul


(system) #4