Documents vanishing

Elastic Stack Elasticsearch
1

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the document it
works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA
{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

I think the obvious question is: what is your mapping for sysMetrics types?
Might you try to add another field (or use an existing field)
particularly a non-analyzed field, retrieve the document by that, and
see what the document retrieved by this other has in it, particularly
for its ID.
Alternatively for debugging, retrieve everything, dump it and see what
you see in your index. Has the _id been converted in some way? How
about the other fields.

-Paul

On 5/4/2013 6:42 PM, Wojons Tech wrote:

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the
document it works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA
{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

When I issue a get-by-id request for an existing document in my "test"
index, I get:

$ curl http://localhost:9200/test/subscriber/7214560007
{"_index":"test","_type":"subscriber","_id":"7214560007","_version":1,"exists":true,
"_source" : { "telno" : "7214560007", "name" : "FXE", "o" : "FLL Executive
Airport", "city" : "Ft. Lauderdale", "state" : "FL" }}

And when I issue another get-by-id request for an _id that I know to not
exist:

$ curl http://localhost:9200/test/subscriber/9999999999
ERROR: HTTP response status: 404

I don't recall any documentation of the meaning of "exists":false (a lot
has been going on and it's likely I've missed it), but empiracally it
doesn't seem to apply to the document itself.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the document it
works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Hey,

how are you indexing your data? Maybe using a TTL? Are you using routing
maybe? These are wild guesses, because I do not know your setup.
Without more information it is hard to help out. Can you recreate your
issue by using curl calls, so we can reproduce it?

--Alex

On Sun, May 5, 2013 at 3:42 AM, Wojons Tech wojonstech@gmail.com wrote:

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the document it
works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

This is the mapping https://gist.github.com/wojons/d821cc639ec52890b3ec

On Sat, May 11, 2013 at 4:57 AM, Alexander Reelsen alr@spinscale.de wrote:

Hey,

how are you indexing your data? Maybe using a TTL? Are you using routing
maybe? These are wild guesses, because I do not know your setup.
Without more information it is hard to help out. Can you recreate your
issue by using curl calls, so we can reproduce it?

--Alex

On Sun, May 5, 2013 at 3:42 AM, Wojons Tech wojonstech@gmail.com wrote:

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the document it
works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Your previous post showed the TTL enabled mapping, but your gist's mapping
(link below) does not.

Also, you need to check your documents to see where _ttl is specified. Here
is the subset of my initial testing. The elf type has TTL enabled but there
is no default (documents without a TTL are immortal). Only the third
document below is mortal (finite TTL). Note that the TTL is in this bulk
input stream's action_and_meta_data and not in the source:

{"create":{"_index":"mortal","_type":"elf","_id":"1"}}
{ "cn" : "Celeborn" , "text" : [ "Lives forever" ] }

{"create":{"_index":"mortal","_type":"elf","_id":"2"}}
{ "cn" : "Galadriel" , "text" : [ "Lives forever" ] }

{"create":{"_index":"mortal","_type":"elf","_id":"3","_ttl":"3m"}}
{ "cn" : "Haldir of Lórien" , "text" : [ "Dies in the battle of Helm's
Deep", "Gone in 3m" ] }

One other behavior: If your mapping does not enable TTL, then any _ttl you
add to a document seems to go into the bit bucket and does not come back in
a response. The _ttl only comes back as a field in this case when it's
added to the _source; but in that case it isn't a _ttl that Elasticsearch
knows about.

Brian

On Saturday, May 11, 2013 8:01:18 AM UTC-4, Wojons Tech wrote:

This is the mapping https://gist.github.com/wojons/d821cc639ec52890b3ec

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

i am very sure that i am do not have any ttl on.

On Sat, May 11, 2013 at 6:58 AM, InquiringMind brian.from.fl@gmail.comwrote:

Your previous post showed the TTL enabled mapping, but your gist's mapping
(link below) does not.

Also, you need to check your documents to see where _ttl is specified.
Here is the subset of my initial testing. The elf type has TTL enabled but
there is no default (documents without a TTL are immortal). Only the third
document below is mortal (finite TTL). Note that the TTL is in this bulk
input stream's action_and_meta_data and not in the source:

{"create":{"_index":"mortal","_type":"elf","_id":"1"}}
{ "cn" : "Celeborn" , "text" : [ "Lives forever" ] }

{"create":{"_index":"mortal","_type":"elf","_id":"2"}}
{ "cn" : "Galadriel" , "text" : [ "Lives forever" ] }

{"create":{"_index":"mortal","_type":"elf","_id":"3","_ttl":"3m"}}
{ "cn" : "Haldir of Lórien" , "text" : [ "Dies in the battle of Helm's
Deep", "Gone in 3m" ] }

One other behavior: If your mapping does not enable TTL, then any _ttl you
add to a document seems to go into the bit bucket and does not come back in
a response. The _ttl only comes back as a field in this case when it's
added to the _source; but in that case it isn't a _ttl that Elasticsearch
knows about.

Brian

On Saturday, May 11, 2013 8:01:18 AM UTC-4, Wojons Tech wrote:

This is the mapping https://gist.github.com/wojons/
d821cc639ec52890b3echttps://gist.github.com/wojons/d821cc639ec52890b3ec

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

I looked at the Javadoc for the GetResponse.isExists method and the related
getSourceAsBytes method. It seems to tell me that your response that
contains "exists":false means that the source of the document doesn't
exist. But if there is an index.type.id that is found then what is that?
Elasticsearch's terminology is confusing me here; the source might not
exist but the index, type, and _id exist or else you'd get an HTTP 404
status. So what is the index, type, and _id if it's not a document? Or is
the term "document" loosely equated to "document source". Here is the
relevant Javadoc:

isExists

public boolean isExists()
Does the document exists.

getSourceAsBytes

public byte getSourceAsBytes()
The source of the document if exists.

isSourceEmpty

public boolean isSourceEmpty()
Is the source empty (not available) or not.

One downside to documentation after the code is written is that the code
becomes difficult to explain. I've never ever created a source-less
document nor have I seen the issue you are seeing, so I don't know all of
the corner cases that I would normally use to recreate my own documentation
of how things work.

Perhaps somewhere in your load you are creating or updating the document
with the meta data but not providing a source. That's the only thing I can
think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the document it
works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

Or, you are providing a source but somehow not storing it. Then your get
request returns with the source not existing, but your normal queries
return fields because the fields are stored and asked for, but the source
is not stored.

Again, just some thoughts. I don't store fields: I always store the full
source and parse it using the supplied Jackson stream parser (works well,
and helps with updates too).

Brian

On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the document
with the meta data but not providing a source. That's the only thing I can
think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the document it
works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

10

Is there anything else i could provide that would help find what the exact
issue is?

On Sat, May 11, 2013 at 7:41 PM, InquiringMind brian.from.fl@gmail.comwrote:

Or, you are providing a source but somehow not storing it. Then your get
request returns with the source not existing, but your normal queries
return fields because the fields are stored and asked for, but the source
is not stored.

Again, just some thoughts. I don't store fields: I always store the full
source and parse it using the supplied Jackson stream parser (works well,
and helps with updates too).

Brian

On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the document
with the meta data but not providing a source. That's the only thing I can
think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the document it
works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/**sysMetrics/t2QjFSAfQjSJRPv-**J8neeAhttp://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA
{"_index":"latest","_type":"sysMetrics","_id":"
t2QjFSAfQjSJRPv-J8neeA","**exists":false}

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

11

Hey,

Not sure if this is related but we had a similar problem before with parent child documents:
http://elasticsearch-users.115913.n3.nabble.com/Having-problem-retrieving-child-documents-with-their-IDs-td4031448.html

Pretty much get only looked at 1 shard (out of default 5) but search worked every time...

D

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

12

Yes, a possibility for us to recreate your problem locally (using curl)
would be great in order to find out if it is a bug or not.

--Alex

On Sun, May 12, 2013 at 4:53 AM, wojonstech wojonstech@gmail.com wrote:

Is there anything else i could provide that would help find what the exact
issue is?

On Sat, May 11, 2013 at 7:41 PM, InquiringMind brian.from.fl@gmail.comwrote:

Or, you are providing a source but somehow not storing it. Then your get
request returns with the source not existing, but your normal queries
return fields because the fields are stored and asked for, but the source
is not stored.

Again, just some thoughts. I don't store fields: I always store the full
source and parse it using the supplied Jackson stream parser (works well,
and helps with updates too).

Brian

On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the document
with the meta data but not providing a source. That's the only thing I can
think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the document it
works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/**sysMetrics/t2QjFSAfQjSJRPv-**J8neeAhttp://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA
{"_index":"latest","_type":"sysMetrics","_id":"
t2QjFSAfQjSJRPv-J8neeA","**exists":false}

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

13

just wanted to let everyone know that i found the answer.

https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/XUK2gUr6ve8

i was writin the documents with routing and not when i did the pulling. for
now i have disabled routing. how does everyone feel about routing.

On Sun, May 12, 2013 at 12:56 AM, Alexander Reelsen alr@spinscale.dewrote:

Yes, a possibility for us to recreate your problem locally (using curl)
would be great in order to find out if it is a bug or not.

--Alex

On Sun, May 12, 2013 at 4:53 AM, wojonstech wojonstech@gmail.com wrote:

Is there anything else i could provide that would help find what the
exact issue is?

On Sat, May 11, 2013 at 7:41 PM, InquiringMind brian.from.fl@gmail.comwrote:

Or, you are providing a source but somehow not storing it. Then your get
request returns with the source not existing, but your normal queries
return fields because the fields are stored and asked for, but the source
is not stored.

Again, just some thoughts. I don't store fields: I always store the full
source and parse it using the supplied Jackson stream parser (works well,
and helps with updates too).

Brian

On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the
document with the meta data but not providing a source. That's the only
thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:

I having a werid issue where I feel like i am loosing documents, a get
request fails to get the document but if i do a search for the document it
works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/**sysMetrics/t2QjFSAfQjSJRPv-**
J8neeAhttp://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA
{"_index":"latest","_type":"sysMetrics","_id":"
t2QjFSAfQjSJRPv-J8neeA","**exists":false}

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

14

phew! thanks for bringing closure to this!

simon

On Monday, May 20, 2013 10:21:49 AM UTC+2, Wojons Tech wrote:

just wanted to let everyone know that i found the answer.

Redirecting to Google Groups

i was writin the documents with routing and not when i did the pulling.
for now i have disabled routing. how does everyone feel about routing.

On Sun, May 12, 2013 at 12:56 AM, Alexander Reelsen <a...@spinscale.de<javascript:>

wrote:

Yes, a possibility for us to recreate your problem locally (using curl)
would be great in order to find out if it is a bug or not.

--Alex

On Sun, May 12, 2013 at 4:53 AM, wojonstech <wojon...@gmail.com<javascript:>

wrote:

Is there anything else i could provide that would help find what the
exact issue is?

On Sat, May 11, 2013 at 7:41 PM, InquiringMind <brian....@gmail.com<javascript:>

wrote:

Or, you are providing a source but somehow not storing it. Then your
get request returns with the source not existing, but your normal queries
return fields because the fields are stored and asked for, but the source
is not stored.

Again, just some thoughts. I don't store fields: I always store the
full source and parse it using the supplied Jackson stream parser (works
well, and helps with updates too).

Brian

On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the
document with the meta data but not providing a source. That's the only
thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:

I having a werid issue where I feel like i am loosing documents, a
get request fails to get the document but if i do a search for the document
it works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/**sysMetrics/t2QjFSAfQjSJRPv-**
J8neeAhttp://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA
{"_index":"latest","_type":"sysMetrics","_id":"
t2QjFSAfQjSJRPv-J8neeA","**exists":false}

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.

For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

15

for all the known elasticsearch masters that may ever find this. what i am
guessing is the issue is the _id for this document would normally route to
a selective shard. based on some hash distrobution. but since i had custom
routing i would see the documents when the document would have ended up on
the shard it got routed to based on the _id anyway.

Searches were not effected because they were not using routing and calling
all nodes.

On Mon, May 20, 2013 at 4:22 AM, simonw
simon.willnauer@elasticsearch.comwrote:

phew! thanks for bringing closure to this!

simon

On Monday, May 20, 2013 10:21:49 AM UTC+2, Wojons Tech wrote:

just wanted to let everyone know that i found the answer.

https://groups.google.com/**forum/?fromgroups#!topic/**
elasticsearch/XUK2gUr6ve8https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/XUK2gUr6ve8

i was writin the documents with routing and not when i did the pulling.
for now i have disabled routing. how does everyone feel about routing.

On Sun, May 12, 2013 at 12:56 AM, Alexander Reelsen a...@spinscale.dewrote:

Yes, a possibility for us to recreate your problem locally (using curl)
would be great in order to find out if it is a bug or not.

--Alex

On Sun, May 12, 2013 at 4:53 AM, wojonstech wojon...@gmail.com wrote:

Is there anything else i could provide that would help find what the
exact issue is?

On Sat, May 11, 2013 at 7:41 PM, InquiringMind brian....@gmail.comwrote:

Or, you are providing a source but somehow not storing it. Then your
get request returns with the source not existing, but your normal queries
return fields because the fields are stored and asked for, but the source
is not stored.

Again, just some thoughts. I don't store fields: I always store the
full source and parse it using the supplied Jackson stream parser (works
well, and helps with updates too).

Brian

On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the
document with the meta data but not providing a source. That's the only
thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:

I having a werid issue where I feel like i am loosing documents, a
get request fails to get the document but if i do a search for the document
it works just fine. So this is the output of a document write

Array
(
[ok] => 1
[_index] => latest
[_type] => sysMetrics
[_id] => t2QjFSAfQjSJRPv-J8neeA
[_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/**s**ysMetrics/t2QjFSAfQjSJRPv-**
J8ne**eAhttp://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA
{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJR
Pv-J8neeA","**exists":false}

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/**
topic/elasticsearch/8diEvl-**OoRc/unsubscribe?hl=en-UShttps://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/**
topic/elasticsearch/8diEvl-**OoRc/unsubscribe?hl=en-UShttps://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.