Source only available after a node restart

Elastic Stack Elasticsearch
1

All,

I am indexing data with the following mapping:

    {
        "typeName" : {
            "properties" : {
                "fullName" : {
                    "type" : "multi_field",
                    "fields":{
                         "fullName" : {"type" : "string",

"index" : "analyzed"},
"untouched" : {"type" : "string",
"index" : "not_analyzed"}
}
}
}
}
}

And indexing it by doing the following (in groovy):

public void simpleIndex(indexId, items){
    if(items.size() > 0){
        BulkRequestBuilder bulkRequestBuilder =

elasticSearchClient.prepareBulk()
items.each { item ->
IndexRequestBuilder indexRequestBuilder =
elasticSearchClient.prepareIndex(indexId, "typeName")
indexRequestBuilder.setId(item.id.toString())
indexRequestBuilder.setSource("fullName",
item.getFullName())
bulkRequestBuilder.add(indexRequestBuilder)
}
BulkResponse response =
bulkRequestBuilder.execute().actionGet()
if (response.hasFailures()) {
log.error("Error indexing: $
{response.buildFailureMessage()}")
}
}
}

This isn't the only type that's in the index. Although, there are
significantly less numbers of this type in the index than the other.
To keep this issue simple, the following code snippet will not pass:

GetResponse getResponse = client.prepareGet(indexName, "typeName",
"1").execute().actionGet()
assert getResponse.exists()
assert getResponse.id == "1"
assert getResponse.fields.size() > 0

It won't pass because fields is empty. However, the id is there and is
correct. The same issue will happen when performing a search as well.
I will get hits back, but no fields. So far, the only thing that fixes
this is restarting the node. When the node comes back up, the fields
will be there. I have tried a refresh, flush, and cache clear, all
with no affect. The issue is the same when using both node and
transport. I have also tried ditching the mapping for fear that the
multi-field type was causing issues (even though it works elsewhere
just fine) with no luck. Am I missing something?

Thanks in advance.

2

You will only get back fields if you explicitly specify them as stored and ask for them in the Get request. Do you get the _source back? GetResponse#source...

On Tuesday, July 12, 2011 at 3:45 AM, Lucas wrote:

All,

I am indexing data with the following mapping:

{
"typeName" : {
"properties" : {
"fullName" : {
"type" : "multi_field",
"fields":{
"fullName" : {"type" : "string",
"index" : "analyzed"},
"untouched" : {"type" : "string",
"index" : "not_analyzed"}
}
}
}
}
}

And indexing it by doing the following (in groovy):

public void simpleIndex(indexId, items){
if(items.size() > 0){
BulkRequestBuilder bulkRequestBuilder =
elasticSearchClient.prepareBulk()
items.each { item ->
IndexRequestBuilder indexRequestBuilder =
elasticSearchClient.prepareIndex(indexId, "typeName")
indexRequestBuilder.setId(item.id.toString())
indexRequestBuilder.setSource("fullName",
item.getFullName())
bulkRequestBuilder.add(indexRequestBuilder)
}
BulkResponse response =
bulkRequestBuilder.execute().actionGet()
if (response.hasFailures()) {
log.error("Error indexing: $
{response.buildFailureMessage()}")
}
}
}

This isn't the only type that's in the index. Although, there are
significantly less numbers of this type in the index than the other.
To keep this issue simple, the following code snippet will not pass:

GetResponse getResponse = client.prepareGet(indexName, "typeName",
"1").execute().actionGet()
assert getResponse.exists()
assert getResponse.id == "1"
assert getResponse.fields.size() > 0

It won't pass because fields is empty. However, the id is there and is
correct. The same issue will happen when performing a search as well.
I will get hits back, but no fields. So far, the only thing that fixes
this is restarting the node. When the node comes back up, the fields
will be there. I have tried a refresh, flush, and cache clear, all
with no affect. The issue is the same when using both node and
transport. I have also tried ditching the mapping for fear that the
multi-field type was causing issues (even though it works elsewhere
just fine) with no luck. Am I missing something?

Thanks in advance.

3

Shay,

I do request the fields back in search, but was using a GetRequest as
easier to express example. There is no _source that comes back in the
get. However, when I add the field as you suggest it does work.
However, the search still has an issue despite expressing the field.
However, I think I found the issue, which is something else I've
noticed. Essentially, I have about 4 types in my index. One of those
types has a property named, let's say 'widget'. Then I also have a
type which has the name 'widget' in the index. Even though my search
is restricting types, I will have to prepend the fieldname with the
type name. For example, setField('widget.fullName') It's not really an
issue, but just not something that's documented. I also don't
understand why restarting the node removed the need to do it (but only
in one case, which was strange). I would imagine it's an edge case
though, and field names with the same name as a type in the index
doesn't come up for a lot of people.

Lucas

On Jul 11, 7:47 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You will only get back fields if you explicitly specify them as stored and ask for them in the Get request. Do you get the _source back? GetResponse#source...

On Tuesday, July 12, 2011 at 3:45 AM, Lucas wrote:

All,

I am indexing data with the following mapping:

{
"typeName" : {
"properties" : {
"fullName" : {
"type" : "multi_field",
"fields":{
"fullName" : {"type" : "string",
"index" : "analyzed"},
"untouched" : {"type" : "string",
"index" : "not_analyzed"}
}
}
}
}
}

And indexing it by doing the following (in groovy):

public void simpleIndex(indexId, items){
if(items.size() > 0){
BulkRequestBuilder bulkRequestBuilder =
elasticSearchClient.prepareBulk()
items.each { item ->
IndexRequestBuilder indexRequestBuilder =
elasticSearchClient.prepareIndex(indexId, "typeName")
indexRequestBuilder.setId(item.id.toString())
indexRequestBuilder.setSource("fullName",
item.getFullName())
bulkRequestBuilder.add(indexRequestBuilder)
}
BulkResponse response =
bulkRequestBuilder.execute().actionGet()
if (response.hasFailures()) {
log.error("Error indexing: $
{response.buildFailureMessage()}")
}
}
}

This isn't the only type that's in the index. Although, there are
significantly less numbers of this type in the index than the other.
To keep this issue simple, the following code snippet will not pass:

GetResponse getResponse = client.prepareGet(indexName, "typeName",
"1").execute().actionGet()
assert getResponse.exists()
assert getResponse.id == "1"
assert getResponse.fields.size() > 0

It won't pass because fields is empty. However, the id is there and is
correct. The same issue will happen when performing a search as well.
I will get hits back, but no fields. So far, the only thing that fixes
this is restarting the node. When the node comes back up, the fields
will be there. I have tried a refresh, flush, and cache clear, all
with no affect. The issue is the same when using both node and
transport. I have also tried ditching the mapping for fear that the
multi-field type was causing issues (even though it works elsewhere
just fine) with no luck. Am I missing something?

Thanks in advance.