Store only Nested Document

Hi All,

I'm trying to create a document with nested fields(with '_source' disabled
and 'store' enabled for the nested column 'info') as shown in the mapping
below:

MAPPING:
{

"record" : {

"_source" : {
"enabled" : false
},
"properties" : {
"column1" : {"type" : "string"},
"column2" : {"type" : "string"},
"info" : {
"type" : "nested",
"store":"yes"
}
}
}
}

Here is a sample of a document I inserted.

RECORD:

{
"column1": "192.168.41.105",
"column2":"some test value",
"info":
{
"seqno" : "7634786",
"bookmark": 10,
"users":[
{
"id":"user1",
"comments":[
{
"comment":"Test Comment 1"
},
{
"comment":"Test Comment 2"
},
{
"comment":"Test Comment 3"
}

             ]
      },
      {
        "id":"user2",
        "comments":[
            {
              "comment":"Test Comment 4"
            },
            {
              "comment":"Test Comment 5"
            },
            {
              "comment":"Test Comment 6"
            }

             ]
      }
    ],
  "packages":[
      {
        "id":"1608",
        "packageresourcestatus":2
      }
    ]
}

}

Now I want to retrieve the nested field properties like:

info.users.comments.comment

but I am not able to do so with '_source' disabled although I have enabled
'store' for the nested field 'info'.

  1. Is there a way to do this? I tried using '_source' include/exclude paths
    but it was returning the whole '_source'.

The code I am trying to retrieve the nested fields is given below:

NestedQueryBuilder nestedQueryBuilder = new NestedQueryBuilder("info",
termQuery("comment", "Test Comment 1") );

        SearchRequestBuilder searchRequestBuilder = client
                        .prepareSearch("test")
                        .setSearchType(SearchType.QUERY_THEN_FETCH)
                        .setQuery(nestedQueryBuilder)
                        .addField("info.bookmark")
                        .addField("info.users")
                        .addField("info.packages")
                        .addField("info.users.id")
                        .addField("info.users.comments")
                        .addField("info.users.comments.comment");
        SearchResponse response = searchRequestBuilder.execute().

actionGet();
SearchHits hits = response.getHits();
System.out.println("Total Hits : " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println("id = " + hit.getId() + "----" + hit.
getFields().get("info.users.comments.comment").getValue());
}

OUTPUT:

Total Hits : 1
Exception in thread "main" java.lang.NullPointerException

I'm getting the null pointer here: hit.getFields().get(
"info.users.comments.comment").getValue()

I can not enable source of the entire document but if there's a way which
lets me only enable the storing of the info field i can do that.

Thanks for your help.

Saurabh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

My assumption (not tested) is that the "store" setting is not "inherited"
by fields of the nested document. Each individual field must be declared as
stored. You can try using a dynamic template, although I have never tried
using a match on a nested field name: "match" : "info.*"

--
Ivan

On Wed, May 8, 2013 at 8:11 AM, Saurabh saurabh113@gmail.com wrote:

Hi All,

I'm trying to create a document with nested fields(with '_source' disabled
and 'store' enabled for the nested column 'info') as shown in the mapping
below:

MAPPING:
{

"record" : {

"_source" : {
"enabled" : false
},
"properties" : {
"column1" : {"type" : "string"},
"column2" : {"type" : "string"},
"info" : {
"type" : "nested",
"store":"yes"
}
}
}
}

Here is a sample of a document I inserted.

RECORD:

{
"column1": "192.168.41.105",
"column2":"some test value",
"info":
{
"seqno" : "7634786",
"bookmark": 10,
"users":[
{
"id":"user1",
"comments":[
{
"comment":"Test Comment 1"
},
{
"comment":"Test Comment 2"
},
{
"comment":"Test Comment 3"
}

             ]
      },
      {
        "id":"user2",
        "comments":[
            {
              "comment":"Test Comment 4"
            },
            {
              "comment":"Test Comment 5"
            },
            {
              "comment":"Test Comment 6"
            }

             ]
      }
    ],
  "packages":[
      {
        "id":"1608",
        "packageresourcestatus":2
      }
    ]
}

}

Now I want to retrieve the nested field properties like:

info.users.comments.comment

but I am not able to do so with '_source' disabled although I have enabled
'store' for the nested field 'info'.

  1. Is there a way to do this? I tried using '_source' include/exclude
    paths but it was returning the whole '_source'.

The code I am trying to retrieve the nested fields is given below:

NestedQueryBuilder nestedQueryBuilder = new NestedQueryBuilder("info",
termQuery("comment", "Test Comment 1") );

        SearchRequestBuilder searchRequestBuilder = client
                        .prepareSearch("test")
                        .setSearchType(SearchType.QUERY_THEN_FETCH)
                        .setQuery(nestedQueryBuilder)
                        .addField("info.bookmark")
                        .addField("info.users")
                        .addField("info.packages")
                        .addField("info.users.id")
                        .addField("info.users.comments")
                        .addField("info.users.comments.comment");
        SearchResponse response = searchRequestBuilder.execute().

actionGet();
SearchHits hits = response.getHits();
System.out.println("Total Hits : " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println("id = " + hit.getId() + "----" +hit
.getFields().get("info.users.comments.comment").getValue());
}

OUTPUT:

Total Hits : 1
Exception in thread "main" java.lang.NullPointerException

I'm getting the null pointer here: hit.getFields().get(
"info.users.comments.comment").getValue()

I can not enable source of the entire document but if there's a way which
lets me only enable the storing of the info field i can do that.

Thanks for your help.

Saurabh

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Dynamic do indeed work with nested documents using path_match. Try adding
the following dynamic template to your mapping:

"dynamic_templates": [
{
"template_1": {
"path_match": "info.*",
"mapping": {
"store" : "yes"
}
}
}
]

Read more on dynamic templates here:

Cheers,

Ivan

On Wed, May 8, 2013 at 9:01 PM, Ivan Brusic ivan@brusic.com wrote:

My assumption (not tested) is that the "store" setting is not "inherited"
by fields of the nested document. Each individual field must be declared as
stored. You can try using a dynamic template, although I have never tried
using a match on a nested field name: "match" : "info.*"

--
Ivan

On Wed, May 8, 2013 at 8:11 AM, Saurabh saurabh113@gmail.com wrote:

Hi All,

I'm trying to create a document with nested fields(with '_source'
disabled and 'store' enabled for the nested column 'info') as shown in the
mapping below:

MAPPING:
{

"record" : {

"_source" : {
"enabled" : false
},
"properties" : {
"column1" : {"type" : "string"},
"column2" : {"type" : "string"},
"info" : {
"type" : "nested",
"store":"yes"
}
}
}
}

Here is a sample of a document I inserted.

RECORD:

{
"column1": "192.168.41.105",
"column2":"some test value",
"info":
{
"seqno" : "7634786",
"bookmark": 10,
"users":[
{
"id":"user1",
"comments":[
{
"comment":"Test Comment 1"
},
{
"comment":"Test Comment 2"
},
{
"comment":"Test Comment 3"
}

             ]
      },
      {
        "id":"user2",
        "comments":[
            {
              "comment":"Test Comment 4"
            },
            {
              "comment":"Test Comment 5"
            },
            {
              "comment":"Test Comment 6"
            }

             ]
      }
    ],
  "packages":[
      {
        "id":"1608",
        "packageresourcestatus":2
      }
    ]
}

}

Now I want to retrieve the nested field properties like:

info.users.comments.comment

but I am not able to do so with '_source' disabled although I have
enabled 'store' for the nested field 'info'.

  1. Is there a way to do this? I tried using '_source' include/exclude
    paths but it was returning the whole '_source'.

The code I am trying to retrieve the nested fields is given below:

NestedQueryBuilder nestedQueryBuilder = new NestedQueryBuilder("info",
termQuery("comment", "Test Comment 1") );

        SearchRequestBuilder searchRequestBuilder = client
                        .prepareSearch("test")
                        .setSearchType(SearchType.QUERY_THEN_FETCH)
                        .setQuery(nestedQueryBuilder)
                        .addField("info.bookmark")
                        .addField("info.users")
                        .addField("info.packages")
                        .addField("info.users.id")
                        .addField("info.users.comments")
                        .addField("info.users.comments.comment");
        SearchResponse response = searchRequestBuilder.execute().

actionGet();
SearchHits hits = response.getHits();
System.out.println("Total Hits : " + hits.getTotalHits());
for (SearchHit hit : hits) {
System.out.println("id = " + hit.getId() + "----" +hit
.getFields().get("info.users.comments.comment").getValue());
}

OUTPUT:

Total Hits : 1
Exception in thread "main" java.lang.NullPointerException

I'm getting the null pointer here: hit.getFields().get(
"info.users.comments.comment").getValue()

I can not enable source of the entire document but if there's a way which
lets me only enable the storing of the info field i can do that.

Thanks for your help.

Saurabh

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.