Mapping and Dynamic Mapping


(Slava G ) #1

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().create(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Shay Banon) #2

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Slava G ) #3

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Shay Banon) #4

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Slava G ) #5

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Shay Banon) #6

You can use dynamic templates to try and define which fields will be stored based on some matching scheme: http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html.

On Saturday, June 4, 2011 at 9:19 PM, slavag wrote:

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com) (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Slava G ) #7

Btw, when I used default-mapping.json and put there:
{
"TEST" : {
"_source" : {"enabled" : false}
}
}

It worked and all fields are stored by default, should it work in this
way ?

P.S. Still have
On Jun 4, 9:21 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You can use dynamic templates to try and define which fields will be stored based on some matching scheme:http://www.elasticsearch.org/guide/reference/mapping/root-object-type....

On Saturday, June 4, 2011 at 9:19 PM, slavag wrote:

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com) (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Shay Banon) #8

No it won't, all fields will not be stored by default.

On Saturday, June 4, 2011 at 9:25 PM, slavag wrote:

Btw, when I used default-mapping.json and put there:
{
"TEST" : {
"_source" : {"enabled" : false}
}
}

It worked and all fields are stored by default, should it work in this
way ?

P.S. Still have
On Jun 4, 9:21 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You can use dynamic templates to try and define which fields will be stored based on some matching scheme:http://www.elasticsearch.org/guide/reference/mapping/root-object-type....

On Saturday, June 4, 2011 at 9:19 PM, slavag wrote:

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com) (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Slava G ) #9

Strange, as I did this on clean elasticsearch and it work as I
described.

On Jun 4, 9:26 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

No it won't, all fields will not be stored by default.

On Saturday, June 4, 2011 at 9:25 PM, slavag wrote:

Btw, when I used default-mapping.json and put there:
{
"TEST" : {
"_source" : {"enabled" : false}
}
}

It worked and all fields are stored by default, should it work in this
way ?

P.S. Still have
On Jun 4, 9:21 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You can use dynamic templates to try and define which fields will be stored based on some matching scheme:http://www.elasticsearch.org/guide/reference/mapping/root-object-type....

On Saturday, June 4, 2011 at 9:19 PM, slavag wrote:

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com) (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Shay Banon) #10

No it won't.

On Saturday, June 4, 2011 at 9:27 PM, slavag wrote:

Strange, as I did this on clean elasticsearch and it work as I
described.

On Jun 4, 9:26 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

No it won't, all fields will not be stored by default.

On Saturday, June 4, 2011 at 9:25 PM, slavag wrote:

Btw, when I used default-mapping.json and put there:
{
"TEST" : {
"_source" : {"enabled" : false}
}
}

It worked and all fields are stored by default, should it work in this
way ?

P.S. Still have
On Jun 4, 9:21 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You can use dynamic templates to try and define which fields will be stored based on some matching scheme:http://www.elasticsearch.org/guide/reference/mapping/root-object-type....

On Saturday, June 4, 2011 at 9:19 PM, slavag wrote:

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com) (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Slava G ) #11

Ok, may be I'm wrong.

Last question, to make a compression on _source, should I also use
"static" mapping or default-mapping.json or there other way so tell to
elasticsearch to compress it ?

On Jun 4, 9:28 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

No it won't.

On Saturday, June 4, 2011 at 9:27 PM, slavag wrote:

Strange, as I did this on clean elasticsearch and it work as I
described.

On Jun 4, 9:26 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

No it won't, all fields will not be stored by default.

On Saturday, June 4, 2011 at 9:25 PM, slavag wrote:

Btw, when I used default-mapping.json and put there:
{
"TEST" : {
"_source" : {"enabled" : false}
}
}

It worked and all fields are stored by default, should it work in this
way ?

P.S. Still have
On Jun 4, 9:21 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You can use dynamic templates to try and define which fields will be stored based on some matching scheme:http://www.elasticsearch.org/guide/reference/mapping/root-object-type....

On Saturday, June 4, 2011 at 9:19 PM, slavag wrote:

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com) (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Shay Banon) #12

You can use the default-mapping.json option, but then you need to apply it to all nodes. Another option is to use index template, and have the default mapping type applied with compression to source (which is more of an API way to do it).

On Saturday, June 4, 2011 at 9:31 PM, slavag wrote:

Ok, may be I'm wrong.

Last question, to make a compression on _source, should I also use
"static" mapping or default-mapping.json or there other way so tell to
elasticsearch to compress it ?

On Jun 4, 9:28 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

No it won't.

On Saturday, June 4, 2011 at 9:27 PM, slavag wrote:

Strange, as I did this on clean elasticsearch and it work as I
described.

On Jun 4, 9:26 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

No it won't, all fields will not be stored by default.

On Saturday, June 4, 2011 at 9:25 PM, slavag wrote:

Btw, when I used default-mapping.json and put there:
{
"TEST" : {
"_source" : {"enabled" : false}
}
}

It worked and all fields are stored by default, should it work in this
way ?

P.S. Still have
On Jun 4, 9:21 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You can use dynamic templates to try and define which fields will be stored based on some matching scheme:http://www.elasticsearch.org/guide/reference/mapping/root-object-type....

On Saturday, June 4, 2011 at 9:19 PM, slavag wrote:

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com) (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Slava G ) #13

Thank You, where can I read about index template ?

On Jun 4, 9:32 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You can use the default-mapping.json option, but then you need to apply it to all nodes. Another option is to use index template, and have the default mapping type applied with compression to source (which is more of an API way to do it).

On Saturday, June 4, 2011 at 9:31 PM, slavag wrote:

Ok, may be I'm wrong.

Last question, to make a compression on _source, should I also use
"static" mapping or default-mapping.json or there other way so tell to
elasticsearch to compress it ?

On Jun 4, 9:28 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

No it won't.

On Saturday, June 4, 2011 at 9:27 PM, slavag wrote:

Strange, as I did this on clean elasticsearch and it work as I
described.

On Jun 4, 9:26 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

No it won't, all fields will not be stored by default.

On Saturday, June 4, 2011 at 9:25 PM, slavag wrote:

Btw, when I used default-mapping.json and put there:
{
"TEST" : {
"_source" : {"enabled" : false}
}
}

It worked and all fields are stored by default, should it work in this
way ?

P.S. Still have
On Jun 4, 9:21 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You can use dynamic templates to try and define which fields will be stored based on some matching scheme:http://www.elasticsearch.org/guide/reference/mapping/root-object-type....

On Saturday, June 4, 2011 at 9:19 PM, slavag wrote:

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com) (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(Shay Banon) #14

http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html

On Saturday, June 4, 2011 at 9:38 PM, slavag wrote:

Thank You, where can I read about index template ?

On Jun 4, 9:32 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You can use the default-mapping.json option, but then you need to apply it to all nodes. Another option is to use index template, and have the default mapping type applied with compression to source (which is more of an API way to do it).

On Saturday, June 4, 2011 at 9:31 PM, slavag wrote:

Ok, may be I'm wrong.

Last question, to make a compression on _source, should I also use
"static" mapping or default-mapping.json or there other way so tell to
elasticsearch to compress it ?

On Jun 4, 9:28 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

No it won't.

On Saturday, June 4, 2011 at 9:27 PM, slavag wrote:

Strange, as I did this on clean elasticsearch and it work as I
described.

On Jun 4, 9:26 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

No it won't, all fields will not be stored by default.

On Saturday, June 4, 2011 at 9:25 PM, slavag wrote:

Btw, when I used default-mapping.json and put there:
{
"TEST" : {
"_source" : {"enabled" : false}
}
}

It worked and all fields are stored by default, should it work in this
way ?

P.S. Still have
On Jun 4, 9:21 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You can use dynamic templates to try and define which fields will be stored based on some matching scheme:http://www.elasticsearch.org/guide/reference/mapping/root-object-type....

On Saturday, June 4, 2011 at 9:19 PM, slavag wrote:

That what I'm trying to understand, I read this section and still
confused, when I'm using such mapping (not dynamic) and then new
fields are cumming - it means I need explicitly to create a mapping
for them ? Or there other way to make mapping dynamic after once I
used static mapping (for _source).

Best Regards.

On Jun 4, 9:14 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

You need to set store to yes in hte mapping for the relevant fields you want to retrieve. Check the mappings section in the guide.

On Saturday, June 4, 2011 at 9:09 PM, slavag wrote:

Hi Shay, thank for a quick response,
What do you mean when saying to explicitly store specific fields , I'm
putting them into to JsonBuilder (in the search I do ask for them) ?

Best Regards.

On Jun 4, 7:47 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com) (http://elasticsearch.com)> wrote:

If you don't store the source, then you need to explicitly store specific fields (and ask for them in the search request) if you want them. It usually makes sense to use the _source field then storing specific fields separately.

On Saturday, June 4, 2011 at 5:55 PM, slavag wrote:

Hi,
I'm trying to disable _source field, using this:
XContentBuilder data = jsonBuilder().startObject()
.startObject("type")
.startObject("_source")
.field("enabled", false)
.endObject()
.endObject()
.endObject();

client.admin().indices().ccreate(new
CreateIndexRequest("index").mapping("type", data ));
And looks like _source is indeed not stored.

Then I'm doing index with:

XContentBuilder jsonBuilder = XContentFactory.jsonBuilder();
jsonBuilder.startObject();
jsonBuilder.field(FROMFIELD,addr.toString());
jsonBuilder.field(RECIPIENTFIELD,addr.toString());
jsonBuilder.field(SUBJECTFIELD,message.getSubject());
jsonBuilder.field("bckdate",backuptimstamp);
jsonBuilder.endObject();

IndexRequestBuilder indexRequest =
client.prepareIndex(this.tenant,this.type,id).setSource(jsonBuilder);
IndexResponse response = indexRequest.execute().actionGet();

The search result returns hits, without source field but as well
without any other field , even on search I'm adding those fields.

Looks like schema is not updated, the question after such a manual
mapping, should schema be updated on index with new fields ?

Thank You and Best Regards.


(system) #15