JsonFieldMapper used for default field within a multi_field element


(dominict) #1

Say I have the following mapping :

{
"stringtest" : {
" properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string" },
"name_untouched" : {type: "string"}
}
}
}
}
}

The JsonStringFieldMapper.indexedValue(String value) only gets called
for name_untouched, and not name. Should the JsonStringFieldMapper
used for both the default "name" and "name_untouched" fields?

Mapping setup via:

server.client().admin().indices().putMapping(putMappingRequest("test").mappingSource(mapping)).actionGet();

Document is indexed via:

server.client().index(indexRequest("test").type("stringtest").id("1")
.source(jsonBuilder().startObject().field("name",
"123").endObject())).actionGet();

server.client().admin().indices().refresh(refreshRequest()).actionGet();

then a count is done:

CountResponse countResponse =
client.count(countRequest("test").querySource(termQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =
client.count(countRequest("test").querySource(termQuery("name.name_name",
"123"))).actionGet();
CountResponse countResponse3 =
client.count(countRequest("test").querySource(termQuery("name.name_untouched",
"123"))).actionGet();
CountResponse countResponse4 =
client.count(countRequest("test").querySource(termQuery("name.name",
"123"))).actionGet();
CountResponse countResponse5 =
client.count(countRequest("test").querySource(termQuery("name.untouched",
"123"))).actionGet();

of the above 5 count operations only count 1 + 3 return the correct
count number of 1.

Should "name.untouched" also work?

It's probably a complete mis-reading of
http://www.elasticsearch.com/docs/elasticsearch/mapping/multi_field/
on my part.

cheers,
/dom


(Shay Banon) #2

You are being modest :). There is a bug in the docs (just fixed it). The
idea is that any mapping unit a multi_field that is not using the same name
as the multi_field can be accessed by navigating through the multi_field
name.

So, the docs are wrong, have a look at the corrected ones (the change is to
rename name_untouched to untouched). In your case, since the extra field(s)
is called name_untouched, then you navigate to it using name.name_untouched.
If you change it to untouched, then name.untouched should work.

cheers,
shay.banon

On Tue, Apr 6, 2010 at 10:11 PM, dominict dominic.tootell@gmail.com wrote:

Say I have the following mapping :

{
"stringtest" : {
" properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string" },
"name_untouched" : {type: "string"}
}
}
}
}
}

The JsonStringFieldMapper.indexedValue(String value) only gets called
for name_untouched, and not name. Should the JsonStringFieldMapper
used for both the default "name" and "name_untouched" fields?

Mapping setup via:

server.client().admin().indices().putMapping(putMappingRequest("test").mappingSource(mapping)).actionGet();

Document is indexed via:

server.client().index(indexRequest("test").type("stringtest").id("1")
.source(jsonBuilder().startObject().field("name",
"123").endObject())).actionGet();

server.client().admin().indices().refresh(refreshRequest()).actionGet();

then a count is done:

CountResponse countResponse =
client.count(countRequest("test").querySource(termQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =
client.count(countRequest("test").querySource(termQuery("name.name_name",
"123"))).actionGet();
CountResponse countResponse3 =

client.count(countRequest("test").querySource(termQuery("name.name_untouched",
"123"))).actionGet();
CountResponse countResponse4 =
client.count(countRequest("test").querySource(termQuery("name.name",
"123"))).actionGet();
CountResponse countResponse5 =
client.count(countRequest("test").querySource(termQuery("name.untouched",
"123"))).actionGet();

of the above 5 count operations only count 1 + 3 return the correct
count number of 1.

Should "name.untouched" also work?

It's probably a complete mis-reading of
http://www.elasticsearch.com/docs/elasticsearch/mapping/multi_field/
on my part.

cheers,
/dom


(dominict) #3

Cheers for the update shay,

I'm more curious about the JsonFieldMapper that is obtained and called
when searching for the default field. i.e. for the mapping:

{
"stringtest" : {
"properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string", index :
"analyzed" },
"orig" : { type : "string", index :
"analyzed" },
"untouched" : {type: "string", index :
"not_analyzed"}
}
}
}
}
}

CountResponse countResponse1 =
server.client().count(countRequest("test").querySource(fieldQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =
server.client().count(countRequest("test").querySource(fieldQuery("name.untouched",
"123"))).actionGet();
CountResponse countResponse3 =
server.client().count(countRequest("test").querySource(fieldQuery("name.orig",
"123"))).actionGet();

upon searching, the method JsonFieldMapper.indexedValue(String) is
called (to format the string for searching I guess). What I'm seeing
is the following:

name is assigned JsonLongFieldMapper
name.orig is assigned JsonStringFieldMapper
name.untouched is assigned JsonStringFieldMapper

Is this meant to happen, or should it be assigned
JsonStringFieldMapper. I'm just looking at plugin development in
0.6.0 (which is working well), and this is where my question is
stemming from.

cheers
/dom

On Apr 6, 11:33 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You are being modest :). There is a bug in the docs (just fixed it). The
idea is that any mapping unit a multi_field that is not using the same name
as the multi_field can be accessed by navigating through the multi_field
name.

So, the docs are wrong, have a look at the corrected ones (the change is to
rename name_untouched to untouched). In your case, since the extra field(s)
is called name_untouched, then you navigate to it using name.name_untouched.
If you change it to untouched, then name.untouched should work.

cheers,
shay.banon

On Tue, Apr 6, 2010 at 10:11 PM, dominict dominic.toot...@gmail.com wrote:

Say I have the following mapping :

{
"stringtest" : {
" properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string" },
"name_untouched" : {type: "string"}
}
}
}
}
}

The JsonStringFieldMapper.indexedValue(String value) only gets called
for name_untouched, and not name. Should the JsonStringFieldMapper
used for both the default "name" and "name_untouched" fields?

Mapping setup via:

server.client().admin().indices().putMapping(putMappingRequest("test").mapp ingSource(mapping)).actionGet();

Document is indexed via:

server.client().index(indexRequest("test").type("stringtest").id("1")
.source(jsonBuilder().startObject().field("name",
"123").endObject())).actionGet();

server.client().admin().indices().refresh(refreshRequest()).actionGet();

then a count is done:

CountResponse countResponse =
client.count(countRequest("test").querySource(termQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =
client.count(countRequest("test").querySource(termQuery("name.name_name",
"123"))).actionGet();
CountResponse countResponse3 =

client.count(countRequest("test").querySource(termQuery("name.name_untouche d",
"123"))).actionGet();
CountResponse countResponse4 =
client.count(countRequest("test").querySource(termQuery("name.name",
"123"))).actionGet();
CountResponse countResponse5 =
client.count(countRequest("test").querySource(termQuery("name.untouched",
"123"))).actionGet();

of the above 5 count operations only count 1 + 3 return the correct
count number of 1.

Should "name.untouched" also work?

It's probably a complete mis-reading of
http://www.elasticsearch.com/docs/elasticsearch/mapping/multi_field/
on my part.

cheers,
/dom


(Shay Banon) #4

No, it should not be a JsonLong one..., can you raise an issue and a
testcase? Would like to nail this for 0.6 if possible.

Just myself being the usual curious me, what plugin are you working on?

cheers,
shay.banon

On Wed, Apr 7, 2010 at 2:27 AM, dominict dominic.tootell@gmail.com wrote:

Cheers for the update shay,

I'm more curious about the JsonFieldMapper that is obtained and called
when searching for the default field. i.e. for the mapping:

{
"stringtest" : {
"properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string", index :
"analyzed" },
"orig" : { type : "string", index :
"analyzed" },
"untouched" : {type: "string", index :
"not_analyzed"}
}
}
}
}
}

CountResponse countResponse1 =
server.client().count(countRequest("test").querySource(fieldQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =

server.client().count(countRequest("test").querySource(fieldQuery("name.untouched",
"123"))).actionGet();
CountResponse countResponse3 =

server.client().count(countRequest("test").querySource(fieldQuery("name.orig",
"123"))).actionGet();

upon searching, the method JsonFieldMapper.indexedValue(String) is
called (to format the string for searching I guess). What I'm seeing
is the following:

name is assigned JsonLongFieldMapper
name.orig is assigned JsonStringFieldMapper
name.untouched is assigned JsonStringFieldMapper

Is this meant to happen, or should it be assigned
JsonStringFieldMapper. I'm just looking at plugin development in
0.6.0 (which is working well), and this is where my question is
stemming from.

cheers
/dom

On Apr 6, 11:33 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You are being modest :). There is a bug in the docs (just fixed it). The
idea is that any mapping unit a multi_field that is not using the same
name
as the multi_field can be accessed by navigating through the multi_field
name.

So, the docs are wrong, have a look at the corrected ones (the change is
to
rename name_untouched to untouched). In your case, since the extra
field(s)
is called name_untouched, then you navigate to it using
name.name_untouched.
If you change it to untouched, then name.untouched should work.

cheers,
shay.banon

On Tue, Apr 6, 2010 at 10:11 PM, dominict dominic.toot...@gmail.com
wrote:

Say I have the following mapping :

{
"stringtest" : {
" properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string" },
"name_untouched" : {type: "string"}
}
}
}
}
}

The JsonStringFieldMapper.indexedValue(String value) only gets called
for name_untouched, and not name. Should the JsonStringFieldMapper
used for both the default "name" and "name_untouched" fields?

Mapping setup via:

server.client().admin().indices().putMapping(putMappingRequest("test").mapp
ingSource(mapping)).actionGet();

Document is indexed via:

server.client().index(indexRequest("test").type("stringtest").id("1")
.source(jsonBuilder().startObject().field("name",
"123").endObject())).actionGet();

server.client().admin().indices().refresh(refreshRequest()).actionGet();

then a count is done:

CountResponse countResponse =
client.count(countRequest("test").querySource(termQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =

client.count(countRequest("test").querySource(termQuery("name.name_name",

"123"))).actionGet();
CountResponse countResponse3 =

client.count(countRequest("test").querySource(termQuery("name.name_untouche
d",

"123"))).actionGet();
CountResponse countResponse4 =
client.count(countRequest("test").querySource(termQuery("name.name",
"123"))).actionGet();
CountResponse countResponse5 =

client.count(countRequest("test").querySource(termQuery("name.untouched",

"123"))).actionGet();

of the above 5 count operations only count 1 + 3 return the correct
count number of 1.

Should "name.untouched" also work?

It's probably a complete mis-reading of
http://www.elasticsearch.com/docs/elasticsearch/mapping/multi_field/
on my part.

cheers,
/dom


(dominict) #5

I'll knock up a test case, and submit an issue.

The plugin I'm looking at is nothing special afraid, just using it as
a nice learning aid.
What I was looking at was the ability to store milliseconds in the
index, but search (and range search) using formatted dates.
I noticed some fields in documents that are being stored in a key/
value store at work, contain timestamps in milliseconds (not sure
why),
but thought it would be handy to be able to find all those docs with
timestamps from today (2010-04-06 to 2010-04-07) for example.
Not a great plugin, I know :slight_smile:

cheers
/dom

On Apr 7, 12:29 am, Shay Banon shay.ba...@elasticsearch.com wrote:

No, it should not be a JsonLong one..., can you raise an issue and a
testcase? Would like to nail this for 0.6 if possible.

Just myself being the usual curious me, what plugin are you working on?

cheers,
shay.banon

On Wed, Apr 7, 2010 at 2:27 AM, dominict dominic.toot...@gmail.com wrote:

Cheers for the update shay,

I'm more curious about the JsonFieldMapper that is obtained and called
when searching for the default field. i.e. for the mapping:

{
"stringtest" : {
"properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string", index :
"analyzed" },
"orig" : { type : "string", index :
"analyzed" },
"untouched" : {type: "string", index :
"not_analyzed"}
}
}
}
}
}

CountResponse countResponse1 =
server.client().count(countRequest("test").querySource(fieldQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =

server.client().count(countRequest("test").querySource(fieldQuery("name.unt ouched",
"123"))).actionGet();
CountResponse countResponse3 =

server.client().count(countRequest("test").querySource(fieldQuery("name.ori g",
"123"))).actionGet();

upon searching, the method JsonFieldMapper.indexedValue(String) is
called (to format the string for searching I guess). What I'm seeing
is the following:

name is assigned JsonLongFieldMapper
name.orig is assigned JsonStringFieldMapper
name.untouched is assigned JsonStringFieldMapper

Is this meant to happen, or should it be assigned
JsonStringFieldMapper. I'm just looking at plugin development in
0.6.0 (which is working well), and this is where my question is
stemming from.

cheers
/dom

On Apr 6, 11:33 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You are being modest :). There is a bug in the docs (just fixed it). The
idea is that any mapping unit a multi_field that is not using the same
name
as the multi_field can be accessed by navigating through the multi_field
name.

So, the docs are wrong, have a look at the corrected ones (the change is
to
rename name_untouched to untouched). In your case, since the extra
field(s)
is called name_untouched, then you navigate to it using
name.name_untouched.
If you change it to untouched, then name.untouched should work.

cheers,
shay.banon

On Tue, Apr 6, 2010 at 10:11 PM, dominict dominic.toot...@gmail.com
wrote:

Say I have the following mapping :

{
"stringtest" : {
" properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string" },
"name_untouched" : {type: "string"}
}
}
}
}
}

The JsonStringFieldMapper.indexedValue(String value) only gets called
for name_untouched, and not name. Should the JsonStringFieldMapper
used for both the default "name" and "name_untouched" fields?

Mapping setup via:

server.client().admin().indices().putMapping(putMappingRequest("test").mapp
ingSource(mapping)).actionGet();

Document is indexed via:

server.client().index(indexRequest("test").type("stringtest").id("1")
.source(jsonBuilder().startObject().field("name",
"123").endObject())).actionGet();

server.client().admin().indices().refresh(refreshRequest()).actionGet();

then a count is done:

CountResponse countResponse =
client.count(countRequest("test").querySource(termQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =

client.count(countRequest("test").querySource(termQuery("name.name_name",

"123"))).actionGet();
CountResponse countResponse3 =

client.count(countRequest("test").querySource(termQuery("name.name_untouche
d",

"123"))).actionGet();
CountResponse countResponse4 =
client.count(countRequest("test").querySource(termQuery("name.name",
"123"))).actionGet();
CountResponse countResponse5 =

client.count(countRequest("test").querySource(termQuery("name.untouched",

"123"))).actionGet();

of the above 5 count operations only count 1 + 3 return the correct
count number of 1.

Should "name.untouched" also work?

It's probably a complete mis-reading of
http://www.elasticsearch.com/docs/elasticsearch/mapping/multi_field/
on my part.

cheers,
/dom


(dominict) #6

Hi there,

It looks as though the only issue at the moment is between the chair
and keyboard (i.e. me).
And the only place I'm seeing issue, is with the plugin I'm working
on. So please consider it a non issue.
I'll comment back if I am ever able to reproduce, in a context other
that my own plugin. Apologies for taking up your time.
Cheers for the update to the docs on the multi_field.

/dom

On Apr 7, 12:57 am, dominict dominic.toot...@gmail.com wrote:

I'll knock up a test case, and submit an issue.

The plugin I'm looking at is nothing special afraid, just using it as
a nice learning aid.
What I was looking at was the ability to store milliseconds in the
index, but search (and range search) using formatted dates.
I noticed some fields in documents that are being stored in a key/
value store at work, contain timestamps in milliseconds (not sure
why),
but thought it would be handy to be able to find all those docs with
timestamps from today (2010-04-06 to 2010-04-07) for example.
Not a great plugin, I know :slight_smile:

cheers
/dom

On Apr 7, 12:29 am, Shay Banon shay.ba...@elasticsearch.com wrote:

No, it should not be a JsonLong one..., can you raise an issue and a
testcase? Would like to nail this for 0.6 if possible.

Just myself being the usual curious me, what plugin are you working on?

cheers,
shay.banon

On Wed, Apr 7, 2010 at 2:27 AM, dominict dominic.toot...@gmail.com wrote:

Cheers for the update shay,

I'm more curious about the JsonFieldMapper that is obtained and called
when searching for the default field. i.e. for the mapping:

{
"stringtest" : {
"properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string", index :
"analyzed" },
"orig" : { type : "string", index :
"analyzed" },
"untouched" : {type: "string", index :
"not_analyzed"}
}
}
}
}
}

CountResponse countResponse1 =
server.client().count(countRequest("test").querySource(fieldQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =

server.client().count(countRequest("test").querySource(fieldQuery("name.unt ouched",
"123"))).actionGet();
CountResponse countResponse3 =

server.client().count(countRequest("test").querySource(fieldQuery("name.ori g",
"123"))).actionGet();

upon searching, the method JsonFieldMapper.indexedValue(String) is
called (to format the string for searching I guess). What I'm seeing
is the following:

name is assigned JsonLongFieldMapper
name.orig is assigned JsonStringFieldMapper
name.untouched is assigned JsonStringFieldMapper

Is this meant to happen, or should it be assigned
JsonStringFieldMapper. I'm just looking at plugin development in
0.6.0 (which is working well), and this is where my question is
stemming from.

cheers
/dom

On Apr 6, 11:33 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You are being modest :). There is a bug in the docs (just fixed it). The
idea is that any mapping unit a multi_field that is not using the same
name
as the multi_field can be accessed by navigating through the multi_field
name.

So, the docs are wrong, have a look at the corrected ones (the change is
to
rename name_untouched to untouched). In your case, since the extra
field(s)
is called name_untouched, then you navigate to it using
name.name_untouched.
If you change it to untouched, then name.untouched should work.

cheers,
shay.banon

On Tue, Apr 6, 2010 at 10:11 PM, dominict dominic.toot...@gmail.com
wrote:

Say I have the following mapping :

{
"stringtest" : {
" properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string" },
"name_untouched" : {type: "string"}
}
}
}
}
}

The JsonStringFieldMapper.indexedValue(String value) only gets called
for name_untouched, and not name. Should the JsonStringFieldMapper
used for both the default "name" and "name_untouched" fields?

Mapping setup via:

server.client().admin().indices().putMapping(putMappingRequest("test").mapp
ingSource(mapping)).actionGet();

Document is indexed via:

server.client().index(indexRequest("test").type("stringtest").id("1")
.source(jsonBuilder().startObject().field("name",
"123").endObject())).actionGet();

server.client().admin().indices().refresh(refreshRequest()).actionGet();

then a count is done:

CountResponse countResponse =
client.count(countRequest("test").querySource(termQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =

client.count(countRequest("test").querySource(termQuery("name.name_name",

"123"))).actionGet();
CountResponse countResponse3 =

client.count(countRequest("test").querySource(termQuery("name.name_untouche
d",

"123"))).actionGet();
CountResponse countResponse4 =
client.count(countRequest("test").querySource(termQuery("name.name",
"123"))).actionGet();
CountResponse countResponse5 =

client.count(countRequest("test").querySource(termQuery("name.untouched",

"123"))).actionGet();

of the above 5 count operations only count 1 + 3 return the correct
count number of 1.

Should "name.untouched" also work?

It's probably a complete mis-reading of
http://www.elasticsearch.com/docs/elasticsearch/mapping/multi_field/
on my part.

cheers,
/dom


(Shay Banon) #7

What you are working on is actually a very valid feature, and I was thinking
about just that the other day. It should be simple to allow for the date
type of handle long as well as strings (that represent the date). What I was
not so sure about is what it should return if stores (simplest I was
thinking about it formatted dates), and what should be stored in the _all
field?

cheers,
shay.banon

On Wed, Apr 7, 2010 at 2:57 AM, dominict dominic.tootell@gmail.com wrote:

I'll knock up a test case, and submit an issue.

The plugin I'm looking at is nothing special afraid, just using it as
a nice learning aid.
What I was looking at was the ability to store milliseconds in the
index, but search (and range search) using formatted dates.
I noticed some fields in documents that are being stored in a key/
value store at work, contain timestamps in milliseconds (not sure
why),
but thought it would be handy to be able to find all those docs with
timestamps from today (2010-04-06 to 2010-04-07) for example.
Not a great plugin, I know :slight_smile:

cheers
/dom

On Apr 7, 12:29 am, Shay Banon shay.ba...@elasticsearch.com wrote:

No, it should not be a JsonLong one..., can you raise an issue and a
testcase? Would like to nail this for 0.6 if possible.

Just myself being the usual curious me, what plugin are you working on?

cheers,
shay.banon

On Wed, Apr 7, 2010 at 2:27 AM, dominict dominic.toot...@gmail.com
wrote:

Cheers for the update shay,

I'm more curious about the JsonFieldMapper that is obtained and called
when searching for the default field. i.e. for the mapping:

{
"stringtest" : {
"properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string", index :
"analyzed" },
"orig" : { type : "string", index :
"analyzed" },
"untouched" : {type: "string", index :
"not_analyzed"}
}
}
}
}
}

CountResponse countResponse1 =

server.client().count(countRequest("test").querySource(fieldQuery("name",

"123"))).actionGet();
CountResponse countResponse2 =

server.client().count(countRequest("test").querySource(fieldQuery("name.unt
ouched",

"123"))).actionGet();
CountResponse countResponse3 =

server.client().count(countRequest("test").querySource(fieldQuery("name.ori
g",

"123"))).actionGet();

upon searching, the method JsonFieldMapper.indexedValue(String) is
called (to format the string for searching I guess). What I'm seeing
is the following:

name is assigned JsonLongFieldMapper
name.orig is assigned JsonStringFieldMapper
name.untouched is assigned JsonStringFieldMapper

Is this meant to happen, or should it be assigned
JsonStringFieldMapper. I'm just looking at plugin development in
0.6.0 (which is working well), and this is where my question is
stemming from.

cheers
/dom

On Apr 6, 11:33 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You are being modest :). There is a bug in the docs (just fixed it).
The

idea is that any mapping unit a multi_field that is not using the
same

name

as the multi_field can be accessed by navigating through the
multi_field

name.

So, the docs are wrong, have a look at the corrected ones (the change
is

to

rename name_untouched to untouched). In your case, since the extra
field(s)
is called name_untouched, then you navigate to it using
name.name_untouched.
If you change it to untouched, then name.untouched should work.

cheers,
shay.banon

On Tue, Apr 6, 2010 at 10:11 PM, dominict <dominic.toot...@gmail.com

wrote:

Say I have the following mapping :

{
"stringtest" : {
" properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string" },
"name_untouched" : {type: "string"}
}
}
}
}
}

The JsonStringFieldMapper.indexedValue(String value) only gets
called

for name_untouched, and not name. Should the JsonStringFieldMapper
used for both the default "name" and "name_untouched" fields?

Mapping setup via:

server.client().admin().indices().putMapping(putMappingRequest("test").mapp

ingSource(mapping)).actionGet();

Document is indexed via:

server.client().index(indexRequest("test").type("stringtest").id("1")

           .source(jsonBuilder().startObject().field("name",

"123").endObject())).actionGet();

server.client().admin().indices().refresh(refreshRequest()).actionGet();

then a count is done:

CountResponse countResponse =
client.count(countRequest("test").querySource(termQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =

client.count(countRequest("test").querySource(termQuery("name.name_name",

"123"))).actionGet();
CountResponse countResponse3 =

client.count(countRequest("test").querySource(termQuery("name.name_untouche

d",

"123"))).actionGet();
CountResponse countResponse4 =
client.count(countRequest("test").querySource(termQuery("name.name
",

"123"))).actionGet();
CountResponse countResponse5 =

client.count(countRequest("test").querySource(termQuery("name.untouched",

"123"))).actionGet();

of the above 5 count operations only count 1 + 3 return the correct
count number of 1.

Should "name.untouched" also work?

It's probably a complete mis-reading of

http://www.elasticsearch.com/docs/elasticsearch/mapping/multi_field/

on my part.

cheers,
/dom


(dominict) #8

I managed to create a test case this morning with Date and String
types. Not sure what was confusing me last night.
I've created the issue, with code sample:

I hadn't really thought much into what would get stored in the _all
field. What I was mulling over in my head was,
if someone searched for 20100407, and the field stored the millis for
say now 1270640073794; they wouldn't match
as 20100407 == 1270594800794.

I was going to have a 2 formats associated with the type, so that the
timestamp, was chomped to say the start of the day

{type: "millis", store_format : "basic_date", search_format :
"YYYYMMDD:HHmmss" }

Then is someone searched for 20100407 or 20100407:124304 then both
search terms would be reduced to the store_format.

Hadn't quite worked out in my head how to actually search for things
for say: YYYYMMDD:HH, but that is where I was going with the
multi_fields investigation.

cheers
/dom

On Apr 7, 9:26 am, Shay Banon shay.ba...@elasticsearch.com wrote:

What you are working on is actually a very valid feature, and I was thinking
about just that the other day. It should be simple to allow for the date
type of handle long as well as strings (that represent the date). What I was
not so sure about is what it should return if stores (simplest I was
thinking about it formatted dates), and what should be stored in the _all
field?

cheers,
shay.banon

On Wed, Apr 7, 2010 at 2:57 AM, dominict dominic.toot...@gmail.com wrote:

I'll knock up a test case, and submit an issue.

The plugin I'm looking at is nothing special afraid, just using it as
a nice learning aid.
What I was looking at was the ability to store milliseconds in the
index, but search (and range search) using formatted dates.
I noticed some fields in documents that are being stored in a key/
value store at work, contain timestamps in milliseconds (not sure
why),
but thought it would be handy to be able to find all those docs with
timestamps from today (2010-04-06 to 2010-04-07) for example.
Not a great plugin, I know :slight_smile:

cheers
/dom

On Apr 7, 12:29 am, Shay Banon shay.ba...@elasticsearch.com wrote:

No, it should not be a JsonLong one..., can you raise an issue and a
testcase? Would like to nail this for 0.6 if possible.

Just myself being the usual curious me, what plugin are you working on?

cheers,
shay.banon

On Wed, Apr 7, 2010 at 2:27 AM, dominict dominic.toot...@gmail.com
wrote:

Cheers for the update shay,

I'm more curious about the JsonFieldMapper that is obtained and called
when searching for the default field. i.e. for the mapping:

{
"stringtest" : {
"properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string", index :
"analyzed" },
"orig" : { type : "string", index :
"analyzed" },
"untouched" : {type: "string", index :
"not_analyzed"}
}
}
}
}
}

CountResponse countResponse1 =

server.client().count(countRequest("test").querySource(fieldQuery("name",

"123"))).actionGet();
CountResponse countResponse2 =

server.client().count(countRequest("test").querySource(fieldQuery("name.unt
ouched",

"123"))).actionGet();
CountResponse countResponse3 =

server.client().count(countRequest("test").querySource(fieldQuery("name.ori
g",

"123"))).actionGet();

upon searching, the method JsonFieldMapper.indexedValue(String) is
called (to format the string for searching I guess). What I'm seeing
is the following:

name is assigned JsonLongFieldMapper
name.orig is assigned JsonStringFieldMapper
name.untouched is assigned JsonStringFieldMapper

Is this meant to happen, or should it be assigned
JsonStringFieldMapper. I'm just looking at plugin development in
0.6.0 (which is working well), and this is where my question is
stemming from.

cheers
/dom

On Apr 6, 11:33 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

You are being modest :). There is a bug in the docs (just fixed it).
The

idea is that any mapping unit a multi_field that is not using the
same

name

as the multi_field can be accessed by navigating through the
multi_field

name.

So, the docs are wrong, have a look at the corrected ones (the change
is

to

rename name_untouched to untouched). In your case, since the extra
field(s)
is called name_untouched, then you navigate to it using
name.name_untouched.
If you change it to untouched, then name.untouched should work.

cheers,
shay.banon

On Tue, Apr 6, 2010 at 10:11 PM, dominict <dominic.toot...@gmail.com

wrote:

Say I have the following mapping :

{
"stringtest" : {
" properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : { type : "string" },
"name_untouched" : {type: "string"}
}
}
}
}
}

The JsonStringFieldMapper.indexedValue(String value) only gets
called

for name_untouched, and not name. Should the JsonStringFieldMapper
used for both the default "name" and "name_untouched" fields?

Mapping setup via:

server.client().admin().indices().putMapping(putMappingRequest("test").mapp

ingSource(mapping)).actionGet();

Document is indexed via:

server.client().index(indexRequest("test").type("stringtest").id("1")

           .source(jsonBuilder().startObject().field("name",

"123").endObject())).actionGet();

server.client().admin().indices().refresh(refreshRequest()).actionGet();

then a count is done:

CountResponse countResponse =
client.count(countRequest("test").querySource(termQuery("name",
"123"))).actionGet();
CountResponse countResponse2 =

client.count(countRequest("test").querySource(termQuery("name.name_name",

"123"))).actionGet();
CountResponse countResponse3 =

client.count(countRequest("test").querySource(termQuery("name.name_untouche

d",

"123"))).actionGet();
CountResponse countResponse4 =
client.count(countRequest("test").querySource(termQuery("name.name
",

"123"))).actionGet();
CountResponse countResponse5 =

client.count(countRequest("test").querySource(termQuery("name.untouched",

"123"))).actionGet();

of the above 5 count operations only count 1 + 3 return the correct
count number of 1.

Should "name.untouched" also work?

It's probably a complete mis-reading of

http://www.elasticsearch.com/docs/elasticsearch/mapping/multi_field/

on my part.

cheers,
/dom


(system) #9