Field analyzer usage


(Sisu Alexandru) #1

Hi,

I have a mapping and a setting, and the "body" field doesn't gets analyzed
by the "x_analyzer".

MAPPING = {
"posting":
{

        "properties":
                {
                "country": {"type": "integer", "store": "yes"},
                "username": {"type": "string", "store": "yes"},
                "body": {"type": "string", "store": "yes", "index":

"analyzed", "analyzer": "x_analyzer"},
}
}

and the index creation setting:

SETTINGS = {
"index": {
"number_of_shards": 1
}
,
"analysis": {
"analyzer": {
"x_analyzer": {
"type": "com.xyz.elasticsearch.analysis.XAnalyzerProvider"
},
}

So, it doesnt work.What am I doing wrong?

Tnx,

Alex

ps: If I change the "x_analyzer" to "default" and
remove "analyzer": "x_analyzer" from the mapping, everything goes okey.


(Shay Banon) #2

What doesn't work? That would help a bit...

On Mon, Nov 28, 2011 at 6:27 PM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Hi,

I have a mapping and a setting, and the "body" field doesn't gets analyzed
by the "x_analyzer".

MAPPING = {
"posting":
{

        "properties":
                {
                "country": {"type": "integer", "store": "yes"},
                "username": {"type": "string", "store": "yes"},
                "body": {"type": "string", "store": "yes", "index":

"analyzed", "analyzer": "x_analyzer"},
}
}

and the index creation setting:

SETTINGS = {
"index": {
"number_of_shards": 1
}
,
"analysis": {
"analyzer": {
"x_analyzer": {
"type": "com.xyz.elasticsearch.analysis.XAnalyzerProvider"
},
}

So, it doesnt work.What am I doing wrong?

Tnx,

Alex

ps: If I change the "x_analyzer" to "default" and
remove "analyzer": "x_analyzer" from the mapping, everything goes okey.


(Sisu Alexandru) #3

Well, the body field is not indexed by the x_analyzer.
Actually all the fields are indexed by the standardanalyzer.

Other info:
My es configuration, doesn't have anything configured in the
elasticsearch.yml.
When I'm changing the name of the x_analyzer to default, it works.

On Mon, Nov 28, 2011 at 7:24 PM, Shay Banon kimchy@gmail.com wrote:

What doesn't work? That would help a bit...

On Mon, Nov 28, 2011 at 6:27 PM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Hi,

I have a mapping and a setting, and the "body" field doesn't gets
analyzed by the "x_analyzer".

MAPPING = {
"posting":
{

        "properties":
                {
                "country": {"type": "integer", "store": "yes"},
                "username": {"type": "string", "store": "yes"},
                "body": {"type": "string", "store": "yes", "index":

"analyzed", "analyzer": "x_analyzer"},
}
}

and the index creation setting:

SETTINGS = {
"index": {
"number_of_shards": 1
}
,
"analysis": {
"analyzer": {
"x_analyzer": {
"type": "com.xyz.elasticsearch.analysis.XAnalyzerProvider"
},
}

So, it doesnt work.What am I doing wrong?

Tnx,

Alex

ps: If I change the "x_analyzer" to "default" and
remove "analyzer": "x_analyzer" from the mapping, everything goes okey.


(Shay Banon) #4

It might be the settings that you provide, I think your analyzer is not
configured correctly, try and put the analysis config under the "index"
element (same as number_of_shards). If it doesn't work, add a debug
statement to your custom analyzer provider, see if it gets created at all.

On Tue, Nov 29, 2011 at 12:46 AM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Well, the body field is not indexed by the x_analyzer.
Actually all the fields are indexed by the standardanalyzer.

Other info:
My es configuration, doesn't have anything configured in the
elasticsearch.yml.
When I'm changing the name of the x_analyzer to default, it works.

On Mon, Nov 28, 2011 at 7:24 PM, Shay Banon kimchy@gmail.com wrote:

What doesn't work? That would help a bit...

On Mon, Nov 28, 2011 at 6:27 PM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Hi,

I have a mapping and a setting, and the "body" field doesn't gets
analyzed by the "x_analyzer".

MAPPING = {
"posting":
{

        "properties":
                {
                "country": {"type": "integer", "store": "yes"},
                "username": {"type": "string", "store": "yes"},
                "body": {"type": "string", "store": "yes", "index":

"analyzed", "analyzer": "x_analyzer"},
}
}

and the index creation setting:

SETTINGS = {
"index": {
"number_of_shards": 1
}
,
"analysis": {
"analyzer": {
"x_analyzer": {
"type":
"com.xyz.elasticsearch.analysis.XAnalyzerProvider"
},
}

So, it doesnt work.What am I doing wrong?

Tnx,

Alex

ps: If I change the "x_analyzer" to "default" and
remove "analyzer": "x_analyzer" from the mapping, everything goes okey.


(Sisu Alexandru) #5

Tnx for the answer.
It turned out that it was my mistake. I was sending the mapping in an
incorrect way..
Here is the mistake:

MAPPING = {
"posting":
{

        "properties":
                {
                "country": {"type": "integer", "store": "yes"},
                "username": {"type": "string", "store": "yes"},
                "body": {"type": "string", "store": "yes", "index":

"analyzed", "analyzer": "x_analyzer"},
}
}

I was doing: mapping_res = conn.put_mapping("posting", {'properties':
MAPPING}, ["testindex"])

The problem was that I already had this 2 elements (!posting! and
!properties!) defined in the MAPPING dict.

So, the MAPPING is now:
MAPPING = {
"country": {"type": "integer", "store": "yes"},
"username": {"type": "string", "store": "yes"},
"body": {"type": "string", "store": "yes", "index":
"analyzed", "analyzer": "x_analyzer"},
}

and the INDEX SETTING is as you've said:
SETTINGS = {
"index": {
"number_of_shards": 1,
"analysis": {
"analyzer": {
"x_analyzer": {
"type": "com.xyz.elasticsearch.analysis.XAnalyzerProvider"
},

}

}

On Tue, Nov 29, 2011 at 8:52 AM, Shay Banon kimchy@gmail.com wrote:

It might be the settings that you provide, I think your analyzer is not
configured correctly, try and put the analysis config under the "index"
element (same as number_of_shards). If it doesn't work, add a debug
statement to your custom analyzer provider, see if it gets created at all.

On Tue, Nov 29, 2011 at 12:46 AM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Well, the body field is not indexed by the x_analyzer.
Actually all the fields are indexed by the standardanalyzer.

Other info:
My es configuration, doesn't have anything configured in the
elasticsearch.yml.
When I'm changing the name of the x_analyzer to default, it works.

On Mon, Nov 28, 2011 at 7:24 PM, Shay Banon kimchy@gmail.com wrote:

What doesn't work? That would help a bit...

On Mon, Nov 28, 2011 at 6:27 PM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Hi,

I have a mapping and a setting, and the "body" field doesn't gets
analyzed by the "x_analyzer".

MAPPING = {
"posting":
{

        "properties":
                {
                "country": {"type": "integer", "store": "yes"},
                "username": {"type": "string", "store": "yes"},
                "body": {"type": "string", "store": "yes", "index":

"analyzed", "analyzer": "x_analyzer"},
}
}

and the index creation setting:

SETTINGS = {
"index": {
"number_of_shards": 1
}
,
"analysis": {
"analyzer": {
"x_analyzer": {
"type":
"com.xyz.elasticsearch.analysis.XAnalyzerProvider"
},
}

So, it doesnt work.What am I doing wrong?

Tnx,

Alex

ps: If I change the "x_analyzer" to "default" and
remove "analyzer": "x_analyzer" from the mapping, everything goes okey.


(system) #6