Sort on missing field


(erik.seilnacht@mulesoft.com) #1

Hi.

I have an index where I need to sort on a sequence number field that was recently added. The problem is that I must use the missing option to allow for sorting since only new entries in the index will have a value.

Even though a Long was specified ( using the java API ) when this field was introduced, the type defaulted to "string"... In order for the sort.missing to work, I understand the field must be numeric.

I've tried using the mapping rest API to change the type to "long" but it didn't work...

root@ec2-184-72-181-182:~# curl -XGET http://search.server.com:9200/index/erik/_mapping?pretty;echo
{
"erik" : {
"properties" : {
"timestamp" : {
"format" : "dateOptionalTime",
"type" : "date"
},
"message" : {
"dynamic" : "true",
"properties" : {
"text" : {
"type" : "string"
}
}
},
"sequence" : {
"type" : "string"
},
"priority" : {
"type" : "string"
},
"fields" : {
"dynamic" : "true",
"properties" : {
"domain" : {
"type" : "string"
},
"serverId" : {
"type" : "string"
}
}
}
}
}
}

curl -XPUT http://search.server.com:9200/index/erik/_mapping -d '{ "sequence" : { "properties" : { "sequence" : { "type" : "long" } } } }';echo
{"error":"MergeMappingException[Merge failed with failures {[mapper [sequence] of different type, current_type [string], merged_type [long]]}]","status":400}

so I used the multi_field option:

curl -XPUT http://search.server.com:9200/index/erik/_mapping -d '{ "sequence" : { "properties" : { "sequence" : { "type" : "multi_field", "fields" : { "sequence" : { "type" : "long" } } } } } }';echo
{"ok":true,"acknowledged":true}

and the mappings were changed to:

{
"erik" : {
"properties" : {
"timestamp" : {
"format" : "dateOptionalTime",
"type" : "date"
},
"message" : {
"dynamic" : "true",
"properties" : {
"text" : {
"type" : "string"
}
}
},
"sequence" : {
"type" : "multi_field",
"fields" : {
"sequence" : {
"type" : "long"
}
}
},
"priority" : {
"type" : "string"
},
"fields" : {
"dynamic" : "true",
"properties" : {
"domain" : {
"type" : "string"
},
"serverId" : {
"type" : "string"
}
}
}
}
}
}

However, my sort query still fails...

root@ec2-184-72-181-182:~# curl -XGET http://search.server.com:9200/index/_search?pretty=true -d ' {"from" : 1, "size" : 2, "query" : { "range" : { "timestamp" : { "from" : "2011-07-22T07:00:45.012Z", "to" : null } } }, "sort" : [ {"timestamp" : {} }, { "sequence" : { "missing" : 0 } } ]} '

ElasticSearchIllegalArgumentException[Sorting on string type field does not support missing parameter]

Not sure what I can do at this point...

Any advice?

Thanks,

-Erik


(Shay Banon) #2

You can't change a field type, once its set. The reason for that is that it
is indexed in a specific manner based on the type. Its strange that it
became string if you indexed it as long though... You need to reindex the
data to use it as long.

On Sat, Jul 23, 2011 at 8:42 PM, erik.seilnacht@mulesoft.com <
erik.seilnacht@mulesoft.com> wrote:

Hi.

I have an index where I need to sort on a sequence number field that was
recently added. The problem is that I must use the missing option to allow
for sorting since only new entries in the index will have a value.

Even though a Long was specified ( using the java API ) when this field was
introduced, the type defaulted to "string"... In order for the
sort.missing
to work, I understand the field must be numeric.

I've tried using the mapping rest API to change the type to "long" but it
didn't work...

root@ec2-184-72-181-182:~# curl -XGET
http://search.server.com:9200/index/erik/_mapping?pretty;echo
{
"erik" : {
"properties" : {
"timestamp" : {
"format" : "dateOptionalTime",
"type" : "date"
},
"message" : {
"dynamic" : "true",
"properties" : {
"text" : {
"type" : "string"
}
}
},
"sequence" : {
"type" : "string"
},
"priority" : {
"type" : "string"
},
"fields" : {
"dynamic" : "true",
"properties" : {
"domain" : {
"type" : "string"
},
"serverId" : {
"type" : "string"
}
}
}
}
}
}

curl -XPUT http://search.server.com:9200/index/erik/_mapping -d '{
"sequence" : { "properties" : { "sequence" : { "type" : "long" } } }
}';echo
{"error":"MergeMappingException[Merge failed with failures {[mapper
[sequence] of different type, current_type [string], merged_type
[long]]}]","status":400}

so I used the multi_field option:

curl -XPUT http://search.server.com:9200/index/erik/_mapping -d '{
"sequence" : { "properties" : { "sequence" : { "type" : "multi_field",
"fields" : { "sequence" : { "type" : "long" } } } } } }';echo
{"ok":true,"acknowledged":true}

and the mappings were changed to:

{
"erik" : {
"properties" : {
"timestamp" : {
"format" : "dateOptionalTime",
"type" : "date"
},
"message" : {
"dynamic" : "true",
"properties" : {
"text" : {
"type" : "string"
}
}
},
"sequence" : {
"type" : "multi_field",
"fields" : {
"sequence" : {
"type" : "long"
}
}
},
"priority" : {
"type" : "string"
},
"fields" : {
"dynamic" : "true",
"properties" : {
"domain" : {
"type" : "string"
},
"serverId" : {
"type" : "string"
}
}
}
}
}
}

However, my sort query still fails...

root@ec2-184-72-181-182:~# curl -XGET
http://search.server.com:9200/index/_search?pretty=true -d ' {"from" : 1,
"size" : 2, "query" : { "range" : { "timestamp" : { "from" :
"2011-07-22T07:00:45.012Z", "to" : null } } }, "sort" : [ {"timestamp" : {}
}, { "sequence" : { "missing" : 0 } } ]} '

ElasticSearchIllegalArgumentException[Sorting on string type field does not
support missing parameter]

Not sure what I can do at this point...

Any advice?

Thanks,

-Erik

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/sort-on-missing-field-tp3193903p3193903.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(system) #3