Searching on exponent numbers


(Craig Berry) #1

Hi there,

I want to be able to provide a text search against a document that contains
a large number (10+ digits) but running into some automatic exponent
conversion issues with my mapping.

Scenario: User can submit a value of 1234567890 in a field and then wants
to be able to return that document by searching for the exact text
"1234567890".

In my current implementation, I'm storing the value in a double typed field
(since the field could contain floating point values) with a sub field that
copies the string value to another field with a custom analyser that splits
the number by decimal point:

"numberField": {
"type": "double",
"fields": {
"asText": {
"type": "string",
"copy_to": [
"all_numbers"
]
}
},
"all_numbers": {
"type": "string",
"analyzer": "number_analyzer"
}

So this means if the value is 31, then it's stored as 31.0 in numberField
and then in all_numbers as "31" and "0". So a text search against the
all_numbers field will return this document.

However, when the number value is very large, it seems the double
conversion to string is using exponents, e.g. 1234567890 becomes
1.23456789E9. This means what's stored in my all_numbers field is "1" and
"23456789E9" and so a text search of "1234567890" returns 0 hits.

At this point I'm just wondering whether my source document should contain
the value twice, once as a number and once as string and avoid needing to
do any custom analysis, however this does mean moving this logic to the
application layer when I'd rather Elastic could just handle it.

Any other suggestions on how I could get Elastic to store the value as I
desire?

FYI I have other custom analysers running on the _all field that means
searching that field doesn't return the 'number' either.

TIA

Craig

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1895aa0e-659f-425a-8ae4-9b8b69f9eea1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

This is a long unresolved issue.

One solution would be adding BigDecimal support. See for example

Jörg

On Fri, May 22, 2015 at 8:20 AM, Craig Berry craig.adrian.berry@gmail.com
wrote:

Hi there,

I want to be able to provide a text search against a document that
contains a large number (10+ digits) but running into some automatic
exponent conversion issues with my mapping.

Scenario: User can submit a value of 1234567890 in a field and then wants
to be able to return that document by searching for the exact text
"1234567890".

In my current implementation, I'm storing the value in a double typed
field (since the field could contain floating point values) with a sub
field that copies the string value to another field with a custom analyser
that splits the number by decimal point:

"numberField": {
"type": "double",
"fields": {
"asText": {
"type": "string",
"copy_to": [
"all_numbers"
]
}
},
"all_numbers": {
"type": "string",
"analyzer": "number_analyzer"
}

So this means if the value is 31, then it's stored as 31.0 in numberField
and then in all_numbers as "31" and "0". So a text search against the
all_numbers field will return this document.

However, when the number value is very large, it seems the double
conversion to string is using exponents, e.g. 1234567890 becomes
1.23456789E9. This means what's stored in my all_numbers field is "1" and
"23456789E9" and so a text search of "1234567890" returns 0 hits.

At this point I'm just wondering whether my source document should contain
the value twice, once as a number and once as string and avoid needing to
do any custom analysis, however this does mean moving this logic to the
application layer when I'd rather Elastic could just handle it.

Any other suggestions on how I could get Elastic to store the value as I
desire?

FYI I have other custom analysers running on the _all field that means
searching that field doesn't return the 'number' either.

TIA

Craig

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1895aa0e-659f-425a-8ae4-9b8b69f9eea1%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1895aa0e-659f-425a-8ae4-9b8b69f9eea1%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG4_yghdUijNB3q3Ppcc-ws1WziNk62vo90ZOqKwUfLfA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3