Base64 strings and range


(dunaeth) #1

Hi,

Would there be any way to analyze base64 string fields so that one can
execute range queries on them. I want to store IPs (v4 and v6) as base64
encoded strings and be able to retrieve those within a given range.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/98b0acb5-c66a-44bb-bd4d-2d7d39a8d5e1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Brian Yoder) #2

Your client converts IP addresses to base64 strings before it indexes the
documents, so it must also use the same conversion to a base64 string when
querying the document.

Ranges are a little more complicated. For example. if you simply encode "9"
and "111" as strings, then you cannot perform range queries, since "111" is
less than "0", string-wise. In this case you must do something that ensures
that "9" is less than "111", typically by creating fixed-length input
strings before encoding; for example "009" and "111". Now the string
comparisons will match the intended numeric comparisons.

IP addresses will be more complex. But still, you will need to apply the
same external conversions in front of your queries that you do in front of
your indexing.

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/893f0e01-388c-4350-8b8c-151a06ed8f59%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Brian Yoder) #3

Quick typing correction:

On Tuesday, January 14, 2014 4:00:58 PM UTC-5, InquiringMind wrote:

Your client converts IP addresses to base64 strings before it indexes the
documents, so it must also use the same conversion to a base64 string when
querying the document.

Ranges are a little more complicated. For example. if you simply encode
"9" and "111" as strings, then you cannot perform range queries, since "111"
is less than "9"
, string-wise. In this case you must do something that
ensures that "9" is less than "111", typically by creating fixed-length
input strings before encoding; for example "009" and "111". Now the string
comparisons will match the intended numeric comparisons.

IP addresses will be more complex. But still, you will need to apply the
same external conversions in front of your queries that you do in front of
your indexing.

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f6055fc2-d753-4f31-a277-aaedf07f61ec%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #4

IPv4 can be encoded to unsigned long (32bit), where range search is
available.

IPv6 is harder as it is 128bit, where no Java type is available except
something like BigDecimal, and no range search.

Do not use base64, it is impossible for executing range queries on it.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHfTMUiXFhmup_FfBZFYvTRZRUqop51gAy_ZxkghKnF1Q%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #5