Elastic Search -Hive mapping


(P lva) #1

Hello,

I have an event timestamp in hive table which I'm breaking into
evnt_time(timestamp) and an evnt_date(string) columns. The idea is to use
the evnt_time as timestamp in kibana and be able to search for date and
stack one day on top of another in a histogram plot in kibana. Eventually
giving me comparison between two days as a stacked chart.

As per elasticsearch-hive documentation, strings in hive will be of type
string in elasticsearch also. However when I load data into elasticsearch
the date column is being considered as timestamp.

How do I prevent this ? Please share if you have any ideas on how to get a
stacked chart.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/37ebc775-09c1-4ccf-9063-18dc8b959c3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Costin Leau) #2

Hi,

Es-Hadoop doesn't perform strict mapping - it uses the Hive mapping to infer the JSON types used by the document which
in turn are interpreted by Elasticsearch, which by default, will try to figure out the type of the data sent, in your
case, a timestamp.
The solution to this is create the mapping before hand in Elasticsearch - this gives you full control over what fields
and types you want (not to mention indexing options,etc).
This is discussed at length in the docs as well - see
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/mapping.html#mapping-date

Cheers,

On 4/9/14 9:31 PM, P lva wrote:

Hello,
I have an event timestamp in hive table which I'm breaking into evnt_time(timestamp) and an evnt_date(string) columns.
The idea is to use the evnt_time as timestamp in kibana and be able to search for date and stack one day on top of
another in a histogram plot in kibana. Eventually giving me comparison between two days as a stacked chart.

As per elasticsearch-hive documentation, strings in hive will be of type string in elasticsearch also. However when I
load data into elasticsearch the date column is being considered as timestamp.

How do I prevent this ? Please share if you have any ideas on how to get a stacked chart.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/37ebc775-09c1-4ccf-9063-18dc8b959c3d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/37ebc775-09c1-4ccf-9063-18dc8b959c3d%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53459C64.3020104%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3