I use Gson to convert Java classes to Json and then directly put them into
ES. One of the classes I use has a HashMap<String, Double> as a central
piece, where it stores its key-value pairs. It's really just that: the keys
are stings, the values doubles. ES however creates a mapping for every key
of the hashmap as you can see above.
Does someone here know what mapping I need to define to avoid that?
Also: Can I change this mapping on the already existing index, or do I have
to create a new index, then create a mapping, then copy over the data from
the old index?
I use Gson to convert Java classes to Json and then directly put them
into ES. One of the classes I use has a HashMap<String, Double> as a
central piece, where it stores its key-value pairs. It's really just
that: the keys are stings, the values doubles. ES however creates a
mapping for every key of the hashmap as you can see above.
Does someone here know what mapping I need to define to avoid that?
Also: Can I change this mapping on the already existing index, or do I
have to create a new index, then create a mapping, then copy over the
data from the old index?
can you tell me, where your problem with the above approach actually is?
You feed a number of key/value pairs into elasticsearch, each key and each
value is evaluated by its type and then put into the mapping, as each key
becomes an own field in elasticsearch, which can be searched for. Wondering
why this is a problem for you? Or why do you want to avoid that?
Also, where and how do you want to change the mapping to?
I am a bit confused what and why you are expecting to be different than it
actually is. Maybe you should not think in java data structures but rather
in JSON, which is being indexed and needs to create a mapping in order to
be able to search for it. Happy to help, if I understand what you are
trying to do. Please elaborate.
I use Gson to convert Java classes to Json and then directly put them
into ES. One of the classes I use has a HashMap<String, Double> as a
central piece, where it stores its key-value pairs. It's really just
that: the keys are stings, the values doubles. ES however creates a
mapping for every key of the hashmap as you can see above.
Does someone here know what mapping I need to define to avoid that?
Also: Can I change this mapping on the already existing index, or do I
have to create a new index, then create a mapping, then copy over the
data from the old index?
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/52D3BCAD.5060208%40swe-blog.net.
thanks for your reply first of all. The problem is that I will have
millions of keys, and that ES creates a mapping for each. That will bloat
the mappings data structure and probably lead to some memory and/or
performance issues somewhere (I don't know enough about ES internals, to
know precisely where, but it cannot be good to have a few million entries
in the mapping where one would do.)
Hope that helps!
Peter
Am Dienstag, 14. Januar 2014 13:01:22 UTC+1 schrieb Alexander Reelsen:
Hey Peter,
can you tell me, where your problem with the above approach actually is?
You feed a number of key/value pairs into elasticsearch, each key and each
value is evaluated by its type and then put into the mapping, as each key
becomes an own field in elasticsearch, which can be searched for. Wondering
why this is a problem for you? Or why do you want to avoid that?
Also, where and how do you want to change the mapping to?
I am a bit confused what and why you are expecting to be different than it
actually is. Maybe you should not think in java data structures but rather
in JSON, which is being indexed and needs to create a mapping in order to
be able to search for it. Happy to help, if I understand what you are
trying to do. Please elaborate.
--Alex
On Mon, Jan 13, 2014 at 11:15 AM, Oliver B. Fischer <mail...@swe-blog.net<javascript:>
wrote:
Hi Peter,
ES allows you to defined dynamic mappings there you can determine the
mapping of a property based on the evaluation of some conditions.
I use Gson to convert Java classes to Json and then directly put them
into ES. One of the classes I use has a HashMap<String, Double> as a
central piece, where it stores its key-value pairs. It's really just
that: the keys are stings, the values doubles. ES however creates a
mapping for every key of the hashmap as you can see above.
Does someone here know what mapping I need to define to avoid that?
Also: Can I change this mapping on the already existing index, or do I
have to create a new index, then create a mapping, then copy over the
data from the old index?
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/52D3BCAD.5060208%40swe-blog.net.
Did you find a resolution for Hashmap mappings? Looking for the same thing.
Thanks,
Sangita.
On Tuesday, January 14, 2014 9:03:35 AM UTC-5, Peter Webber wrote:
Hi Alex,
thanks for your reply first of all. The problem is that I will have
millions of keys, and that ES creates a mapping for each. That will bloat
the mappings data structure and probably lead to some memory and/or
performance issues somewhere (I don't know enough about ES internals, to
know precisely where, but it cannot be good to have a few million entries
in the mapping where one would do.)
Hope that helps!
Peter
Am Dienstag, 14. Januar 2014 13:01:22 UTC+1 schrieb Alexander Reelsen:
Hey Peter,
can you tell me, where your problem with the above approach actually is?
You feed a number of key/value pairs into elasticsearch, each key and each
value is evaluated by its type and then put into the mapping, as each key
becomes an own field in elasticsearch, which can be searched for. Wondering
why this is a problem for you? Or why do you want to avoid that?
Also, where and how do you want to change the mapping to?
I am a bit confused what and why you are expecting to be different than
it actually is. Maybe you should not think in java data structures but
rather in JSON, which is being indexed and needs to create a mapping in
order to be able to search for it. Happy to help, if I understand what you
are trying to do. Please elaborate.
I use Gson to convert Java classes to Json and then directly put them
into ES. One of the classes I use has a HashMap<String, Double> as a
central piece, where it stores its key-value pairs. It's really just
that: the keys are stings, the values doubles. ES however creates a
mapping for every key of the hashmap as you can see above.
Does someone here know what mapping I need to define to avoid that?
Also: Can I change this mapping on the already existing index, or do I
have to create a new index, then create a mapping, then copy over the
data from the old index?
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/52D3BCAD.5060208%40swe-blog.net.
Hi,
I think first you need to define is what is the requirement (or
expectation) of hashMap (sub)object in index -> do you need to search
on key/value pairs?
Like for example, query = 'hashMap.N_10290607:XY' ?
If not, if you only need to store json serialization of haspmap as
part of bigger object inside index (_source) to retrieve it later, you
can define not to expand mapping dynamically ("dynamic" : false) in
the 'hashMap' (sub)object - part of mapping for, like bellow:
This is done by a custom mapping. If you do not define a custom
mapping elasticsearch will, by default, create and expand mapping for
hashMap for all keys which will sooner or later create issues in
index. For more details check
If you don't know how to create mapping in the first place you can get
existing mapping
(Elasticsearch Platform — Find real-time answers at scale | Elastic),
remove fields added/recognised by elasticsearch automatically, define
"dynamic" : false for the 'hashMap' object and use new mapping when
reindexing data.
Did you find a resolution for Hashmap mappings? Looking for the same thing.
Thanks,
Sangita.
On Tuesday, January 14, 2014 9:03:35 AM UTC-5, Peter Webber wrote:
Hi Alex,
thanks for your reply first of all. The problem is that I will have
millions of keys, and that ES creates a mapping for each. That will bloat
the mappings data structure and probably lead to some memory and/or
performance issues somewhere (I don't know enough about ES internals, to
know precisely where, but it cannot be good to have a few million entries in
the mapping where one would do.)
Hope that helps!
Peter
Am Dienstag, 14. Januar 2014 13:01:22 UTC+1 schrieb Alexander Reelsen:
Hey Peter,
can you tell me, where your problem with the above approach actually is?
You feed a number of key/value pairs into elasticsearch, each key and each
value is evaluated by its type and then put into the mapping, as each key
becomes an own field in elasticsearch, which can be searched for. Wondering
why this is a problem for you? Or why do you want to avoid that?
Also, where and how do you want to change the mapping to?
I am a bit confused what and why you are expecting to be different than
it actually is. Maybe you should not think in java data structures but
rather in JSON, which is being indexed and needs to create a mapping in
order to be able to search for it. Happy to help, if I understand what you
are trying to do. Please elaborate.
I use Gson to convert Java classes to Json and then directly put them
into ES. One of the classes I use has a HashMap<String, Double> as a
central piece, where it stores its key-value pairs. It's really just
that: the keys are stings, the values doubles. ES however creates a
mapping for every key of the hashmap as you can see above.
Does someone here know what mapping I need to define to avoid that?
Also: Can I change this mapping on the already existing index, or do I
have to create a new index, then create a mapping, then copy over the
data from the old index?
Many Thanks!
|
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
How would I define a custom mapping if I wanted to index/store only the
keys in the map and discard the values?
Prasanna
On Monday, March 17, 2014 2:28:39 AM UTC-7, Tomislav Poljak wrote:
Hi,
I think first you need to define is what is the requirement (or
expectation) of hashMap (sub)object in index -> do you need to search
on key/value pairs?
Like for example, query = 'hashMap.N_10290607:XY' ?
If not, if you only need to store json serialization of haspmap as
part of bigger object inside index (_source) to retrieve it later, you
can define not to expand mapping dynamically ("dynamic" : false) in
the 'hashMap' (sub)object - part of mapping for, like bellow:
This is done by a custom mapping. If you do not define a custom
mapping elasticsearch will, by default, create and expand mapping for
hashMap for all keys which will sooner or later create issues in
index. For more details check
remove fields added/recognised by elasticsearch automatically, define
"dynamic" : false for the 'hashMap' object and use new mapping when
reindexing data.
Did you find a resolution for Hashmap mappings? Looking for the same
thing.
Thanks,
Sangita.
On Tuesday, January 14, 2014 9:03:35 AM UTC-5, Peter Webber wrote:
Hi Alex,
thanks for your reply first of all. The problem is that I will have
millions of keys, and that ES creates a mapping for each. That will
bloat
the mappings data structure and probably lead to some memory and/or
performance issues somewhere (I don't know enough about ES internals,
to
know precisely where, but it cannot be good to have a few million
entries in
the mapping where one would do.)
Hope that helps!
Peter
Am Dienstag, 14. Januar 2014 13:01:22 UTC+1 schrieb Alexander Reelsen:
Hey Peter,
can you tell me, where your problem with the above approach actually
is?
You feed a number of key/value pairs into elasticsearch, each key and
each
value is evaluated by its type and then put into the mapping, as each
key
becomes an own field in elasticsearch, which can be searched for.
Wondering
why this is a problem for you? Or why do you want to avoid that?
Also, where and how do you want to change the mapping to?
I am a bit confused what and why you are expecting to be different
than
it actually is. Maybe you should not think in java data structures but
rather in JSON, which is being indexed and needs to create a mapping
in
order to be able to search for it. Happy to help, if I understand what
you
are trying to do. Please elaborate.
I use Gson to convert Java classes to Json and then directly put
them
into ES. One of the classes I use has a HashMap<String, Double> as a
central piece, where it stores its key-value pairs. It's really just
that: the keys are stings, the values doubles. ES however creates a
mapping for every key of the hashmap as you can see above.
Does someone here know what mapping I need to define to avoid that?
Also: Can I change this mapping on the already existing index, or do
I
have to create a new index, then create a mapping, then copy over
the
data from the old index?
Many Thanks!
|
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
--
You received this message because you are subscribed to the Google
Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.