# Need advice on my Open Source project using elastic search and question on MapperParsingException **URL:** \ **Category:** Elasticsearch\ **Created:** [February 5, 2015, 9:45pm UTC](https://discuss.elastic.co/t/need-advice-on-my-open-source-project-using-elastic-search-and-question-on-mapperparsingexception/22002 "2015-02-05T21:45:17Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Vicente\_de\_Rivera\_II](https://avatars.discourse-cdn.com/v4/letter/v/898d66/32.png) [@Vicente\_de\_Rivera\_II](https://discuss.elastic.co/u/Vicente_de_Rivera_II)\ **Post date:** [February 5, 2015, 9:45pm UTC](https://discuss.elastic.co/t/need-advice-on-my-open-source-project-using-elastic-search-and-question-on-mapperparsingexception/22002/1 "2015-02-05T21:45:17Z")
Hi all, New to ES here. I am currently working on an open source project called _PoE Merchant_. It is a "forum shops" indexer for a game called _Path of Exile._ I have an issue and a question below. But I would really appreciate any further advice on how should I proceed. _Project goals:_ 1. Index all the "Items for sale" in _Path of Exile_ forums shops (e.g. [http://www.pathofexile.com/forum/view-forum/561](http://www.pathofexile.com/forum/view-forum/561)). 1. At least finish re/indexing every 15mins. 2. Run application as a stand-alone Desktop (I'll use Java SE). 3. Provide a Lightning fast search in a very convenient way. 4. Provide some other Quality of Life features. _So far, my setup is:_ 1. Windows 7 2. Eclipse Luna 4.4.1 3. Jsoup for parsing the HTML. 4. ES 1.4 1. start ES via elasticsearch.bat _More context:_ 1. Path of Exile is a Free to Play Action RPG - it's like Diablo 3 but for hardcore gamers and it is not Pay to Win. 2. Path of Exile only provides the forums for player to setup "Shops". 3. A player makes a forum thread under on the shop subforums - 1 subforum will be equivalent to 1 Index. 4. A thread contains 1 or more "Item for sale" - this will be my Document 5. A player will update this shop thread and add new Items - this is why i need to re-index every x minutes. 6. Here is a sample shop: [http://www.pathofexile.com/forum/view-thread/1127160](http://www.pathofexile.com/forum/view-thread/1127160) 1. The bulk of the data I'll need to index is located in the last tag. _Problem currently:_ The data I'm gonna be scraping from the forums is fortunately contained in a . Currently I am trying things out and I encountered this issue am I curious of: The following is valid JSON data, but am I getting a org.elasticsearch.index.mapper.MapperParsingException: failed to parse [x.user] { ``` "user": [ 0, { "x": 1 } ] ``` } Why can't elasticsearch index this? Is there a guide on what JSON format/structure to use for ES? Given that one of my goals is fast index and fast search., are there any considerations regarding structuring my data? _Full stack trace:_ org.elasticsearch.index.mapper.MapperParsingException: failed to parse [x.user] at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:415) at org.elasticsearch.index.mapper.object.ObjectMapper.serializeObject(ObjectMapper.java:555) at org.elasticsearch.index.mapper.object.ObjectMapper.serializeNonDynamicArray(ObjectMapper.java:686) at org.elasticsearch.index.mapper.object.ObjectMapper.serializeArray(ObjectMapper.java:623) at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:492) at org.elasticsearch.index.mapper.object.ObjectMapper.putDynamicMapper(ObjectMapper.java:661) at org.elasticsearch.index.mapper.object.ObjectMapper.serializeObject(ObjectMapper.java:581) at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:490) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:541) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:490) at org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:392) at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:198) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:511) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:419) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: unknown property [x] at org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:290) at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:235) at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:405) ... 16 more Thanks a lot! Thirdy -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/229d5b30-4e8c-4f30-b0ae-2492524e9290%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/229d5b30-4e8c-4f30-b0ae-2492524e9290%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Vicente\_de\_Rivera\_II](https://avatars.discourse-cdn.com/v4/letter/v/898d66/32.png) [@Vicente\_de\_Rivera\_II](https://discuss.elastic.co/u/Vicente_de_Rivera_II)\ **Post date:** [February 5, 2015, 9:48pm UTC](https://discuss.elastic.co/t/need-advice-on-my-open-source-project-using-elastic-search-and-question-on-mapperparsingexception/22002/2 "2015-02-05T21:48:00Z")
sorry about the formatting, I pasted the JSON first, and the color went yellow. whoops. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/70a509b3-59c0-4e8c-b672-64ecfa0e71be%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/70a509b3-59c0-4e8c-b672-64ecfa0e71be%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Vicente\_de\_Rivera\_II](https://avatars.discourse-cdn.com/v4/letter/v/898d66/32.png) [@Vicente\_de\_Rivera\_II](https://discuss.elastic.co/u/Vicente_de_Rivera_II)\ **Post date:** [February 5, 2015, 9:53pm UTC](https://discuss.elastic.co/t/need-advice-on-my-open-source-project-using-elastic-search-and-question-on-mapperparsingexception/22002/3 "2015-02-05T21:53:45Z")
Code to reproduce the exception: @Test public void testESIndex() throws Exception { String json = "{" + ""user":[0, {"x":1}]" + "}"; Client client = new TransportClient() .addTransportAddress(new InetSocketTransportAddress( "localhost", 9300)); IndexResponse indexResponse = index(client, "a", "y", "{"x":" + json + "}"); logger.info(indexResponseToString(indexResponse)); client.close(); } public static IndexResponse index(Client client, String index, String type, String json) { IndexResponse actionGet = client.prepareIndex(index, type) .setSource(json).execute().actionGet(); return actionGet; } public static String indexResponseToString(IndexResponse actionGet) { return actionGet.getIndex() + "/" + actionGet.getType() + "/" - actionGet.getId() + " isCreated: " + actionGet.isCreated(); } -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 5, 2015, 10:02pm UTC](https://discuss.elastic.co/t/need-advice-on-my-open-source-project-using-elastic-search-and-question-on-mapperparsingexception/22002/4 "2015-02-05T22:02:19Z")
Unsure but your JSON looks incorrect to me. You are trying to send in an array a number 0 and an object. David > Le 5 févr. 2015 à 22:53, Vicente de Rivera III [thirdy.derivera@gmail.com](mailto:thirdy.derivera@gmail.com) a écrit : > > Code to reproduce the exception: > > @Test > public void testESIndex() throws Exception { > String json = "{" + > ""user":[0, {"x":1}]" + > "}"; > Client client = new TransportClient() > .addTransportAddress(new InetSocketTransportAddress( > "localhost", 9300)); > > ``` > IndexResponse indexResponse = index(client, "a", "y", "{\"x\":" + json + "}"); > > logger.info(indexResponseToString(indexResponse)); > > client.close(); > > ``` > > } > > public static IndexResponse index(Client client, String index, String type, String json) { > IndexResponse actionGet = client.prepareIndex(index, type) > .setSource(json).execute().actionGet(); > return actionGet; > } > > public static String indexResponseToString(IndexResponse actionGet) { > return actionGet.getIndex() + "/" + actionGet.getType() + "/" > + actionGet.getId() + " isCreated: " + actionGet.isCreated(); > } > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/626A9565-D5E7-4488-AF36-11F6E15C0D9E%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/626A9565-D5E7-4488-AF36-11F6E15C0D9E%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Vicente\_de\_Rivera\_II](https://avatars.discourse-cdn.com/v4/letter/v/898d66/32.png) [@Vicente\_de\_Rivera\_II](https://discuss.elastic.co/u/Vicente_de_Rivera_II)\ **Post date:** [February 6, 2015, 12:26am UTC](https://discuss.elastic.co/t/need-advice-on-my-open-source-project-using-elastic-search-and-question-on-mapperparsingexception/22002/5 "2015-02-06T00:26:05Z")
Thank you for the response. Both [jsonlint.com](http://jsonlint.com) and [JSON Validator - JSONLint tool to validate JSON data](http://codebeautify.org/jsonvalidate) says this JSON is valid: { "x": { "user": [ 0, { "x": 1 } ] } } I guess I'll parse that raw js data, build POJOs out of it and then pass these pojos into ES. (Was hoping I could dump this data into Elasticsearch without any trouble). Also, sorry to ask this, is Elastic search a good choice for my goals? I've never used any search technology before. On Friday, February 6, 2015 at 6:02:40 AM UTC+8, David Pilato wrote: > Unsure but your JSON looks incorrect to me. You are trying to send in an > array a number 0 and an object. > > David > > Le 5 févr. 2015 à 22:53, Vicente de Rivera III \<[thirdy....@gmail.com](mailto:thirdy....@gmail.com) > \\> a écrit : > > Code to reproduce the exception: > > @Test > public void testESIndex() throws Exception { > String json = "{" + > ""user":[0, {"x":1}]" + > "}"; > Client client = new TransportClient() > .addTransportAddress(new InetSocketTransportAddress( > "localhost", 9300)); > IndexResponse indexResponse = index(client, "a", "y", "{"x":" + json + > "}"); > logger.info(indexResponseToString(indexResponse)); > client.close(); > } > public static IndexResponse index(Client client, String index, String > type, String json) { > IndexResponse actionGet = client.prepareIndex(index, type) > .setSource(json).execute().actionGet(); > return actionGet; > } > > public static String indexResponseToString(IndexResponse actionGet) { > return actionGet.getIndex() + "/" + actionGet.getType() + "/" > > - actionGet.getId() + " isCreated: " + actionGet.isCreated(); > } > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9358cf2e-c72e-4455-9b45-e7b443a124db%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9358cf2e-c72e-4455-9b45-e7b443a124db%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 6, 2015, 5:57am UTC](https://discuss.elastic.co/t/need-advice-on-my-open-source-project-using-elastic-search-and-question-on-mapperparsingexception/22002/6 "2015-02-06T05:57:57Z")
It's valid in term of JS not for Elasticsearch. I mean that Elasticsearch needs to create a mapping for each field. Your field user can not be at the same time a number and an object. You can write this (pseudo JSON): User : { age : 10, Name: Joe } Or this: User : [{ age : 10, Name: Joe },{ age : 12, Name: John }] ## HTH David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 6 févr. 2015 à 01:26, Vicente de Rivera III [thirdy.derivera@gmail.com](mailto:thirdy.derivera@gmail.com) a écrit : > > Thank you for the response. > > Both [jsonlint.com](http://jsonlint.com) and [JSON Validator - JSONLint tool to validate JSON data](http://codebeautify.org/jsonvalidate) says this JSON is valid: > > { > "x": { > "user": [ > 0, > { > "x": 1 > } > ] > } > } > > I guess I'll parse that raw js data, build POJOs out of it and then pass these pojos into ES. (Was hoping I could dump this data into Elasticsearch without any trouble). > > Also, sorry to ask this, is Elastic search a good choice for my goals? I've never used any search technology before. > > > On Friday, February 6, 2015 at 6:02:40 AM UTC+8, David Pilato wrote: > > Unsure but your JSON looks incorrect to me. You are trying to send in an array a number 0 and an object. > > > > David > > > > > Le 5 févr. 2015 à 22:53, Vicente de Rivera III [thirdy....@gmail.com](mailto:thirdy....@gmail.com) a écrit : > > > > > > Code to reproduce the exception: > > > > > > @Test > > > public void testESIndex() throws Exception { > > > String json = "{" + > > > ""user":[0, {"x":1}]" + > > > "}"; > > > Client client = new TransportClient() > > > .addTransportAddress(new InetSocketTransportAddress( > > > "localhost", 9300)); > > > > > > ``` > > > IndexResponse indexResponse = index(client, "a", "y", "{\"x\":" + json + "}"); > > > > > > logger.info(indexResponseToString(indexResponse)); > > > > > > client.close(); > > > > > > ``` > > > > > > } > > > > > > public static IndexResponse index(Client client, String index, String type, String json) { > > > IndexResponse actionGet = client.prepareIndex(index, type) > > > .setSource(json).execute().actionGet(); > > > return actionGet; > > > } > > > > > > public static String indexResponseToString(IndexResponse actionGet) { > > > return actionGet.getIndex() + "/" + actionGet.getType() + "/" > > > + actionGet.getId() + " isCreated: " + actionGet.isCreated(); > > > } > > > > > > -- > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1b558488-1064-4056-b666-25e6226fdd66%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9358cf2e-c72e-4455-9b45-e7b443a124db%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9358cf2e-c72e-4455-9b45-e7b443a124db%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0AA64161-1385-4DDE-957B-8D7B63495A8D%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/0AA64161-1385-4DDE-957B-8D7B63495A8D%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:34am UTC](https://discuss.elastic.co/t/need-advice-on-my-open-source-project-using-elastic-search-and-question-on-mapperparsingexception/22002/7 "2017-07-06T00:34:34Z")