# Add dictionary to index
**URL:** https://discuss.elastic.co/t/add-dictionary-to-index/92415
**Category:** Elasticsearch
**Created:** [July 10, 2017, 7:08am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415 "2017-07-10T07:08:52Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 10, 2017, 7:08am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/1 "2017-07-10T07:08:52Z")
Hi,
Very simple and straight forward question:
Is there a way to push/insert a whole dictionary into my index pattern?
without iterate over the dictionary and push each object into my index pattern
Thx a lot 👍
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 10, 2017, 8:06am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/2 "2017-07-10T08:06:57Z")
> [@avivc](#):
>
> Is there a way to push/insert a whole dictionary into my index pattern?
We accept JSON as the format for documents which is obviously capable of representing arrays with complex objects.
However, by default the underlying index used for search is a flattened representation and if you need to match \>1 property of nested objects in your searches you may need to mark these objects as "nested" in your index schema definition. See [Nested Objects | Elasticsearch: The Definitive Guide [2.x] | Elastic](https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html)
---
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 10, 2017, 8:18am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/3 "2017-07-10T08:18:53Z")
Tnx @Mark_Harwood
The thing is that I'm delete and rebuild my index pattern every day automatically. I'm not sure that add my schema definition will help
When I'm trying to insert a dict to my index pattern, Elastic pop an error `"unsupported index mapping"`. Only iteration on the dict helps
Any idea?
Tnx
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 10, 2017, 8:25am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/4 "2017-07-10T08:25:21Z")
I think I need to see examples of the commands you're issuing and the errors you get back to understand more.
An "index pattern" is what Kibana calls the string that is used to configure access to one or more concrete indices e.g. they might query index pattern `logs*` but under the covers elasticsearch would access indices `logs20170101` and `logs20170102`. When you index content you refer to a single physical `index` not a pattern otherwise we wouldn't know which of the potentially multiple indices you want to store content in.
---
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 10, 2017, 8:38am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/5 "2017-07-10T08:38:01Z")
Tnx for the explanation ✌
I'm using Python as an intermediate script to fetch data from my API into ES
The code is:
> ```
> if res.status_code == 200:
> lbapi_Response_Data = lbapi_results['Results']
> del item['_id']
> elasticsearch.index(index="rally_look_back", doc_type="Defect", body=lbapi_Response_Data)
>
> ```
**lbapi\_Response\_Data** is a dict and usually I'm iterating over it object by object and push it into my index
When running to code above, I'm getting the error:
`raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)`
`RequestError: TransportError(400, u'mapper_parsing_exception', u'failed to parse')`
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 10, 2017, 8:56am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/6 "2017-07-10T08:56:56Z")
> [@avivc](#):
>
> lbapi\_Response\_Data
If you have Kibana installed you can use the "dev tools" to test raw elasticsearch interactions which can be easily shared on this forum. It also makes debugging easier if we can remove the client layers (in your case the Python client) from the equation. There's 2 parts to this - what you sort of thing you claim you will store and what sort of thing you actually then try store. The former is the mapping which you can get directly from the REST api (using Kibana dev tools or CURL as follows:)
```
GET rally_look_back/_mapping
```
To get the JSON that is being sent to be indexed I'd need to see the serialized form of `lbapi_Response_Data` if you can print that out
---
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 10, 2017, 9:15am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/7 "2017-07-10T09:15:11Z")
I'm using Kibana, but why to remove python?
See mapping [here](http://jsoneditoronline.org/?id=a4643108c32514298c5d17a8c2974e93)
`To get the JSON that is being sent to be indexed I'd need to see the serialized form of lbapi_Response_Data if you can print that out`
What do you mean?
**(How do you create reference to previous comment in your comment?)**
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 10, 2017, 9:20am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/8 "2017-07-10T09:20:47Z")
> [@avivc](#):
>
> why to remove python?
Python talks to the REST layer. Posting raw REST requests here means anyone can copy/paste the commands into Kibana or CURL to recreate your issue.
> [@avivc](#):
>
> What do you mean?
In your python code:
```
print lbapi_Response_Data
```
---
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 10, 2017, 9:30am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/10 "2017-07-10T09:30:17Z")
> [@Mark\_Harwood](#):
>
> Python talks to the REST layer. Posting raw REST requests here means anyone can copy/paste the commands into Kibana or CURL to recreate your issue.
It's sounds like security issue. What's it relevant to this topic?
> [@Mark\_Harwood](#):
>
> print lbapi\_Response\_Data
It's a bit problem due to internal data. Any chance I can describe the data structure?
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 10, 2017, 9:35am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/12 "2017-07-10T09:35:21Z")
> [@avivc](#):
>
> What's it relevant to this topic?
You need to supply an example people can reproduce. One command to create an index mapping. The other to insert some data into that index. Often going through the process of putting together a simplified example such as this will reveal the issue.
---
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 10, 2017, 9:37am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/13 "2017-07-10T09:37:56Z")
Got it !
My python is the tool that helps me communicate between the API and ES
What's the alternative?
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 10, 2017, 9:52am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/14 "2017-07-10T09:52:42Z")
> [@avivc](#):
>
> What's the alternative?
The Kibana dev tools (pictured) are one way. CURL is another.
Regardless, let's try another tack - can you share the stack trace from the server when you get the error?
---
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 12, 2017, 7:21am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/15 "2017-07-12T07:21:01Z")
Sure @Mark_Harwood
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 12, 2017, 7:30am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/16 "2017-07-12T07:30:11Z")
Looks like the client stack trace
---
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 12, 2017, 7:31am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/17 "2017-07-12T07:31:53Z")
> [@Mark\_Harwood](#):
>
> Looks like the client stack trace
My python script fetch data from the API and index it into my ES
That's all the component I'm using
This error occur when trying:
` es.index(index="rally_look_back", doc_type="Defect", body=lbapi_Response_Data)`
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 12, 2017, 8:41am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/18 "2017-07-12T08:41:38Z")
The server stack trace can be found in the "logs" directory of the elasticsearch server.
---
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 12, 2017, 9:18am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/19 "2017-07-12T09:18:06Z")
**Sorry can't attached the file due to file suffix**
[2017-07-12T09:43:01,665][INFO][o.e.c.m.MetaDataDeleteIndexService] [o0WbhJX] [rally\_look\_back/OcyGznwZQzeqADI5-uf0UQ] deleting index
[2017-07-12T10:21:50,466][INFO][o.e.c.m.MetaDataCreateIndexService] [o0WbhJX] [rally\_look\_back] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings []
[2017-07-12T10:21:50,547][DEBUG][o.e.a.b.TransportShardBulkAction] [o0WbhJX] [rally\_look\_back][1] failed to execute bulk item (index) BulkShardRequest [[rally\_look\_back][1]] containing [index {[rally\_look\_back][Defect][AV01rOjSwejzqYBhaG6D], source[n/a, actual length: [154.5kb], max length: 2kb]}]
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
at org.elasticsearch.index.mapper.DocumentParser.wrapInMapperParsingException(DocumentParser.java:176) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:69) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:277) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:536) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:513) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.bulk.TransportShardBulkAction.prepareIndexOperationOnPrimary(TransportShardBulkAction.java:450) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:458) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:143) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:113) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:69) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:939) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:908) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:113) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:322) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:264) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:888) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:885) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.index.shard.IndexShardOperationsLock.acquire(IndexShardOperationsLock.java:147) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationLock(IndexShard.java:1656) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:897) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction.access$400(TransportReplicationAction.java:93) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:281) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:260) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:252) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:627) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.4.2.jar:5.4.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0\_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0\_131]
at java.lang.Thread.run(Unknown Source) [?:1.8.0\_131]
Caused by: org.elasticsearch.common.compress.NotXContentException: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes
at org.elasticsearch.common.compress.CompressorFactory.compressor(CompressorFactory.java:58) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.common.xcontent.XContentHelper.createParser(XContentHelper.java:53) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:62) ~[elasticsearch-5.4.2.jar:5.4.2]
... 30 more
[2017-07-12T12:13:53,570][INFO][o.e.c.m.MetaDataDeleteIndexService] [o0WbhJX] [rally\_look\_back/dwtzT4qvTZCppcv3p6i2DA] deleting index
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 12, 2017, 9:37am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/20 "2017-07-12T09:37:10Z")
Looks like invalid JSON is being sent. Need to see the contents of `lbapi_Response_Data`
---
### Author:  [@avivc](https://discuss.elastic.co/u/avivc)
#### Post date: [July 12, 2017, 9:40am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/21 "2017-07-12T09:40:33Z")
If JSON is invalid, how come that I can add obejcts into my index if I iterate over `lbapi_Response_Data` ?
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 12, 2017, 9:43am UTC](https://discuss.elastic.co/t/add-dictionary-to-index/92415/22 "2017-07-12T09:43:51Z")
Because it's an array not a dict?
[Next page](https://discuss.elastic.co/t/add-dictionary-to-index/92415.md?page=2)