# ElasticSearch SerializationError exception using Python API

**URL:** <https://discuss.elastic.co/t/elasticsearch-serializationerror-exception-using-python-api/82838>\
**Category:** Elasticsearch\
**Created:** [April 19, 2017, 8:18am UTC](https://discuss.elastic.co/t/elasticsearch-serializationerror-exception-using-python-api/82838 "2017-04-19T08:18:51Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dtrizna](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@dtrizna](https://discuss.elastic.co/u/dtrizna)\
**Post date:** [April 19, 2017, 8:18am UTC](https://discuss.elastic.co/t/elasticsearch-serializationerror-exception-using-python-api/82838/1 "2017-04-19T08:18:51Z")

</div>

Hi, community!

It looks like that I can't get with this case by my own. Looked through similar topics here, on Stack Overflow and over the google, but..

I'm trying to update some specific document within ES (in fact, it is Kibana search).  
I can do this, using update method within ES module and syntax, described here: [python - How to update a document using elasticsearch-py? - Stack Overflow](http://stackoverflow.com/questions/30598152/how-to-update-a-document-using-elasticsearch-py)

`es.update(index='.kibana', doc_type='search', id='2f015f70-1ec1-11e7-88fa-XXXXX', body={ 'some valid body' })`

All works fine, the document is updated and Kibana understand changes.  
Although, I'm not Ok with static body of the query. There's necessity to update it according to some previous script manipulations. As a result the syntax looks somehow like this:

`b = 'exaclty the same valid query'`  
`es.update(index='.kibana', doc_type='search', id='2f015f70-1ec1-11e7-88fa-XXXXXX', body={ b })`

Even with the same syntax the update operation fails with:

> elasticsearch.exceptions.SerializationError.  
> TypeError('Unable to serialize set(' HERE THE VALUE OF b VAR '))

As I understand, then Serialization is process of data format transformation, in this case string to JSON..?  
So I've tried to manipulate with b variable using json module, with no success. The same exception appears after the variable was led through `json.dumps(b)` and `json.loads(json.dumps(b))` operations.

Do you have any ideas, how to overcome this behavior?  
Thanks!

Dmitry

---

<div class="post-metadata">

**Author:** ![dtrizna](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@dtrizna](https://discuss.elastic.co/u/dtrizna)\
**Post date:** [April 20, 2017, 11:38am UTC](https://discuss.elastic.co/t/elasticsearch-serializationerror-exception-using-python-api/82838/2 "2017-04-20T11:38:36Z")

</div>

As there's no answer from community, assume that workaround I used in this case may be useful for someone else in the future.

Unfortunately, I still don't know how to mitigate this ES behavior using Python Elastic module.  
Next, what I've tried to do, is to use native ES Update API through HTTP using built in Python libraries like urllib\* and requests.

I've tried both following requests examples:

`requests.post('http://localhost:9200/.kibana/search/2f015f70-1ec1-11e7-88fa-XXXX/_update',data='some valid POST body')`  
and  
`requests.post('http://localhost:9200/.kibana/search/2f015f70-1ec1-11e7-88fa-XXXX/_update',json='some valid POST body')`

Although request was created with completely valid syntax (it worked within Dev Tools Console),  
ES returned another type of exception:

`(..)"type":"parse_exception","reason":"Failed to derive xcontent"(..)`

So in the end I was forced to make call to OS shell, execute `curl -XPOST <url> -d@<filename>` flag.  
Withing that file is placed that valid POST body, that in this case is sent to ES without any problems.

Code fragment:

```
post_body = <dynamicaly generated query, loong story>

with open('query.json', 'w') as file:
    file.writelines(post_body)

call = subprocess.Popen("curl -XPOST http://localhost:9200/.kibana/search/977e80c0-201d-XXXXX/_update?pretty=true -d@query.json",
                        shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(result, error) = call.communicate()

```

Only that way I've got the desired results..

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 18, 2017, 11:52am UTC](https://discuss.elastic.co/t/elasticsearch-serializationerror-exception-using-python-api/82838/3 "2017-05-18T11:52:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.