Failed to decode ndjson new format with python

Hi here,
Thanks in advance for your time and help

I can find on the web couple of examples using python scripts to "decode" data from the Kibana API with json
Such as :

dashboard = requests.get('http://' + hostname + ':' + port + '/.kibana/dashboard/'+ sys.argv[1] + '?pretty')
json_dashboard = dashboard.json()

With the new format ndjson, I adapted my code as below :

import ndjson
import requests

response ='')
items = response.json(cls=ndjson.Decoder)

And it works fine in most of the cases !
But for few visualizations I'm getting :
Traceback (most recent call last): File "./script", line 181, in <module> main(clientId, menuId) File "./script", line 146, in main showIds(clientInfo["ClientKibanaPort"],user_choice) File "./script", line 72, in showIds items = response.json(cls=ndjson.Decoder) File "/usr/lib/python2.7/site-packages/requests/", line 818, in json self.content.decode(encoding), **kwargs File "/usr/lib64/python2.7/site-packages/simplejson/", line 533, in loads return cls(encoding=encoding, **kw).decode(s) File "/usr/lib/python2.7/site-packages/ndjson/", line 8, in decode text = '[{}]'.format(lines) UnicodeEncodeError: 'ascii' codec can't encode characters in position 44598-44606: ordinal not in range(128)

I've tried to convert my 'response' to utf8 like that :

text = ndjson.dumps()

But afterwards I've got :

Traceback (most recent call last): File "./script", line 188, in <module> main(clientId, menuId) File "./script", line 153, in main showIds(clientInfo["ClientKibanaPort"],user_choice) File "./script", line 73, in showIds text = ndjson.dumps(response) File "/usr/lib/python2.7/site-packages/ndjson/", line 26, in dumps return json.dumps(*args, **kwargs) File "/usr/lib64/python2.7/json/", line 250, in dumps sort_keys=sort_keys, **kw).encode(obj) File "/usr/lib/python2.7/site-packages/ndjson/", line 16, in encode line = super(Encoder, self).encode(each, *args, **kwargs) File "/usr/lib64/python2.7/json/", line 201, in encode return encode_basestring_ascii(o) UnicodeDecodeError: 'utf8' codec can't decode byte 0xc2 in position 127: unexpected end of data

I know isn't completely related to Kibana but other users might experience this kind of issue I presume


Thanks for sharing this here, Guillaume! Have you reported this bug to the ndjson package maintainers? Can we rule out that this is a bug with that package? If you can help me understand that this is a bug on our side and not ndjson then I'd like to know what you think is causing this error?

Hi @cjcenizal

Been working on it yesterday - I actually found out at least on my problems

Can't really figure out what happened but in few visualizations it appears that I had "­­­­­­­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­" in a custom label. (it doesn't seem to be spaces in a row but another character ...)

Well, once the export done, the custom label looks like that :

    \"customLabel\":\"­­­­­­­­ ­  ­  ­  ­  ­  ­  ­  ­  ­  ­  ­  ­\",

Which seems to be converted by spaces too by the browser but it actually looks like :

on my NotePad

and like :


on my output console

Would you have any idea of what is this ?


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.