# Ingest-attachment using CBOR example **URL:** \ **Category:** Elasticsearch\ **Created:** [March 22, 2017, 5:26pm UTC](https://discuss.elastic.co/t/ingest-attachment-using-cbor-example/79626 "2017-03-22T17:26:35Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![markbaumgarten](https://avatars.discourse-cdn.com/v4/letter/m/74df32/32.png) [@markbaumgarten](https://discuss.elastic.co/u/markbaumgarten)\ **Post date:** [March 22, 2017, 5:26pm UTC](https://discuss.elastic.co/t/ingest-attachment-using-cbor-example/79626/1 "2017-03-22T17:26:35Z")
Hello, sorry for repeating this question which IMO was left unanswered...: [Example for ingest attachment plug in with CBOR format for indexing documents](https://discuss.elastic.co/t/example-for-ingest-attachment-plug-in-with-cbor-format-for-indexing-documents/66443) - I too would really like to see an example. We are struggling indexing large documents - and currently have no idea how to use "CBOR...without using json" as specified in the docs here: [https://www.elastic.co/guide/en/elasticsearch/plugins/5.2/ingest-attachment.html](https://www.elastic.co/guide/en/elasticsearch/plugins/5.2/ingest-attachment.html) ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 22, 2017, 5:43pm UTC](https://discuss.elastic.co/t/ingest-attachment-using-cbor-example/79626/2 "2017-03-22T17:43:52Z")
@spinscale do you know? ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [March 22, 2017, 10:45pm UTC](https://discuss.elastic.co/t/ingest-attachment-using-cbor-example/79626/3 "2017-03-22T22:45:17Z")
Hey, you can use the built-in xcontent builder to create CBOR data, see the following links for some help - [CborXContentParserTests](https://github.com/elastic/elasticsearch/blob/master/core/src/test/java/org/elasticsearch/common/xcontent/cbor/CborXContentParserTests.java) - [Attachment PR from back then](https://github.com/elastic/elasticsearch/pull/16601) - [wikipedia list of cbor clients](https://en.wikipedia.org/wiki/CBOR) The trick is to just use the cbor builder to create your content, and add a bytearray to the field, you want to process instead of a base64 encoded string. Hope this helps a bit. --Alex ---
**Author:** ![markbaumgarten](https://avatars.discourse-cdn.com/v4/letter/m/74df32/32.png) [@markbaumgarten](https://discuss.elastic.co/u/markbaumgarten)\ **Post date:** [March 24, 2017, 10:32am UTC](https://discuss.elastic.co/t/ingest-attachment-using-cbor-example/79626/4 "2017-03-24T10:32:03Z")
Thanks for your kind replies, but I'm still stuck with this issue. This is how I fail with CBOR using python(in case someone could help): 1. Assuming I need to have a pipeline set up - so I start with curl here: ```auto curl -XPUT 'localhost:9200/_ingest/pipeline/attachment?pretty' -H 'Content-Type: application/json' -d' { "description" : "Extract attachment information", "processors" : [ { "attachment" : { "field" : "data" } } ] } ' ``` 1. Here's my python code, which reads an .odt file, cborcifies it, and then tries to index it: ```auto import elasticsearch import cbor2 from elasticsearch import Elasticsearch es = Elasticsearch() es.indices.delete(index="test-index", ignore=[400, 404]) filename = 'forf.odt' with open(filename, 'rb') as f: doc = { 'data': cbor2.loads(f.read()) } res = es.index(index="test-index", doc_type='tweet', id=1, body=doc, pipeline='attachment') ``` 1. The error from the client side is: python test\_cbor\_es.py Traceback (most recent call last): File "test\_cbor\_es.py", line 13, in res = es.index(index="test-index", doc\_type='tweet', id=1, body=doc, pipeline='attachment') File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 73, in \_wrapped return func(\*args, params=params, \*\*kwargs) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/ **init**.py", line 300, in index \_make\_path(index, doc\_type, id), params=params, body=body) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 318, in perform\_request status, headers, data = connection.perform\_request(method, url, params, body, ignore=ignore, timeout=timeout) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http\_urllib3.py", line 127, in perform\_request self.log\_request\_fail(method, full\_url, url, body, duration, response.status, raw\_data) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 100, in log\_request\_fail body = body.decode('utf-8') File "/usr/lib/python2.7/encodings/utf\_8.py", line 16, in decode return codecs.utf\_8\_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0xc6 in position 60: invalid continuation byte 1. The es server error: [2017-03-24T10:29:48,466][ERROR][o.e.a.i.TransportIndexAction] [8\_gffZR] failed to execute pipeline [attachment] elasticsearch1 | org.elasticsearch.ElasticsearchParseException: Failed to parse content to map elasticsearch1 | at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:107) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:78) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.action.index.IndexRequest.sourceAsMap(IndexRequest.java:410) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.ingest.PipelineExecutionService.innerExecute(PipelineExecutionService.java:164) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.ingest.PipelineExecutionService.access$000(PipelineExecutionService.java:41) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.ingest.PipelineExecutionService$1.doRun(PipelineExecutionService.java:65) [elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596) [elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0\_121] elasticsearch1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0\_121] elasticsearch1 | at java.lang.Thread.run(Thread.java:745) [?:1.8.0\_121] elasticsearch1 | Caused by: com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 middle byte 0x32 elasticsearch1 | at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@892135f; line: 1, column: 63] elasticsearch1 | at com.fasterxml.jackson.core.JsonParser.\_constructError(JsonParser.java:1702) ~[jackson-core-2.8.6.jar:2.8.6] elasticsearch1 | at com.fasterxml.jackson.core.base.ParserMinimalBase.\_reportError(ParserMinimalBase.java:558) ~[jackson-core-2.8.6.jar:2.8.6] elasticsearch1 | at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.\_reportInvalidOther(UTF8StreamJsonParser.java:3550) ~[jackson-core-2.8.6.jar:2.8.6] elasticsearch1 | at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.\_reportInvalidOther(UTF8StreamJsonParser.java:3557) ~[jackson-core-2.8.6.jar:2.8.6] elasticsearch1 | at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.\_decodeUtf8\_2(UTF8StreamJsonParser.java:3327) ~[jackson-core-2.8.6.jar:2.8.6] elasticsearch1 | at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.\_finishString2(UTF8StreamJsonParser.java:2517) ~[jackson-core-2.8.6.jar:2.8.6] elasticsearch1 | at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.\_finishAndReturnString(UTF8StreamJsonParser.java:2469) ~[jackson-core-2.8.6.jar:2.8.6] elasticsearch1 | at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:315) ~[jackson-core-2.8.6.jar:2.8.6] elasticsearch1 | at org.elasticsearch.common.xcontent.json.JsonXContentParser.text(JsonXContentParser.java:86) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readValue(AbstractXContentParser.java:352) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMap(AbstractXContentParser.java:300) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMap(AbstractXContentParser.java:263) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.common.xcontent.support.AbstractXContentParser.map(AbstractXContentParser.java:218) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:105) ~[elasticsearch-5.2.2.jar:5.2.2] elasticsearch1 | ... 10 more ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 24, 2017, 10:46am UTC](https://discuss.elastic.co/t/ingest-attachment-using-cbor-example/79626/5 "2017-03-24T10:46:57Z")
Please format your code using `` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21). It will make your post more readable. Or use markdown style like: ```` ``` CODE ``` ```` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 21, 2017, 10:47am UTC](https://discuss.elastic.co/t/ingest-attachment-using-cbor-example/79626/6 "2017-04-21T10:47:20Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.