Ingest-attachment using CBOR examples

[twigbranch]

Need some clarification regarding the following thread:

Ingest-attachment using CBOR example

Hello, sorry for repeating this question which IMO was left unanswered...: Example for ingest attachment plug in with CBOR format for indexing documents I too would really like to see an example. We are struggling indexing large documents - and currently have no idea how to use "CBOR...without using json" as specified in the docs here: https://www.elastic.co/guide/en/elasticsearch/plugins/5.2/ingest-attachment.html

Alex says:
The trick is to just use the cbor builder to create your content, and add a bytearray to the field, you want to process instead of a base64 encoded string.

What exactly is meant by "add a bytearray to the field?"

I have been mapping an 'attachment-data' field with type 'keyword' and writing to that a base64 encoded string through the pipeline. I'm using ruby client library to encode the data.

Now, I'm trying to cbor the attachment data, but it's not clear what type the 'attachment-data' field should be. When I use 'keyword' or 'binary', it seems to be expecting base64 encoded data.

[twigbranch]

Any ideas?

With the attachment pipeline field set to 'attachment_data' with type 'keyword', I'm trying to set the 'Content-Type' to 'application/cbor' and the body to a CBORed( { attachment_data: } ) but I get error:

[2017-07-19T19:44:53,404][DEBUG][o.e.a.b.TransportBulkAction] [J_rrNYb] failed to execute pipeline [attachment] for document [my_index/my_type/my_id]
org.elasticsearch.ElasticsearchParseException: Failed to parse content to map
	at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:143) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:114) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.action.index.IndexRequest.sourceAsMap(IndexRequest.java:355) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.ingest.PipelineExecutionService.innerExecute(PipelineExecutionService.java:164) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.ingest.PipelineExecutionService.access$000(PipelineExecutionService.java:41) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.ingest.PipelineExecutionService$2.doRun(PipelineExecutionService.java:88) [elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.5.0.jar:5.5.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_31]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_31]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_31]
Caused by: com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 middle byte 0xec
 at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@474770f0; line: -1, column: 32]
	at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1702) ~[jackson-core-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:558) ~[jackson-core-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.dataformat.cbor.CBORParser._reportInvalidOther(CBORParser.java:3152) ~[jackson-dataformat-cbor-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.dataformat.cbor.CBORParser._reportInvalidOther(CBORParser.java:3157) ~[jackson-dataformat-cbor-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.dataformat.cbor.CBORParser._finishLongText(CBORParser.java:2062) ~[jackson-dataformat-cbor-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.dataformat.cbor.CBORParser._finishTextToken(CBORParser.java:1973) ~[jackson-dataformat-cbor-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.dataformat.cbor.CBORParser.getText(CBORParser.java:1353) ~[jackson-dataformat-cbor-2.8.6.jar:2.8.6]
	at org.elasticsearch.common.xcontent.json.JsonXContentParser.text(JsonXContentParser.java:86) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readValue(AbstractXContentParser.java:369) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMap(AbstractXContentParser.java:317) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMap(AbstractXContentParser.java:280) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.common.xcontent.support.AbstractXContentParser.map(AbstractXContentParser.java:235) ~[elasticsearch-5.5.0.jar:5.5.0]
	at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:141) ~[elasticsearch-5.5.0.jar:5.5.0]
	... 10 more

What should the attachment processor 'field' type be?
What should the index request body contain?
What should the content-type be?