Ingest attachment java ES

Elastic Stack Elasticsearch
1

Hi ,

I am using ES 5.6.3 and a custom application to index the file system content.

In my index type, some fields needs to be stored as normal fields and file content needs to use ingest attachment pipeline.

PUT _ingest/pipeline/attachment { "description" : "Extract attachment information", "processors" : [ { "attachment" : { "field" : "content", "properties": [ "content", "title" ] } } ] }

It means a record will have string fields as well as base64 binary fields from file system. The code is as below

 Map<String, String> emp = new HashMap<>();


String str1 = "test";
String str2 = Base64.getEncoder().encodeToString("parsed content".getBytes());

emp.add("meta", str1);
emp.add("content", str2);

IndexRequest ir = new IndexRequest("employee", "details", "1");

ir.source(emp).setPipeline("attachment");  

BulkRequestBuilder bulkBuilder = client.prepareBulk();
bulkBuilder.add(ir);		
BulkResponse bulkRes = bulkBuilder.execute().actionGet();

The record is not getting indexed.

I need both normal and binary field to be available in all records.

Please provide your suggestions

2

hey.

if you are using two hashmaps (emp1 and emp2) because those should be different entities, then you also need to have two different IndexRequests, one for each employee, and add both to your bulk request.

hope this helps.

--Alex

3

Thanks for your reply. Both emp1 and emp2 are fields of same record with id1. Creating two index requests will replace the first record again.

6

@spinscale any help on this case?

7

This is a community forum and does not come with SLAs. Asking after one day of inactivity rather makes me stop to respond to a thread at all. If you need proper response times, there is commercial support.

You can only call source() once in your code, every new call overwrites the old one. You need to merge your data before that.

8

Sorry bothering. Found the issue. Earlier in the index I have indexed it as normal content for the Id 1. When I tried to update the document with Id 1 as binary field for the content field. It was not updating the document. I indexed with document Id 2 and got indexed successfully.

This post can be closed.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.