# Ingest Attachment Plugin update index **URL:** \ **Category:** Elasticsearch\ **Tags:** ingest-pipeline\ **Created:** [November 16, 2021, 4:11pm UTC](https://discuss.elastic.co/t/ingest-attachment-plugin-update-index/289346 "2021-11-16T16:11:04Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![fribeiro-keep](https://avatars.discourse-cdn.com/v4/letter/f/a3d4f5/32.png) [@fribeiro-keep](https://discuss.elastic.co/u/fribeiro-keep)\ **Post date:** [November 16, 2021, 4:11pm UTC](https://discuss.elastic.co/t/ingest-attachment-plugin-update-index/289346/1 "2021-11-16T16:11:04Z")
Hey there, I'm trying to implement the [Ingest Attachment plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/using-ingest-attachment.html) on Elasticsearch 6.8 My goal is to append the document text to an existing index. This is a sample of my current index ```auto { "_index": "koha_biblios", "_type": "data", "_id": "7289", "_version": 1, "_seq_no": 1390, "_primary_term": 1, "found": true, "_source": { "title": [ "SHIP PERFORMANCE" ] }, "author": [ "HUGHES, C. N." ], "itype": [ "MON" ] } ``` First I add a new mapping to the existing index ```auto PUT koha_biblios { "mappings" : { "data" : { "properties" : { "attachment.data" : { "type": "text", "analyzer" : "analyzer_standard" } } } } } ``` Then I create the pipeline ```auto PUT _ingest/pipeline/attachment { "description" : "Extract attachment information", "processors" : [ { "attachment" : { "field" : "data", "indexed_chars" : -1 } } ] } ``` And the when I submit the file to ingest ```auto PUT koha_biblios/data/7289?pipeline=attachment { "data": "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0=" } ``` I lost all the information and get only the document data ```auto { "_index": "koha_biblios", "_type": "data", "_id": "7289", "_version": 3, "_seq_no": 11142, "_primary_term": 1, "found": true, "_source": { "data": "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0=", "attachment": { "content_type": "application/rtf", "language": "ro", "content": "Lorem ipsum dolor sit amet", "content_length": 28 } } } ``` Is it possible to append the ingested document instead of replacing all the existing index? What am I missing? Best Regards, Filipe ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 16, 2021, 7:42pm UTC](https://discuss.elastic.co/t/ingest-attachment-plugin-update-index/289346/2 "2021-11-16T19:42:53Z")
This behavior is expected as you are calling the index API which index a new version of the document by overwriting the previous one if any. You can try [Update By Query API | Elasticsearch Guide [7.15] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html) which supports the `pipeline` parameter. ---
**Author:** ![fribeiro-keep](https://avatars.discourse-cdn.com/v4/letter/f/a3d4f5/32.png) [@fribeiro-keep](https://discuss.elastic.co/u/fribeiro-keep)\ **Post date:** [November 17, 2021, 11:09am UTC](https://discuss.elastic.co/t/ingest-attachment-plugin-update-index/289346/3 "2021-11-17T11:09:49Z")
@dadoonet thank you for your answer. Still have two questions. How can I update a certain document (by id) and how can I provide the file to the pipeline using Update By Query API? ```auto POST koha_biblios/_update_by_query?pipeline=attachment { "script": { "source": "ctx._source.attachment.data=e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0=", "lang": "painless" }, "query": { "term": { "_id": "7282" } } } ``` This gives me an error -\> illegal\_argument\_exception ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 17, 2021, 1:52pm UTC](https://discuss.elastic.co/t/ingest-attachment-plugin-update-index/289346/4 "2021-11-17T13:52:52Z")
Could you provide a full recreation script as described in [About the Elasticsearch category](https://discuss.elastic.co/t/about-the-elasticsearch-category/21). It will help to better understand what you are doing. Please, try to keep the example as simple as possible. A full reproduction script is something anyone can **copy and paste in Kibana dev console** , click on the run button to reproduce your use case. It will help readers to understand, reproduce and if needed fix your problem. It will also most likely help to get a faster answer. ---
**Author:** ![fribeiro-keep](https://avatars.discourse-cdn.com/v4/letter/f/a3d4f5/32.png) [@fribeiro-keep](https://discuss.elastic.co/u/fribeiro-keep)\ **Post date:** [November 17, 2021, 2:55pm UTC](https://discuss.elastic.co/t/ingest-attachment-plugin-update-index/289346/5 "2021-11-17T14:55:52Z")
I hope this helps ```auto DELETE biblios PUT biblios/data/1 { "date-of-acquisition": [ "2021-02-03" ], "title": [ "TÉCNICAS DE INDEXAÇÃO EM SISTEMAS DOCUMENTAIS AUTOMATIZADOS, 25-26 FEVEREIRO 1998", "OFERTA ANO 1999 (FH)" ] } PUT _ingest/pipeline/attachment { "description" : "Extract attachment information", "processors" : [ { "attachment" : { "field" : "data" } } ] } POST biblios/_update_by_query?pipeline=attachment { "script": { "source": "ctx._source.attachment='e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0='", "lang": "painless" }, "query": { "term": { "_id": "1" } } } ``` For some reason, you have to run the last POST individually in order to reproduce the error. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 17, 2021, 3:50pm UTC](https://discuss.elastic.co/t/ingest-attachment-plugin-update-index/289346/6 "2021-11-17T15:50:02Z")
Do this: ```auto DELETE biblios PUT biblios/_doc/1 { "date-of-acquisition": [ "2021-02-03" ], "title": [ "TÉCNICAS DE INDEXAÇÃO EM SISTEMAS DOCUMENTAIS AUTOMATIZADOS, 25-26 FEVEREIRO 1998", "OFERTA ANO 1999 (FH)" ] } PUT _ingest/pipeline/attachment { "description" : "Extract attachment information", "processors" : [ { "attachment" : { "field" : "data" } } ] } POST biblios/_update_by_query?pipeline=attachment { "script": { "source": "ctx._source.data='e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0='", "lang": "painless" }, "query": { "term": { "_id": "1" } } } GET biblios/_doc/1 ``` The main change is ``` "source": "ctx._source.attachment= ``` to ``` "source": "ctx._source.data= ``` ---
**Author:** ![fribeiro-keep](https://avatars.discourse-cdn.com/v4/letter/f/a3d4f5/32.png) [@fribeiro-keep](https://discuss.elastic.co/u/fribeiro-keep)\ **Post date:** [November 17, 2021, 4:00pm UTC](https://discuss.elastic.co/t/ingest-attachment-plugin-update-index/289346/7 "2021-11-17T16:00:56Z")
That's great! I now see all the previous data plus the ingested document. Thank you @dadoonet ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 15, 2021, 4:01pm UTC](https://discuss.elastic.co/t/ingest-attachment-plugin-update-index/289346/8 "2021-12-15T16:01:53Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.