# Exception on ingest an attachment using client.index()

**URL:** <https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641>\
**Category:** Elasticsearch\
**Created:** [August 3, 2019, 4:01pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641 "2019-08-03T16:01:16Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![n2nelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/n2nelastic/32/51656_2.png) [@n2nelastic](https://discuss.elastic.co/u/n2nelastic)\
**Post date:** [August 3, 2019, 4:01pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/1 "2019-08-03T16:01:16Z")

</div>

I have properly set up an index and pipeline to ingest base64 encoded documents.

Using the elasticsearch client, I call client.index() to process a base64 document, but I get "ResponseError: Exception" from client.index()

```auto
// KIBANA
PUT /descriptions
PUT _ingest/pipeline/attachment
{
  "description" : "Extract attachment information",
  "processors" : [
    {
      "attachment" : {
        "field" : "data"
      }
    }
  ]
}

// NODE.JS code
// The 'data' variable is base64 encoded
function indexBase64Data(data) {

    console.log(" ");
    console.log("calling client.index()");
    const index = client.index({
        id: 101,
        index: "descriptions",
        pipeline: "attachment",
        body: {
            "data": data
        }
    });
    index.then(() => {
        console.log("Index was successful...");
    },
    (error) => {
        console.error("ERROR in client.index: " + error);
    })

}

// ERROR from client.index() is: "ResponseError: Exception"

```

Should I be using something other than client.index()??  
What am I doing wrong here?  
-David

---

<div class="post-metadata">

**Author:** ![n2nelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/n2nelastic/32/51656_2.png) [@n2nelastic](https://discuss.elastic.co/u/n2nelastic)\
**Post date:** [August 4, 2019, 2:39pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/2 "2019-08-04T14:39:48Z")

</div>

Maybe I should put my question a different way.

What is the **Elasticsearch Node.js Client** call to _ingest_ a base64 encoded document/attachment into an existing index (here "descriptions").

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 5, 2019, 10:00am UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/3 "2019-08-05T10:00:27Z")

</div>

I am sure the responseerror is more verbose than just `Exception` - can you please share the full error message? You can also increase debugging by [enabling stacktraces](https://www.elastic.co/guide/en/elasticsearch/reference/7.2/common-options.html#common-options-error-options).

Did you install the field mapper attachment plugin before doing this?

---

<div class="post-metadata">

**Author:** ![n2nelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/n2nelastic/32/51656_2.png) [@n2nelastic](https://discuss.elastic.co/u/n2nelastic)\
**Post date:** [August 5, 2019, 3:02pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/4 "2019-08-05T15:02:45Z")

</div>

Hi Alexander!

Adding "error\_trace: true" to my call, the elasticsearch.log:

```auto

[2019-08-05T09:09:40,350][DEBUG][o.e.a.b.TransportBulkAction] [DESKTOP-SBLSQL0] failed to execute pipeline [attachment] for document [descriptions/_doc/101]
org.elasticsearch.ElasticsearchException: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Illegal base64 character 3a
at org.elasticsearch.ingest.CompoundProcessor.newCompoundProcessorException(CompoundProcessor.java:194) ~[elasticsearch-7.3.0.jar:7.3.0]
at org.elasticsearch.ingest.CompoundProcessor.execute(CompoundProcessor.java:133) ~[elasticsearch-7.3.0.jar:7.3.0]
at org.elasticsearch.ingest.Pipeline.execute(Pipeline.java:100) ~[elasticsearch-7.3.0.jar:7.3.0]
at org.elasticsearch.ingest.IngestService.innerExecute(IngestService.java:427) ~[elasticsearch-7.3.0.jar:7.3.0]
at org.elasticsearch.ingest.IngestService.access$100(IngestService.java:70) ~[elasticsearch-7.3.0.jar:7.3.0]
at org.elasticsearch.ingest.IngestService$3.doRun(IngestService.java:355) [elasticsearch-7.3.0.jar:7.3.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.3.0.jar:7.3.0]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.0.jar:7.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Illegal base64 character 3a
... 11 more
Caused by: java.lang.IllegalArgumentException: Illegal base64 character 3a
at java.util.Base64$Decoder.decode0(Base64.java:743) ~[?:?]
at java.util.Base64$Decoder.decode(Base64.java:535) ~[?:?]
at java.util.Base64$Decoder.decode(Base64.java:558) ~[?:?]
at org.elasticsearch.ingest.IngestDocument.getFieldValueAsBytes(IngestDocument.java:183) ~[elasticsearch-7.3.0.jar:7.3.0]
at org.elasticsearch.ingest.attachment.AttachmentProcessor.execute(AttachmentProcessor.java:79) ~[?:?]
at org.elasticsearch.ingest.CompoundProcessor.execute(CompoundProcessor.java:123) ~[elasticsearch-7.3.0.jar:7.3.0]
... 9 more

```

Also, I tried looking for the plugin you suggested by Googling [elastic.co](http://elastic.co/), and there is no "field mapper attachment plugin" that I can find, only the following:

Mapper plugins:

- Mapper Size Plugin
- Mapper Murmur3 Plugin
- Mapper Annotated Plugin

Please continue to help me on this.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 5, 2019, 3:13pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/5 "2019-08-05T15:13:42Z")

</div>

I believe [this is the plugin you need](https://www.elastic.co/guide/en/elasticsearch/plugins/current/ingest-attachment.html). This processor is used within an ingest pipeline.

---

<div class="post-metadata">

**Author:** ![n2nelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/n2nelastic/32/51656_2.png) [@n2nelastic](https://discuss.elastic.co/u/n2nelastic)\
**Post date:** [August 5, 2019, 8:25pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/6 "2019-08-05T20:25:28Z")

</div>

Hi Christian. I already do have this plugin installed. I suppose I should have mentioned it, but I thought it would be obvious since I am using a pipeline.

```auto
// Kibana
GET _ingest/pipeline

{
  "attachment" : {
    "description" : "Extract attachment information",
    "processors" : [
      {
        "attachment" : {
          "field" : "data"
        }
      }
    ]
  },
  "xpack_monitoring_6" : {
    "description" : "This pipeline upgrades documents from the older version of the Monitoring API to the newer version (7) by fixing breaking changes in those older documents before they are indexed from the older version (6).",
    "version" : 7000199,
    "processors" : [
      {
        "script" : {
          "source" : "ctx._type = null"
        }
      },
      {
        "gsub" : {
          "field" : "_index",
          "pattern" : "(.monitoring-\\w+-)6(-.+)",
          "replacement" : "$17$2"
        }
      }
    ]
  },
  "xpack_monitoring_7" : {
    "description" : "This is a placeholder pipeline for Monitoring API version 7 so that future versions may fix breaking changes.",
    "version" : 7000199,
    "processors" : []
  }
}

```

Let me ask you this: is my approach using client.index() from the Elasticsearch Node.js Client correct in your opinion?

---

<div class="post-metadata">

**Author:** ![n2nelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/n2nelastic/32/51656_2.png) [@n2nelastic](https://discuss.elastic.co/u/n2nelastic)\
**Post date:** [August 5, 2019, 8:36pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/7 "2019-08-05T20:36:19Z")

</div>

Alexander,

Also I should have mentioned that "ResponseError: Exception" is in fact all that is returned. Nothing more.

Adding "error\_trace: true" to the client.index() call returned a lot, and I added part of the log file earlier.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 6, 2019, 7:58am UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/8 "2019-08-06T07:58:40Z")

</div>

a part of the exception states

> [@n2nelastic](#):
>
> Caused by: java.lang.IllegalArgumentException: Illegal base64 character 3a

can you share the original request including the base64 string?

---

<div class="post-metadata">

**Author:** ![n2nelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/n2nelastic/32/51656_2.png) [@n2nelastic](https://discuss.elastic.co/u/n2nelastic)\
**Post date:** [August 6, 2019, 1:21pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/9 "2019-08-06T13:21:19Z")

</div>

The base64 string:

```auto
DATA> data:application/octet-stream;base64,Ym9vazoxfGFiY3xkZWYKYm9vazoyfE1ha2luZyB0aGUgTW9kZXJuIFdvcmxkfFZhY2xhdiBTbWlsCmJvb2s6M3xUYXAgRGFuY2luZyB0byBXb3JrOiBXYXJyZW4gQnVmZmV0fENhcm9sIExvb21pcwpib29rOjZ8VGhlIEhlYWRzcGFjZSBHdWlkZSB0byBNZWRpdGF0aW9uIGFuZCBNaW5kZnVsbmVzc3xBbmR5IFB1ZGRpY29tYmUKYm9vazo3fDIxIExlc3NvbnMgZm9yIHRoZSAyMXN0IENlbnR1cnl8WXV2YWwgTm9haCBIYXJhcmkKYm9vazo4fEJhZCBCbG9vZHxKb2huIENhcnJleXJvdQpib29rOjl8QXJteSBvZiBOb25lOiBBdXRvbm9tb3VzIFdlYXBvbnMgYW5kIHRoZSBGdXR1cmUgb2YgV2FyfFBhdWwgU2NoYXJyZQpib29rOjEwfEVkdWNhdGVkfFRhcmEgV2VzdG92ZXIK

```

Original request to client.index() using "Elasticsearch Node.js Client"

```auto
function indexBase64Data(data) {

    console.log("DATA> " + data);
    console.log(" ");
    console.log("calling client.index()");
    const index = client.index({
        id: 102,
        index: "descriptions",
        pipeline: "attachment",
        error_trace: true,
        body: {
            "data": data
        }
    });
    index.then(() => {
        console.log("Index was successful...");
    },
    (error) => {
        console.log("index> " + index);
        console.error("ERROR in client.index: " + error);
    })

}

```

The elasticsearch.log file:

```auto
[2019-08-06T08:08:58,619][DEBUG][o.e.a.b.TransportBulkAction] [DESKTOP-SBLSQL0] failed to execute pipeline [attachment] for document [descriptions/_doc/101]
org.elasticsearch.ElasticsearchException: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Illegal base64 character 3a
	at org.elasticsearch.ingest.CompoundProcessor.newCompoundProcessorException(CompoundProcessor.java:194) ~[elasticsearch-7.3.0.jar:7.3.0]
	at org.elasticsearch.ingest.CompoundProcessor.execute(CompoundProcessor.java:133) ~[elasticsearch-7.3.0.jar:7.3.0]
	at org.elasticsearch.ingest.Pipeline.execute(Pipeline.java:100) ~[elasticsearch-7.3.0.jar:7.3.0]
	at org.elasticsearch.ingest.IngestService.innerExecute(IngestService.java:427) ~[elasticsearch-7.3.0.jar:7.3.0]
	at org.elasticsearch.ingest.IngestService.access$100(IngestService.java:70) ~[elasticsearch-7.3.0.jar:7.3.0]
	at org.elasticsearch.ingest.IngestService$3.doRun(IngestService.java:355) [elasticsearch-7.3.0.jar:7.3.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.3.0.jar:7.3.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.0.jar:7.3.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Illegal base64 character 3a
	... 11 more
Caused by: java.lang.IllegalArgumentException: Illegal base64 character 3a
	at java.util.Base64$Decoder.decode0(Base64.java:743) ~[?:?]
	at java.util.Base64$Decoder.decode(Base64.java:535) ~[?:?]
	at java.util.Base64$Decoder.decode(Base64.java:558) ~[?:?]
	at org.elasticsearch.ingest.IngestDocument.getFieldValueAsBytes(IngestDocument.java:183) ~[elasticsearch-7.3.0.jar:7.3.0]
	at org.elasticsearch.ingest.attachment.AttachmentProcessor.execute(AttachmentProcessor.java:79) ~[?:?]
	at org.elasticsearch.ingest.CompoundProcessor.execute(CompoundProcessor.java:123) ~[elasticsearch-7.3.0.jar:7.3.0]
	... 10 more

```

And as you can see, the ingest plugin is installed:

```auto
// KIBANA
GET /_cat/plugins?v

// Results in:
name component version
DESKTOP-SBLSQL0 ingest-attachment 7.3.0

```

THANK YOU!!! Alexander for continuing to help me on this.  
I especially like the Elasticsearch Node.js Client, and I want to learn how to ingest base64 encoded files very much indeed.

- David

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 6, 2019, 9:32pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/10 "2019-08-06T21:32:22Z")

</div>

try removing the prefix, marking this as application/octet-stream and just go with the raw base 64 string

> [@n2nelastic](#):
>
> data:application/octet-stream;base64,

---

<div class="post-metadata">

**Author:** ![n2nelastic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/n2nelastic/32/51656_2.png) [@n2nelastic](https://discuss.elastic.co/u/n2nelastic)\
**Post date:** [August 6, 2019, 11:48pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/11 "2019-08-06T23:48:02Z")

</div>

Yes Alexander... that worked.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 3, 2019, 11:57pm UTC](https://discuss.elastic.co/t/exception-on-ingest-an-attachment-using-client-index/193641/12 "2019-09-03T23:57:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.