# Errors indexing binary attachment using ingest-attachment pipeline

**URL:** <https://discuss.elastic.co/t/errors-indexing-binary-attachment-using-ingest-attachment-pipeline/300766>\
**Category:** Elasticsearch\
**Tags:** ingest-pipeline\
**Created:** [March 26, 2022, 5:37pm UTC](https://discuss.elastic.co/t/errors-indexing-binary-attachment-using-ingest-attachment-pipeline/300766 "2022-03-26T17:37:43Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Enxiad](https://avatars.discourse-cdn.com/v4/letter/e/bb73d2/32.png) [@Enxiad](https://discuss.elastic.co/u/Enxiad)\
**Post date:** [March 26, 2022, 5:37pm UTC](https://discuss.elastic.co/t/errors-indexing-binary-attachment-using-ingest-attachment-pipeline/300766/1 "2022-03-26T17:37:44Z")

</div>

Hi all, I am ingesting a document A and I receive the error: " **Last unit does not have enough valid bits**" when the document contains only the word "Lorem".

For document B which contains the words "Lorem ipsum" then I get the error " **Illegal base64 character 20**" because of the space.

How do I approach fixing these errors and also enable indexing a document that has spaces?

Content of document A:

```auto
Lorem

```

CBOR encoded data for A:

```auto
\xa1\x64\x64\x61\x74\x61\x65\x4c\x6f\x72\x65\x6d

```

Diagnostic decoding of encoded data A:

```auto
{"data": "Lorem"}

```

Content of binary document B:

```auto
Lorem ipsum

```

CBOR encoded data for B:

```auto
\xa1\x64\x64\x61\x74\x61\x6b\x4c\x6f\x72\x65\x6d\x20\x69\x70\x73\x75\x6d

```

Diagnostic decoding of encoded data B:

```auto
{"data": "Lorem ipsum"}

```

Create ES index

```auto
curl -X PUT "localhost:9200/example?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index": {
      "number_of_shards": 1,  
      "number_of_replicas": 0 
    }
  },
  "mappings": {
    "properties": {
      "data": {
        "type": "binary"
      }
    }
  }
}'

```

Create ES ingest attachment pipeline

```auto
curl -X PUT "localhost:9200/_ingest/pipeline/cbor-attachment?pretty" -H 'Content-Type: application/json' -d'
{
  "description" : "Extract attachment information encoded in CBOR",
  "processors" : [
    {
      "attachment": {
        "description" : "Ingest attachment",
        "field": "data",
        "indexed_chars": -1
      }
    }
  ]
}'

```

Ingest document A

```auto
echo -e '\xa1\x64\x64\x61\x74\x61\x65\x4c\x6f\x72\x65\x6d' | \
    curl -X PUT \
       "localhost:9200/example/_doc/test1?pipeline=cbor-attachment" \
       -H 'Content-Type: application/cbor' --data-binary @-

```

**Error ingesting document A**

```auto
illegal_argument_exception: Last unit does not have enough valid bits

```

Ingest document B

```auto
echo -e '\xa1\x64\x64\x61\x74\x61\x6b\x4c\x6f\x72\x65\x6d\x20\x69\x70\x73\x75\x6d' | \
    curl -X PUT \
       "localhost:9200/example/_doc/test2?pipeline=cbor-attachment" \
       -H 'Content-Type: application/cbor' --data-binary @-

```

**Error ingesting document B**

```auto
illegal_argument_exception: illegal base64 character 20

```

---

<div class="post-metadata">

**Author:** ![Enxiad](https://avatars.discourse-cdn.com/v4/letter/e/bb73d2/32.png) [@Enxiad](https://discuss.elastic.co/u/Enxiad)\
**Post date:** [March 26, 2022, 11:49pm UTC](https://discuss.elastic.co/t/errors-indexing-binary-attachment-using-ingest-attachment-pipeline/300766/2 "2022-03-26T23:49:09Z")

</div>

Any ideas, please @dadoonet @spinscale @Badger @rugenl or anyone else experienced with Elasticsearch? Can't find anything in the docs about these two issues.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 27, 2022, 12:05am UTC](https://discuss.elastic.co/t/errors-indexing-binary-attachment-using-ingest-attachment-pipeline/300766/3 "2022-03-27T00:05:11Z")

</div>

Please be patient in waiting for responses to your question and refrain from pinging multiple times asking for a response or opening multiple topics for the same question. This is a community forum, it may take time for someone to reply to your question. For more information please refer to the [Community Code of Conduct](https://www.elastic.co/community/codeofconduct) specifically the section "Be patient". Also, please refrain from pinging folks directly, this is a forum and anyone that participates might be able to assist you.

If you are in need of a service with an SLA that covers response times for questions then you may want to consider talking to us about a [subscription](https://www.elastic.co/subscriptions).

It's fine to answer on your own thread after 2 or 3 days (not including weekends) if you don't have an answer.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 27, 2022, 12:06am UTC](https://discuss.elastic.co/t/errors-indexing-binary-attachment-using-ingest-attachment-pipeline/300766/4 "2022-03-27T00:06:38Z")

</div>

I never used cbor but I'm using BASE64 Json document.  
I don't think I can help.

---

<div class="post-metadata">

**Author:** ![Enxiad](https://avatars.discourse-cdn.com/v4/letter/e/bb73d2/32.png) [@Enxiad](https://discuss.elastic.co/u/Enxiad)\
**Post date:** [March 27, 2022, 12:44am UTC](https://discuss.elastic.co/t/errors-indexing-binary-attachment-using-ingest-attachment-pipeline/300766/5 "2022-03-27T00:44:49Z")

</div>

I apologize.

I figured this was a different question from the other thread I created, since that was a different error message and I am also trying something else in here now. I also didn't know pinging people was considered rude, I am new here.

This attempt is for a proof of concept which unfortunately I can't get any approval for a subscription since I haven't been able to get Elasticsearch to work.

Thank you for your response, and once again I am sorry to have bothered you.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 27, 2022, 6:37am UTC](https://discuss.elastic.co/t/errors-indexing-binary-attachment-using-ingest-attachment-pipeline/300766/6 "2022-03-27T06:37:30Z")

</div>

What is the goal of your POC?

BTW have a look at FSCrawler project. It could help also to build easily your POC.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 24, 2022, 6:38am UTC](https://discuss.elastic.co/t/errors-indexing-binary-attachment-using-ingest-attachment-pipeline/300766/7 "2022-04-24T06:38:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.