# Logstash is not showing base64 encoded data for pdf's extracted from urls

**URL:** https://discuss.elastic.co/t/logstash-is-not-showing-base64-encoded-data-for-pdfs-extracted-from-urls/330386
**Category:** Logstash
**Created:** [April 20, 2023, 10:42am UTC](https://discuss.elastic.co/t/logstash-is-not-showing-base64-encoded-data-for-pdfs-extracted-from-urls/330386 "2023-04-20T10:42:56Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Disha\_Bodade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/disha_bodade/32/84522_2.png) [@Disha\_Bodade](https://discuss.elastic.co/u/Disha_Bodade)
#### Post date: [April 20, 2023, 10:42am UTC](https://discuss.elastic.co/t/logstash-is-not-showing-base64-encoded-data-for-pdfs-extracted-from-urls/330386/1 "2023-04-20T10:42:56Z")

</div>

Hi Team,  
I am using logstash http filter to get pdf from url and extract it.  
http filter has downloaded pdf and extracted its content on target\_field. But the contents are not proper and also its not base64 encoded. How I can handle this encoding at logstash, so that I can use ingest pipeline at elasticsearch to index PDF's

Below is the example filter

```auto
        http {
                url => "%{uri}"
                cacert => "/etc/logstash/conf.d/read-pdfs/ca.pem"
                target_body => "content_body"
       }

```

the content body has data in below format in addition with english text in it.

```auto
��������뿟˃O��t��̦\u0007W��\u0017x������\u001f\u001eFǧ'�\u001f�{I��\u0005�Q\u0012��\u0015<���{��.���\u001d_��\u001d��\"�q}��ǀ(�XT�c\\F��q.��/@��U\u0016=�@��\u0003�ʫ_?���:���G�?��As���[Ϸ\"�4Y�\u0012�\u001cTLp\u0012Hd1�/�(cEt�Ҝ\u0019?�Et:������I\u0014\u001d\\=�����'�Qrp1�>D�r:��x��>�G\u001fi��$%}�\u001aܽw��v\u0013�gq�:%R\"�DgӻmG�ϊI\u001e���j��`��8�0�FЬ\u0014p�����> �,\u0003M~���C�W��H\f��p�8\f�ʾA�8\f��2�0

```

How can I handle this encoding and decoding either at logstash or elasticsearch.

Thanks,  
Disha

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [April 20, 2023, 11:18am UTC](https://discuss.elastic.co/t/logstash-is-not-showing-base64-encoded-data-for-pdfs-extracted-from-urls/330386/2 "2023-04-20T11:18:03Z")

</div>

Default encoding is UTF-8. You can use the codec with charset:

```auto
        http {
                url => "%{uri}"
                cacert => "/etc/logstash/conf.d/read-pdfs/ca.pem"
                target_body => "content_body"
                 codec => plain { charset=>"ISO8859-1" }
       }

```

The list supported character-set is [here](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-plain.html#plugins-codecs-plain-charset)

---

<div class="post-metadata">

### Author: ![Disha\_Bodade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/disha_bodade/32/84522_2.png) [@Disha\_Bodade](https://discuss.elastic.co/u/Disha_Bodade)
#### Post date: [April 20, 2023, 12:35pm UTC](https://discuss.elastic.co/t/logstash-is-not-showing-base64-encoded-data-for-pdfs-extracted-from-urls/330386/3 "2023-04-20T12:35:35Z")

</div>

@Rios ,  
http filter does not has codec setting, I tried to add it to logstash output plugin but still same result.

```auto
output {
 elasticsearch {
                hosts => ["https://esost1:9200"]
                index => "my_index"
                codec => plain { charset=>"ISO8859-1" }
                ssl=> true
                ssl_certificate_verification => false
  # cacert => "/etc/logstash/certs/ca.pem"
        }
}

```

Thanks,  
Disha

---

<div class="post-metadata">

### Author: ![Disha\_Bodade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/disha_bodade/32/84522_2.png) [@Disha\_Bodade](https://discuss.elastic.co/u/Disha_Bodade)
#### Post date: [April 28, 2023, 4:58am UTC](https://discuss.elastic.co/t/logstash-is-not-showing-base64-encoded-data-for-pdfs-extracted-from-urls/330386/4 "2023-04-28T04:58:43Z")

</div>

Is there any way, I can create base64 formatted string in logstash? That way I can convert extracted contents from UTF-8 to base64 and then used attachment preprocessor.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 28, 2023, 5:05am UTC](https://discuss.elastic.co/t/logstash-is-not-showing-base64-encoded-data-for-pdfs-extracted-from-urls/330386/5 "2023-04-28T05:05:27Z")

</div>

> [@Disha\_Bodade](#):
>
> Is there any way, I can create base64 formatted string in logstash?

I would lean towards a ruby filter. [This](https://discuss.elastic.co/t/base64-decode-issue/319874/2) is about decoding base64, but should give some clues about encoding.

---

<div class="post-metadata">

### Author: ![Disha\_Bodade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/disha_bodade/32/84522_2.png) [@Disha\_Bodade](https://discuss.elastic.co/u/Disha_Bodade)
#### Post date: [April 28, 2023, 6:58am UTC](https://discuss.elastic.co/t/logstash-is-not-showing-base64-encoded-data-for-pdfs-extracted-from-urls/330386/6 "2023-04-28T06:58:36Z")

</div>

@Badger,  
The input source is database, has one field as url, I am using logstash http filter to get data from those urls, urls has pdf, word, text files.

files size is big as upto 6MB.  
Is it possible to encode whole content extracted by http filter into base64 format and then use attachment processor on that encoded data?

Or let me know any other available option.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 26, 2023, 6:58am UTC](https://discuss.elastic.co/t/logstash-is-not-showing-base64-encoded-data-for-pdfs-extracted-from-urls/330386/7 "2023-05-26T06:58:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
