# How to specify file to Ingest Attachment

**URL:** https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843
**Category:** Elasticsearch
**Created:** [February 21, 2017, 9:34am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843 "2017-02-21T09:34:39Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![fjosef](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fjosef/32/15728_2.png) [@fjosef](https://discuss.elastic.co/u/fjosef)
#### Post date: [February 21, 2017, 9:34am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/1 "2017-02-21T09:34:39Z")

</div>

Hi everyone!

I have a Wordpress website and replaced native search with ElasticSearch using ElasticPress plugin.

Every thing is working perfect, but now we want to index binary file contents (especially pdf). For testing, I'm using Kibana and all explained in [documentation](https://www.elastic.co/guide/en/elasticsearch/plugins/5.2/using-ingest-attachment.html) are working good.

Literally I read all the documentation and discussions about Ingest Attachment and was not able to find how I must pass pdf file itself.

All examples I found, using "data" field and passing base64 encoded text:

```
PUT my_index/my_type/my_id?pipeline=attachment
{
  "data": "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0="
}

```

I checked Ingest Attachment plugin itself, it comes with Tika preinstalled and suppose to extract file content.

Also I read [this one](https://discuss.elastic.co/t/mapper-attachment-plugin-vs-pre-parsing-and-extracting-content-from-binary-files/73764) from Taylor Lovett, creator of ElasticPress. It is interesting topic.

Please someone give me more clear example, also do I really need to use Ingest or just pre-parse file contents, then index them.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [February 21, 2017, 9:36am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/2 "2017-02-21T09:36:59Z")

</div>

You need to base64 encode the entire PDF before you put if in that json format, then it is inserted as the `data` value.

---

<div class="post-metadata">

### Author: ![fjosef](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fjosef/32/15728_2.png) [@fjosef](https://discuss.elastic.co/u/fjosef)
#### Post date: [February 21, 2017, 9:40am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/3 "2017-02-21T09:40:48Z")

</div>

Thanks for your fast response!  
But as you mentioned, still we need some heavy lifting. Then what is really benefits of Ingest?

Also can give me some example codes to do this?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [February 21, 2017, 9:44am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/4 "2017-02-21T09:44:22Z")

</div>

I have used the `base64` command on the shell previously, I haven't really worked with large amounts of binary docs to automate it further sorry.

> [@fjosef](#):
>
> Then what is really benefits of Ingest?

It can be used for more than PDFs 🙂

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 21, 2017, 9:54am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/5 "2017-02-21T09:54:22Z")

</div>

Have a look at FSCrawler project. It exposes a REST endpoint where you can simply upload your binary file.

---

<div class="post-metadata">

### Author: ![fjosef](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fjosef/32/15728_2.png) [@fjosef](https://discuss.elastic.co/u/fjosef)
#### Post date: [February 21, 2017, 9:58am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/6 "2017-02-21T09:58:44Z")

</div>

I don't have upload problem, they are already uploaded.

Also if other plugins needed, then what is the benefits of Ingest?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 21, 2017, 10:10am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/7 "2017-02-21T10:10:01Z")

</div>

They are uploaded where?

---

<div class="post-metadata">

### Author: ![fjosef](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fjosef/32/15728_2.png) [@fjosef](https://discuss.elastic.co/u/fjosef)
#### Post date: [February 21, 2017, 10:14am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/8 "2017-02-21T10:14:43Z")

</div>

As I said, it is a Wordpress site and pdf files are attached to the post using custom fields.  
So they are on the server and have a known path.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 21, 2017, 10:43am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/9 "2017-02-21T10:43:53Z")

</div>

If you want to search for them, you need to index them.  
One way or another you need to send their content to elasticsearch.

- You can extract yourself the content and just send what you want to index to elasticsearch.
- You can send the binary BASE64 to elasticsearch ingest which will do the extraction
- You can send the binary to FSCrawler which will do the extraction before sending to elasticsearch

---

<div class="post-metadata">

### Author: ![fjosef](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fjosef/32/15728_2.png) [@fjosef](https://discuss.elastic.co/u/fjosef)
#### Post date: [February 21, 2017, 10:56am UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/10 "2017-02-21T10:56:29Z")

</div>

Yes I know I must extract content and index binaries. Then from what I'v got, it isn't simply give file path to Ingest (of course after creating pipeline and mapping) and Ingest do the extraction?

Also binary must be BASE64 formatted?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 21, 2017, 2:34pm UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/11 "2017-02-21T14:34:29Z")

</div>

No. Elasticsearch never fetches data from a source. You have to push it.  
Note that you can write your own plugin which fetches it if you wish.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 21, 2017, 2:35pm UTC](https://discuss.elastic.co/t/how-to-specify-file-to-ingest-attachment/75843/12 "2017-03-21T14:35:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
