# Automate ingest through a pipeline (ingest attachment processor)

**URL:** <https://discuss.elastic.co/t/automate-ingest-through-a-pipeline-ingest-attachment-processor/255054>\
**Category:** Elastic Community and Ecosystem\
**Created:** [November 11, 2020, 12:22pm UTC](https://discuss.elastic.co/t/automate-ingest-through-a-pipeline-ingest-attachment-processor/255054 "2020-11-11T12:22:56Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Fidel\_Perez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fidel_perez/32/78585_2.png) [@Fidel\_Perez](https://discuss.elastic.co/u/Fidel_Perez)\
**Post date:** [November 11, 2020, 12:22pm UTC](https://discuss.elastic.co/t/automate-ingest-through-a-pipeline-ingest-attachment-processor/255054/1 "2020-11-11T12:22:56Z")

</div>

hello there.  
Does anyone know if there a tool something like beats to ingest human created files such a pdf or word files (dockx) using my attachment pipeline that processes files with ingest attachment pluing

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 11, 2020, 1:32pm UTC](https://discuss.elastic.co/t/automate-ingest-through-a-pipeline-ingest-attachment-processor/255054/2 "2020-11-11T13:32:43Z")

</div>

You can use the [ingest attachment plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/ingest-attachment.html).

There an example here: [https://www.elastic.co/guide/en/elasticsearch/plugins/current/using-ingest-attachment.html](https://www.elastic.co/guide/en/elasticsearch/plugins/current/using-ingest-attachment.html)

```auto
PUT _ingest/pipeline/attachment
{
  "description" : "Extract attachment information",
  "processors" : [
    {
      "attachment" : {
        "field" : "data"
      }
    }
  ]
}
PUT my_index/_doc/my_id?pipeline=attachment
{
  "data": "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0="
}
GET my_index/_doc/my_id

```

The `data` field is basically the BASE64 representation of your binary file.

But you need to crawl yourself the filesystem.

You can use [FSCrawler](https://fscrawler.readthedocs.io). There's [a tutorial](https://fscrawler.readthedocs.io/en/latest/user/tutorial.html) to help you getting started.

---

<div class="post-metadata">

**Author:** ![Fidel\_Perez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fidel_perez/32/78585_2.png) [@Fidel\_Perez](https://discuss.elastic.co/u/Fidel_Perez)\
**Post date:** [November 11, 2020, 5:00pm UTC](https://discuss.elastic.co/t/automate-ingest-through-a-pipeline-ingest-attachment-processor/255054/3 "2020-11-11T17:00:39Z")

</div>

hi @dadoonet thanks for your response. i know about the plugin and how to used,but so far have only seen this the use of ingest-attachment through a pipeline using API calls. what i want is a tool that do the ingest for me. i tried filebeats and it reads my system logs on its own. that is what im looking for, something to read the my files stored in my device or in a server or even a cloud storage without the need to write code to do the readings of files turned in base64 encoded and then send the files to be ingested.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 11, 2020, 6:38pm UTC](https://discuss.elastic.co/t/automate-ingest-through-a-pipeline-ingest-attachment-processor/255054/4 "2020-11-11T18:38:32Z")

</div>

Did you read the last part of my answer?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 9, 2020, 6:38pm UTC](https://discuss.elastic.co/t/automate-ingest-through-a-pipeline-ingest-attachment-processor/255054/5 "2020-12-09T18:38:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.