# Synchronize whole files (mostly PDFs) to ES from directory and e-mails
**URL:** https://discuss.elastic.co/t/synchronize-whole-files-mostly-pdfs-to-es-from-directory-and-e-mails/279962
**Category:** Logstash
**Created:** [July 29, 2021, 10:08am UTC](https://discuss.elastic.co/t/synchronize-whole-files-mostly-pdfs-to-es-from-directory-and-e-mails/279962 "2021-07-29T10:08:30Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@jporzelt](https://discuss.elastic.co/u/jporzelt)
#### Post date: [July 29, 2021, 10:08am UTC](https://discuss.elastic.co/t/synchronize-whole-files-mostly-pdfs-to-es-from-directory-and-e-mails/279962/1 "2021-07-29T10:08:30Z")
Hi all,
we would like to setup a document synchronization from a file directory and from a e-mail imap folder to elasticsearch.
For the e-mails we are interested in the attachments. Here the [imap input plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-imap.html) looks promising.
For the files (e.g. PDFs) we would like to setup a directory that gets scanned and whenever a new file is inserted the whole file content (e.g. as a byte array) gets transferred.
Here the [file input plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html) looks like a candidate but maybe not. From the documentation it is not clear to me if we could configure it for the purpose I described.
Any opinions/suggestions? Maybe there are other plugins or totally different solutions out there, we are not aware of.
Thanks in advance,
Johannes
---
### Author:  [@jporzelt](https://discuss.elastic.co/u/jporzelt)
#### Post date: [August 2, 2021, 1:03pm UTC](https://discuss.elastic.co/t/synchronize-whole-files-mostly-pdfs-to-es-from-directory-and-e-mails/279962/2 "2021-08-02T13:03:43Z")
Nobody any experience in this topic or at least any suggestions/opinions?
---
### Author:  [@jporzelt](https://discuss.elastic.co/u/jporzelt)
#### Post date: [August 5, 2021, 3:24pm UTC](https://discuss.elastic.co/t/synchronize-whole-files-mostly-pdfs-to-es-from-directory-and-e-mails/279962/3 "2021-08-05T15:24:38Z")
Actually the IMAP input worked quite well.
```auto
input {
imap {
...
}
}
filter {
#duplicate event for each attachment
split {
field => "attachments"
}
if ".pdf" in [attachments][filename] {
...
} else {
#drop non-pdf attachments
drop { }
}
}
output {
...
}
```
---
### Author:  [@cknz](https://discuss.elastic.co/u/cknz)
#### Post date: [August 9, 2021, 8:08am UTC](https://discuss.elastic.co/t/synchronize-whole-files-mostly-pdfs-to-es-from-directory-and-e-mails/279962/4 "2021-08-09T08:08:54Z")
The file input plugin will most emphatically not be your friend I think. Logstash is really mostly about... well Logs, and is not really kitted out for use as an generalised Elasticsearch ETL tool.
I would suggest something a bit more custom perhaps. Considering you're doing things with PDFs, you might want to check out more about Ingest Pipelines and what those might offer (sorry, I'm not familiar with non-Logs use-cases with Ingest Pipelines).
No experience with the imap plugin, but its interesting to see that it can do that.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 6, 2021, 8:09am UTC](https://discuss.elastic.co/t/synchronize-whole-files-mostly-pdfs-to-es-from-directory-and-e-mails/279962/5 "2021-09-06T08:09:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.