# Ingesting documents (pdf, word, .txt) to elasticsearch **URL:** \ **Category:** Elasticsearch\ **Created:** [February 15, 2017, 3:17am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128 "2017-02-15T03:17:12Z")\ **Posts on this page:** 12\ **Page:** 2
**Author:** ![Mark\_Dendrix\_Garcia](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Mark\_Dendrix\_Garcia](https://discuss.elastic.co/u/Mark_Dendrix_Garcia)\ **Post date:** [February 17, 2017, 9:18am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/22 "2017-02-17T09:18:32Z")
can I do that automatically ? without typing the whole ./fscrawler jobs --restart command ? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 17, 2017, 9:47am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/23 "2017-02-17T09:47:40Z")
You mean? Reindex everything at every run? Not really with one single command. But, you can start fscrawler from a crontab and run something like: ```auto bin/fscrawler job_name --restart --loop 1 ``` It will run FSCrawler only once then exit. And anytime you restart this command it will restart from scratch for only one run. ---
**Author:** ![Mark\_Dendrix\_Garcia](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Mark\_Dendrix\_Garcia](https://discuss.elastic.co/u/Mark_Dendrix_Garcia)\ **Post date:** [February 17, 2017, 10:15am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/24 "2017-02-17T10:15:07Z")
Yes but I mean is zero administration on re-indexing. The project involves on running the fscrawler and just leave it there then employees start to dump every files every records on every folders. ---
**Author:** ![Mark\_Dendrix\_Garcia](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Mark\_Dendrix\_Garcia](https://discuss.elastic.co/u/Mark_Dendrix_Garcia)\ **Post date:** [February 17, 2017, 10:16am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/25 "2017-02-17T10:16:31Z")
its like every 15minutes I run the whole thing ./fscrawler jobs --restart command like a task scheduler bat file in windows? ---
**Author:** ![Mark\_Dendrix\_Garcia](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Mark\_Dendrix\_Garcia](https://discuss.elastic.co/u/Mark_Dendrix_Garcia)\ **Post date:** [February 17, 2017, 10:16am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/26 "2017-02-17T10:16:41Z")
BTW im on a CentOS machine ---
**Author:** ![Mark\_Dendrix\_Garcia](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Mark\_Dendrix\_Garcia](https://discuss.elastic.co/u/Mark_Dendrix_Garcia)\ **Post date:** [February 17, 2017, 10:31am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/27 "2017-02-17T10:31:03Z")
@dadoonet Also so this FsCrawler is set to run once? because of the timestamp identifications ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 17, 2017, 11:01am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/28 "2017-02-17T11:01:49Z")
Exactly. But with `--loop 1`. With this option, FSCrawler will run only once then exit. ---
**Author:** ![Mark\_Dendrix\_Garcia](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Mark\_Dendrix\_Garcia](https://discuss.elastic.co/u/Mark_Dendrix_Garcia)\ **Post date:** [February 17, 2017, 11:04am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/29 "2017-02-17T11:04:23Z")
I think i managed to build a solution by adding it on the crontab itself "\* \* \* \* \* bash /fscrawler.sh jobs --restart" in this command every one minute crontab runs the script itself making the automation without needing the timestamp to be adjust. ---
**Author:** ![Mark\_Dendrix\_Garcia](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Mark\_Dendrix\_Garcia](https://discuss.elastic.co/u/Mark_Dendrix_Garcia)\ **Post date:** [February 17, 2017, 11:06am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/30 "2017-02-17T11:06:09Z")
Anyway what is the behavior of FSCrawler when --loop 1 and someone deleted a file in the directory ? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 17, 2017, 11:21am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/31 "2017-02-17T11:21:56Z")
> [@Mark\_Dendrix\_Garcia](#): > > "\* \* \* \* \* bash /fscrawler.sh jobs --restart" This is wrong. You will end up with a lot of processes running in parallel. You should use: ```auto * * * * * bash /fscrawler.sh jobs --restart --loop 1 ``` > [@Mark\_Dendrix\_Garcia](#): > > Anyway what is the behavior of FSCrawler when --loop 1 and someone deleted a file in the directory ? Well. If you don't use `--restart` option, any time you launch again fscrawler, it should detect files which have been removed in the meantime. If you use `--restart` I _think_ that it will not detect files removal. But that's a guess as I never tested that. `loop` has no effect on detection. It's only there to exit after a given number of runs. ---
**Author:** ![Mark\_Dendrix\_Garcia](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Mark\_Dendrix\_Garcia](https://discuss.elastic.co/u/Mark_Dendrix_Garcia)\ **Post date:** [February 21, 2017, 1:09am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/32 "2017-02-21T01:09:56Z")
@dadoonet This is noted, Thank you so much! will bring updates today. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 21, 2017, 1:10am UTC](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128/33 "2017-03-21T01:10:04Z")
Closed on March 21, 2017, 1:10am UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. [Previous page](https://discuss.elastic.co/t/ingesting-documents-pdf-word-txt-to-elasticsearch/75128.md?page=1)