# FScrawler: Production Support Available? **URL:** \ **Category:** Elasticsearch\ **Created:** [April 22, 2020, 1:58pm UTC](https://discuss.elastic.co/t/fscrawler-production-support-available/229277 "2020-04-22T13:58:16Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Yamini\_Shashank](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yamini_shashank/32/66024_2.png) [@Yamini\_Shashank](https://discuss.elastic.co/u/Yamini_Shashank)\ **Post date:** [April 22, 2020, 1:58pm UTC](https://discuss.elastic.co/t/fscrawler-production-support-available/229277/1 "2020-04-22T13:58:16Z")
Does elasticsearch provide production support for fscrawler? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 22, 2020, 3:44pm UTC](https://discuss.elastic.co/t/fscrawler-production-support-available/229277/2 "2020-04-22T15:44:30Z")
Welcome! FSCrawler is a community project. Not supported by Elastic. But I do support it as best as I can here 😉 ---
**Author:** ![Yamini\_Shashank](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yamini_shashank/32/66024_2.png) [@Yamini\_Shashank](https://discuss.elastic.co/u/Yamini_Shashank)\ **Post date:** [April 23, 2020, 6:11am UTC](https://discuss.elastic.co/t/fscrawler-production-support-available/229277/3 "2020-04-23T06:11:41Z")
Thank you @dadoonet for your response. Do you know if someone can help me get some feedback on past production usage please? Also, do you know if there is a similar feature/plugin supported by Elasticsearch? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 23, 2020, 6:36am UTC](https://discuss.elastic.co/t/fscrawler-production-support-available/229277/4 "2020-04-23T06:36:16Z")
I can. Ask your questions here. > [@Yamini\_Shashank](#): > > Also, do you know if there is a similar feature/plugin supported by Elasticsearch? The ingest attachment plugin is officially supported. Also have a look at Workplace search. ---
**Author:** ![Yamini\_Shashank](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yamini_shashank/32/66024_2.png) [@Yamini\_Shashank](https://discuss.elastic.co/u/Yamini_Shashank)\ **Post date:** [April 23, 2020, 11:23am UTC](https://discuss.elastic.co/t/fscrawler-production-support-available/229277/5 "2020-04-23T11:23:27Z")
I am using FScrawler to index a bulk set of documents maintained in a fixed folder structure. Let's say 1 TB. While my POC (100 files) has been successful to index this directory content (stored on azure) into ES in short time, i'm looking at a continuous change/creation/deletion of files (say 100 files per day) using FScrawler. Maintenance of the crawler in terms of documents processed, error monitoring, failure recovery etc are still not clear. While crawler enables content search which is my primary requirement, any solution within fscrawler or alternatives will be appreciated. Excellent creation btw! Thank you in advance! PS: I will look up the suggestions you already posted last. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 23, 2020, 1:30pm UTC](https://discuss.elastic.co/t/fscrawler-production-support-available/229277/6 "2020-04-23T13:30:30Z")
> Maintenance of the crawler in terms of documents processed, error monitoring, failure recovery etc are still not clear. I'd say that the crawler is not designed (yet) to be resilient. It means that if it crashes, some of the documents might have been indexed while some others won't. When you restart the job, it will most likely index again documents that have been already indexed during the run when it crashed. The other documents will be indexed. There are a lot of things to make it work better and sadly I don't have enough free time to spend on this yet. My primary goals are for the future: - Use a WatchService instead of going recursively in each folder - Connect to WorkPlace search to offer a nice UI experience - Also storing the jobs inside elasticsearch instead of the local disk would be much better 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 21, 2020, 1:30pm UTC](https://discuss.elastic.co/t/fscrawler-production-support-available/229277/7 "2020-05-21T13:30:32Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.