# FSCrawler Ingestion Issue **URL:** \ **Category:** Logstash\ **Created:** [September 7, 2020, 5:56am UTC](https://discuss.elastic.co/t/fscrawler-ingestion-issue/247706 "2020-09-07T05:56:00Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Umesh\_S\_Bhimani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/umesh_s_bhimani/32/69094_2.png) [@Umesh\_S\_Bhimani](https://discuss.elastic.co/u/Umesh_S_Bhimani)\ **Post date:** [September 7, 2020, 5:56am UTC](https://discuss.elastic.co/t/fscrawler-ingestion-issue/247706/1 "2020-09-07T05:56:00Z")
I have installed FSCrawler using a user account on a server and am trying to use a service account later to create a job and ingest the data. Does it require to use the same account used for FSCrawler installation to ingest the data? Also, the data I am trying to ingest is around 8-9TB, the maximum ingestion of 3TB worked for me so far. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [September 23, 2020, 1:17am UTC](https://discuss.elastic.co/t/fscrawler-ingestion-issue/247706/2 "2020-09-23T01:17:20Z")
The account which needs to run FSCrawler needs to have a read access to the files you want to scan and a write access to the FSCrawler configuration dir. Did the job progress? Anything in logs? ---
**Author:** ![Umesh\_S\_Bhimani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/umesh_s_bhimani/32/69094_2.png) [@Umesh\_S\_Bhimani](https://discuss.elastic.co/u/Umesh_S_Bhimani)\ **Post date:** [September 23, 2020, 3:25am UTC](https://discuss.elastic.co/t/fscrawler-ingestion-issue/247706/3 "2020-09-23T03:25:34Z")
Yes, the data ingestion is in progress. But I broke it down in to smaller chunks of 2-3 TB each to ingest into one main index. It took around 3-4 days ingest the 2-3 TB of data. I have created 5 separate fscrawler services & the services are running smoothly ever since no issues identified yet. Does the fscrawler pick the lastmodifieddate from the settings.json as the start point of ingestion? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [September 23, 2020, 8:17am UTC](https://discuss.elastic.co/t/fscrawler-ingestion-issue/247706/4 "2020-09-23T08:17:22Z")
FSCrawler uses a `_status.json` file to store the last timestamp that was used when it started to run. This date is then compared to the files which are crawled again to see if there have been any change. I hope this clarifies. ---
**Author:** ![Umesh\_S\_Bhimani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/umesh_s_bhimani/32/69094_2.png) [@Umesh\_S\_Bhimani](https://discuss.elastic.co/u/Umesh_S_Bhimani)\ **Post date:** [September 23, 2020, 11:17am UTC](https://discuss.elastic.co/t/fscrawler-ingestion-issue/247706/5 "2020-09-23T11:17:43Z")
Yes this helps, I stopped my fscrawler service 2 days back but it still has not created the \_status.json file. Does it take time to create the file? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 21, 2020, 11:17am UTC](https://discuss.elastic.co/t/fscrawler-ingestion-issue/247706/6 "2020-10-21T11:17:52Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.