# Fscrawler index large file

**URL:** <https://discuss.elastic.co/t/fscrawler-index-large-file/128823>\
**Category:** Elasticsearch\
**Created:** [April 20, 2018, 8:07am UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823 "2018-04-20T08:07:07Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![asmael](https://avatars.discourse-cdn.com/v4/letter/a/87869e/32.png) [@asmael](https://discuss.elastic.co/u/asmael)\
**Post date:** [April 20, 2018, 8:07am UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/1 "2018-04-20T08:07:07Z")

</div>

hello ,  
I want to know how can I index a large file I want the whole content to be added , I already tried this : "\_indexed\_chars": "100%" but it dosen't work  
thank you

---

<div class="post-metadata">

**Author:** ![JKhondhu](https://avatars.discourse-cdn.com/v4/letter/j/ed655f/32.png) [@JKhondhu](https://discuss.elastic.co/u/JKhondhu)\
**Post date:** [April 20, 2018, 8:13am UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/2 "2018-04-20T08:13:22Z")

</div>

What format is the large file?  
How would you like to make use of it once it is indexed in Elasticsearch?

---

<div class="post-metadata">

**Author:** ![asmael](https://avatars.discourse-cdn.com/v4/letter/a/87869e/32.png) [@asmael](https://discuss.elastic.co/u/asmael)\
**Post date:** [April 20, 2018, 8:30am UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/3 "2018-04-20T08:30:08Z")

</div>

the format is : 3 287 ko I want to index it to elasticsearch with all of its content

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 20, 2018, 12:38pm UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/4 "2018-04-20T12:38:21Z")

</div>

What are the logs of FSCrawler please?

---

<div class="post-metadata">

**Author:** ![asmael](https://avatars.discourse-cdn.com/v4/letter/a/87869e/32.png) [@asmael](https://discuss.elastic.co/u/asmael)\
**Post date:** [April 20, 2018, 1:37pm UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/5 "2018-04-20T13:37:49Z")

</div>

th file is bieng indexed but not Not all of the content makes it into elasticsearch .  
what do you mean by logs where I can find them if you are talking about my setting there is :  
{  
"name" : "test4",  
"fs": {

"url": "C:\Users\aelkhattabi\Desktop\elastic\testindex",  
"update\_rate": "15m",  
"excludes": ["~\*"],  
"json\_support": false,  
"filename\_as\_id": false,  
"add\_filesize": true,  
"remove\_deleted": true,  
"add\_as\_inner\_object": false,  
"store\_source": false,  
"\_indexed\_chars": "100%",  
"index\_content": true,  
"attributes\_support": false,  
"raw\_metadata": true,  
"xml\_support": false,  
"index\_folders": true,  
"lang\_detect": false,  
"continue\_on\_error": false,  
"pdf\_ocr": true,  
"ocr": {  
"language": "eng"  
}  
},  
"elasticsearch" : {  
"nodes" : [ {  
"host" : "127.0.0.1",  
"port" : 9200,  
"scheme" : "HTTP"  
} ],  
"bulk\_size" : 100,  
"flush\_interval" : "5s"  
},  
"rest" : {  
"scheme" : "HTTP",  
"host" : "127.0.0.1",  
"port" : 8080,  
"endpoint" : "fscrawler"  
}  
}

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 20, 2018, 2:29pm UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/6 "2018-04-20T14:29:26Z")

</div>

> th file is bieng indexed but not Not all of the content makes it into elasticsearch.

Can you share what the indexed document looks like please?

> what do you mean by logs where I can find them

This is the FSCrawler output.

Can you share the document you are trying to index?

---

<div class="post-metadata">

**Author:** ![asmael](https://avatars.discourse-cdn.com/v4/letter/a/87869e/32.png) [@asmael](https://discuss.elastic.co/u/asmael)\
**Post date:** [April 20, 2018, 2:45pm UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/7 "2018-04-20T14:45:34Z")

</div>

![Capture](https://us1.discourse-cdn.com/elastic/original/3X/c/a/ca5692038e54af3fcb4e2b0673f2fbd35aade702.PNG)

thats the content indexed and this is my files

 ![files](https://us1.discourse-cdn.com/elastic/original/3X/1/d/1db5c7a4bf6edf2476f85a07a29b94a4eedefcba.PNG)

I have no errors  
 ![cmd](https://us1.discourse-cdn.com/elastic/original/3X/a/5/a56570a1b03dc18ba12c9e57bb44196a39574c8e.PNG)

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 20, 2018, 3:28pm UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/8 "2018-04-20T15:28:46Z")

</div>

So your document has been indexed. I can see document content extracted in the `content` field. So what is the problem?

---

<div class="post-metadata">

**Author:** ![asmael](https://avatars.discourse-cdn.com/v4/letter/a/87869e/32.png) [@asmael](https://discuss.elastic.co/u/asmael)\
**Post date:** [April 20, 2018, 3:33pm UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/9 "2018-04-20T15:33:31Z")

</div>

the problem is it doesn't index the whole content , maybe because its a large file

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 20, 2018, 5:02pm UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/10 "2018-04-20T17:02:13Z")

</div>

I see.

Change

`"_indexed_chars": "100%",`

By:

`"indexed_chars": "100%",`

---

<div class="post-metadata">

**Author:** ![asmael](https://avatars.discourse-cdn.com/v4/letter/a/87869e/32.png) [@asmael](https://discuss.elastic.co/u/asmael)\
**Post date:** [April 20, 2018, 5:15pm UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/11 "2018-04-20T17:15:08Z")

</div>

It worked for me with "indexed\_chars": "100%"  
thank you a lot

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 18, 2018, 5:15pm UTC](https://discuss.elastic.co/t/fscrawler-index-large-file/128823/12 "2018-05-18T17:15:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
