# FSCrawler: Error while crawling - Invalid UTF-8 start byte 0xb5

**URL:** <https://discuss.elastic.co/t/fscrawler-error-while-crawling-invalid-utf-8-start-byte-0xb5/230096>\
**Category:** Elasticsearch\
**Created:** [April 28, 2020, 8:00am UTC](https://discuss.elastic.co/t/fscrawler-error-while-crawling-invalid-utf-8-start-byte-0xb5/230096 "2020-04-28T08:00:03Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yamini\_Shashank](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yamini_shashank/32/66024_2.png) [@Yamini\_Shashank](https://discuss.elastic.co/u/Yamini_Shashank)\
**Post date:** [April 28, 2020, 8:00am UTC](https://discuss.elastic.co/t/fscrawler-error-while-crawling-invalid-utf-8-start-byte-0xb5/230096/1 "2020-04-28T08:00:03Z")

</div>

Hi @dadoonet,

I'v installed "fscrawler-es7-2.7-SNAPSHOT" .  
I'v created a PDF from MS Word with simple text.  
Here is a snapshot:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/0/d059599654ad9472e6e06120d6bf3d070fa42697.png)

Now i'v placed it in a folder structure such as:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/9/99fcb1ff499687bc8b82194f233029a657117966.png)

My configuration in yaml:

```auto
---
name: "hvr"
fs:
  url: "F:\\root_folder_in_snapshot"
  update_rate: "1m"
  excludes:
  - "*/~*"
  json_support: false
  filename_as_id: false
  add_filesize: true
  remove_deleted: true
  add_as_inner_object: false
  store_source: false
  index_content: true
  attributes_support: true
  raw_metadata: false
  xml_support: true
  index_folders: true
  lang_detect: false
  continue_on_error: false
  ocr:
    language: "eng"
    enabled: true
    pdf_strategy: "ocr_and_text"
    path: "F:\\Tesseract-OCR"
  follow_symlinks: false
elasticsearch:
  nodes:
  - url: "myelasticurl"
  bulk_size: 100
  flush_interval: "5s"
  byte_size: "10mb"

```

But i end up with this error:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/5/35eb3baf5f4edd044c83d37e0d07f2dca2602d0a.png)

Any help will be appreciated.

---

<div class="post-metadata">

**Author:** ![Yamini\_Shashank](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yamini_shashank/32/66024_2.png) [@Yamini\_Shashank](https://discuss.elastic.co/u/Yamini_Shashank)\
**Post date:** [April 28, 2020, 8:11am UTC](https://discuss.elastic.co/t/fscrawler-error-while-crawling-invalid-utf-8-start-byte-0xb5/230096/2 "2020-04-28T08:11:42Z")

</div>

Apologies, the issue was because of the

```auto
"xml_support: true"

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 28, 2020, 10:17am UTC](https://discuss.elastic.co/t/fscrawler-error-while-crawling-invalid-utf-8-start-byte-0xb5/230096/3 "2020-04-28T10:17:39Z")

</div>

Great that you found what the problem was and shared the solution. I meant when we spoke on Github that you open a new issue on Github, not specifically here but that's fine as it's not lost anywhere. 😉

For your next post, please don't post images of text as they are hard to read, may not display correctly for everyone, and are not searchable.

---

<div class="post-metadata">

**Author:** ![Yamini\_Shashank](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yamini_shashank/32/66024_2.png) [@Yamini\_Shashank](https://discuss.elastic.co/u/Yamini_Shashank)\
**Post date:** [April 28, 2020, 10:27am UTC](https://discuss.elastic.co/t/fscrawler-error-while-crawling-invalid-utf-8-start-byte-0xb5/230096/4 "2020-04-28T10:27:28Z")

</div>

Actually i seem to have bumped into another error:

```auto
10:23:48,194 INFO [f.p.e.c.f.c.BootstrapChecks] Memory [Free/Total=Percent]: HEAP [418.8mb/7.1gb=5.75%], RAM [9.9gb/31.9gb=31.16%], Swap [7gb/39.9gb=17.64%].
10:23:48,529 INFO [f.p.e.c.f.c.FsCrawlerCli] attributes_support is set to true but getting group is not available on [windows server 2016].
10:23:48,540 INFO [f.p.e.c.f.FsCrawlerImpl] attributes_support is set to true but getting group is not available on [windows server 2016].
10:23:49,086 INFO [f.p.e.c.f.c.v.ElasticsearchClientV7] Elasticsearch Client for version 7.x connected to a node running version 7.5.0
10:23:49,147 INFO [f.p.e.c.f.FsCrawlerImpl] Starting FS crawler
10:23:49,147 INFO [f.p.e.c.f.FsCrawlerImpl] FS crawler started in watch mode. It will run unless you stop it with CTRL+C.
10:23:49,353 INFO [f.p.e.c.f.FsParserAbstract] FS crawler started for [hvr] for [F:\hvr_copy] every [1m]
10:23:49,681 WARN [o.a.t.p.PDFParser] J2KImageReader not loaded. JPEG2000 files will not be processed.
See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
for optional dependencies.

10:24:50,586 WARN [f.p.e.c.f.FsParserAbstract] Can't find stored field name to check existing filenames in path [F:\hvr_copy\00\2a]. Please set store: true on field [file.filename]
10:24:50,586 WARN [f.p.e.c.f.FsParserAbstract] Error while crawling F:\hvr_copy: Mapping is incorrect: please set stored: true on field [file.filename].

```

Do i need to create a elasticsearch schema mapping prior to crawling data to an index?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 28, 2020, 10:47am UTC](https://discuss.elastic.co/t/fscrawler-error-while-crawling-invalid-utf-8-start-byte-0xb5/230096/5 "2020-04-28T10:47:05Z")

</div>

> [@Yamini\_Shashank](#):
>
> Do i need to create a elasticsearch schema mapping prior to crawling data to an index?

This probably means that the index has been created before FSCrawler started. And created with an incompatible mapping. If you don't want to do anything specific with the mapping and you don't care of the existing data, just delete the index:

```auto
DELETE hvr*

```

And restart FSCrawler:

```auto
bin/fscrawler hvr --restart

```

---

<div class="post-metadata">

**Author:** ![Yamini\_Shashank](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yamini_shashank/32/66024_2.png) [@Yamini\_Shashank](https://discuss.elastic.co/u/Yamini_Shashank)\
**Post date:** [April 28, 2020, 11:56am UTC](https://discuss.elastic.co/t/fscrawler-error-while-crawling-invalid-utf-8-start-byte-0xb5/230096/6 "2020-04-28T11:56:50Z")

</div>

It Worked! Thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 26, 2020, 11:56am UTC](https://discuss.elastic.co/t/fscrawler-error-while-crawling-invalid-utf-8-start-byte-0xb5/230096/7 "2020-05-26T11:56:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.