FScrawler issue while crawling through a remote host

Elastic Stack Elasticsearch
1

Hi,

There is an issue while crawling through files from remote host. Please see the code snippet below

---
name: "new_attachment"
server:
  hostname: "**************"
  port: 22
  username: "**********"
  password: "**************"
  protocol: "ssh"
fs:
  url: "/E:/foldertoindex"
  update_rate: "15m"
  excludes:
  - "*/~*"
  json_support: false
  filename_as_id: false
  add_filesize: true
  remove_deleted: true
  add_as_inner_object: false
  store_source: false
  index_content: true
  attributes_support: false
  raw_metadata: false
  xml_support: false
  index_folders: true
  lang_detect: false
  continue_on_error: false
  ocr:
    language: "eng"
    enabled: true
    pdf_strategy: "ocr_and_text"
  follow_symlinks: false
elasticsearch:
  nodes:
  - url: "http://*************:9200"
  bulk_size: 100
  flush_interval: "5s"
  byte_size: "10mb"

Error is as follows:

04:32:42,146 DEBUG [f.p.e.c.f.FsParserAbstract] Fs crawler thread [new_attachmen
t] is now running. Run #46...
04:32:42,146 DEBUG [f.p.e.c.f.c.s.FileAbstractorSSH] Opening SSH connection to *username@hostname*
04:32:42,783 DEBUG [f.p.e.c.f.c.s.FileAbstractorSSH] SSH connection successful
04:32:42,815 WARN  [f.p.e.c.f.FsParserAbstract] Error while crawling /E:/foldert
oindex: begin 0, end -1, length 17
04:32:42,815 WARN  [f.p.e.c.f.FsParserAbstract] Full stacktrace
java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 17
        at java.lang.String.checkBoundsBeginEnd(String.java:3319) ~[?:?]
        at java.lang.String.substring(String.java:1874) ~[?:?]
        at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.indexDirectory(Fs
ParserAbstract.java:541) ~[fscrawler-core-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.run(FsParserAbstr
act.java:142) [fscrawler-core-2.7-SNAPSHOT.jar:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]
04:32:42,819 DEBUG [f.p.e.c.f.FsParserAbstract] Fs crawler is going to sleep for
 15m
04:47:42,905 DEBUG [f.p.e.c.f.FsParserAbstract] Fs crawler is now waking up agai
n...
04:47:42,905 DEBUG [f.p.e.c.f.FsParserAbstract] Fs crawler thread [new_attachmen
t] is now running. Run #47...
04:47:42,905 DEBUG [f.p.e.c.f.c.s.FileAbstractorSSH] Opening SSH connection to D
IR\RGadge@NL0123EPC0921.dir.slb.com
04:47:43,589 DEBUG [f.p.e.c.f.c.s.FileAbstractorSSH] SSH connection successful
04:47:43,629 WARN  [f.p.e.c.f.FsParserAbstract] Error while crawling /E:/foldert
oindex: begin 0, end -1, length 17
04:47:43,629 WARN  [f.p.e.c.f.FsParserAbstract] Full stacktrace
java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 17
        at java.lang.String.checkBoundsBeginEnd(String.java:3319) ~[?:?]
        at java.lang.String.substring(String.java:1874) ~[?:?]
        at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.indexDirectory(Fs
ParserAbstract.java:541) ~[fscrawler-core-2.7-SNAPSHOT.jar:?]
        at fr.pilato.elasticsearch.crawler.fs.FsParserAbstract.run(FsParserAbstr
act.java:142) [fscrawler-core-2.7-SNAPSHOT.jar:?]

Regards,
Umesh

2

Could you report this there?

Thanks !

3

Done, I have created a new bug in GitHub

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.