FSCrawler won't crawl new files

Elastic Stack Elasticsearch
1

The crawler seem not to add any new files. So I investigated and found below warning message on all profiles:

Can't find stored field name to check existing filenames in path
[/some/path]. Please set store: true on field [file.filename]

I currently run updated master branch. I have tried to look for it generically as it is ES-specific and came up with this:

PUT /myindex/bla/_mapping
{
    "properties": {
      "file.filename": {
        "type": "keyword", 
        "store" : true
      }
    }
}

But it gives back:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Mapper for [file.filename] conflicts with existing mapping in other types:\n[mapper [file.filename] has different [store] values]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Mapper for [file.filename] conflicts with existing mapping in other types:\n[mapper [file.filename] has different [store] values]"
  },
  "status": 400
}

What can I do to fix this?

2

Did you upgrade FSCrawler or started from scratch (no index)?

3

Bonsoir.

I upgraded and used --upgrade before starting jobs.

4

Yeah.

You need to read Upgrade to 2.5 Section.

I did not implement a upgrade mechanism for this as I don’t believe there are many users in production with this project but I might be wrong.

Do you think we should support an upgrade option for 2.5? Could you open an issue? At the very least I should warn users running the upgrade command that no upgrade is made for 2.5 yet.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.