ElasticSearch Indexing question

[dadoonet (David Pilato)]

@Yorko

Is there any way to not include the \r\n whitespaces in the _source, in my docs and docx there are different parts separated by spaces, i would prefer the new lines to be in but not printed...

No. It's indexed as it is extracted by Tika. But TBH I did not understand what is the problem. May be illustrate with an example what you have now and what you would like to see?

@Yorko

Is there a way to make it recycle memory or will java just keep eating all the memory until there is no more or the scan finishes?

May be some enhancements need to be done in fscrawler project. For sure I should support adding easily memory settings to the fscrawler job. For now, you have to hack the script or set $JAVA_OPTS.
I opened Add FS_JAVA_OPTS JVM option · Issue #134 · dadoonet/fscrawler · GitHub for this. Feel free to contribute!

@Yorko

I've tried hooking the results into Kibana and i didn't get any results in the discover tab, here is the template i used:

I never tested it with Kibana for now. I'd advice that you first test with simple curl commands that everything has been indexed as expected. Is it the case?