Failed to create elasticsearch client, connection refused while running fscrawler pod

Elastic Stack Elasticsearch
1

Hello,
our fscrawler is unable to make a connection with Elasticsearch in the linux server.
Previously with the same configuration, it was working fine but don't know why suddenly we starts to face this issue

-- we are using fscrawler 2.4 and after facing issue upgraded to 2.7 also. and our elk stack is 6.8.14

sharing logs for the same:

kubectl logs fscrawler-cognitiveclaims-batch0-1641014100-9zqvg -n app-claimselk cronjob-claimselk-fscrawler-batch0
05:15:05,974 DEBUG [f.p.e.c.f.u.FsCrawlerUtil] Mapping [2/_settings.json] already exists
05:15:05,976 DEBUG [f.p.e.c.f.u.FsCrawlerUtil] Mapping [2/_settings_folder.json] already exists
05:15:05,982 DEBUG [f.p.e.c.f.u.FsCrawlerUtil] Mapping [5/_settings.json] already exists
05:15:05,983 DEBUG [f.p.e.c.f.u.FsCrawlerUtil] Mapping [5/_settings_folder.json] already exists
05:15:06,000 DEBUG [f.p.e.c.f.u.FsCrawlerUtil] Mapping [6/_settings.json] already exists
05:15:06,000 DEBUG [f.p.e.c.f.u.FsCrawlerUtil] Mapping [6/_settings_folder.json] already exists
05:15:06,004 DEBUG [f.p.e.c.f.FsCrawler] Starting job [processformstemplates]...
05:15:06,142 TRACE [f.p.e.c.f.FsCrawler] settings used for this crawler: [{
  "name" : "processformstemplates0",
  "fs" : {
    "url" : "/usr/share/fscrawler/data",
    "update_rate" : "15m",
    "excludes" : [ "*/~*" ],
    "json_support" : false,
    "filename_as_id" : true,
    "add_filesize" : true,
    "remove_deleted" : false,
    "add_as_inner_object" : false,
    "store_source" : false,
    "index_content" : true,
    "attributes_support" : false,
    "raw_metadata" : false,
    "xml_support" : false,
    "index_folders" : true,
    "lang_detect" : false,
    "continue_on_error" : false,
    "pdf_ocr" : true,
    "ocr" : {
      "language" : "eng"
    }
  },
  "elasticsearch" : {
    "nodes" : [ {
      "host" : "elasticsearch-claimselk-http.app-claimselk.svc.cluster.local",
      "port" : 9200,
      "scheme" : "HTTP"
    } ],
    "index" : "formslettersandtemplates",
    "bulk_size" : 100,
    "flush_interval" : "5s",
    "username" : "elastic"
  },
  "rest" : {
    "port" : 0
  }
}]
**05:15:06,819 WARN  [f.p.e.c.f.c.ElasticsearchClientManager] failed to create elasticsearch client, disabling crawler...**
**05:15:06,819 FATAL [f.p.e.c.f.FsCrawler] Fatal error received while running the crawler: [Connection refused]**
**05:15:06,819 DEBUG [f.p.e.c.f.FsCrawler] error caught**
**java.net.ConnectException: Connection refused**
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_121]
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_121]
        at org.elasticsearch.client.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171) ~[elasticsearch-rest-client-6.0.0-beta1.jar:6.0.0-beta1]
        at org.elasticsearch.client.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145) ~[elasticsearch-rest-client-6.0.0-beta1.jar:6.0.0-beta1]
        at org.elasticsearch.client.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348) ~[elasticsearch-rest-client-6.0.0-beta1.jar:6.0.0-beta1]
        at org.elasticsearch.client.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192) ~[elasticsearch-rest-client-6.0.0-beta1.jar:6.0.0-beta1]
        at org.elasticsearch.client.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[elasticsearch-rest-client-6.0.0-beta1.jar:6.0.0-beta1]
        at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_121]
05:15:06,824 DEBUG [f.p.e.c.f.FsCrawlerImpl] Closing FS crawler [processformstemplates0]
05:15:06,824 DEBUG [f.p.e.c.f.FsCrawlerImpl] FS crawler Rest service stopped
05:15:06,824 DEBUG [f.p.e.c.f.c.ElasticsearchClientManager] Closing Elasticsearch client manager
05:15:06,824 DEBUG [f.p.e.c.f.c.ElasticsearchClient] Closing REST client
05:15:06,825 DEBUG [f.p.e.c.f.c.ElasticsearchClient] REST client closed
05:15:06,825 DEBUG [f.p.e.c.f.FsCrawlerImpl] ES Client Manager stopped
05:15:06,826 INFO  [f.p.e.c.f.FsCrawlerImpl] FS crawler [processformstemplates0] stopped
05:15:06,827 DEBUG [f.p.e.c.f.FsCrawlerImpl] Closing FS crawler [processformstemplates0]
05:15:06,828 DEBUG [f.p.e.c.f.FsCrawlerImpl] FS crawler Rest service stopped
05:15:06,828 DEBUG [f.p.e.c.f.c.ElasticsearchClientManager] Closing Elasticsearch client manager
05:15:06,828 DEBUG [f.p.e.c.f.c.ElasticsearchClient] Closing REST client
05:15:06,828 DEBUG [f.p.e.c.f.c.ElasticsearchClient] REST client closed
05:15:06,828 DEBUG [f.p.e.c.f.FsCrawlerImpl] ES Client Manager stopped
05:15:06,828 INFO  [f.p.e.c.f.FsCrawlerImpl] FS crawler [processformstemplates0] stopped
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.