Hello everyone this is my first post in this forum, well I would like to synchronize a MongoDB database with Elasticsearch thanks to Logstash, I used version 8.3.3 of ELK with a MongoDriver4.4. I have configured the pipeline input
input {
jdbc {
jdbc_driver_class => "com.wisecoders.dbschema.mongodb.JdbcDriver"
jdbc_driver_library => "/usr/share/logstash/logstash-core/lib/jars/mongojdbc4.4.jar"
jdbc_connection_string => "jdbc:mongodb://localhost:27017/testdb"
jdbc_user => ""
schedule => "* * * * *"
statement => "db.address.find({},{'_id': false});"
}
}
The Docker service has been configured as follows
logstash:
build:
context: logstash/
args:
ELK_VERSION: $ELK_VERSION
volumes:
- ./logstash/pipeline/:/usr/share/logstash/pipeline/:ro
- ./logstash/config/:/usr/share/logstash/config/:ro
- ./logstash/empty/:/usr/share/logstash/.DbSchema/logs/:rw
- ./logstash/driver/bson-4.7.1.jar/:/usr/share/logstash/logstash-core/lib/jars/bson-4.7.1.jar/:ro
- ./logstash/driver/bson-record-codec-4.7.1.jar/:/usr/share/logstash/logstash-core/lib/jars/bson-record-codec-4.7.1.jar/:ro
- ./logstash/driver/graal-sdk-22.2.0.jar/:/usr/share/logstash/logstash-core/lib/jars/graal-sdk-22.2.0.jar/:ro
- ./logstash/driver/gson-2.9.0.jar/:/usr/share/logstash/logstash-core/lib/jars/gson-2.9.0.jar/:ro
- ./logstash/driver/icu4j-71.1.jar/:/usr/share/logstash/logstash-core/lib/jars/icu4j-71.1.jar/:ro
- ./logstash/driver/js-22.2.0.jar/:/usr/share/logstash/logstash-core/lib/jars/js-22.2.0.jar/:ro
- ./logstash/driver/js-scriptengine-22.2.0.jar/:/usr/share/logstash/logstash-core/lib/jars/js-scriptengine-22.2.0.jar/:ro
- ./logstash/driver/mongodb-driver-core-4.7.1.jar/:/usr/share/logstash/logstash-core/lib/jars/mongodb-driver-core-4.7.1.jar/:ro
- ./logstash/driver/mongodb-driver-sync-4.7.1.jar/:/usr/share/logstash/logstash-core/lib/jars/mongodb-driver-sync-4.7.1.jar/:ro
- ./logstash/driver/mongojdbc4.4.jar/:/usr/share/logstash/logstash-core/lib/jars/mongojdbc4.4.jar/:ro
- ./logstash/driver/regex-22.2.0.jar/:/usr/share/logstash/logstash-core/lib/jars/regex-22.2.0.jar/:ro
- ./logstash/driver/truffle-api-22.2.0.jar/:/usr/share/logstash/logstash-core/lib/jars/truffle-api-22.2.0.jar/:ro
ports:
- "5000:5000/tcp"
- "5000:5000/udp"
- "9600:9600"
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
depends_on:
- elasticsearch
When I run the docker-compose up command, all services start until I get the following error:
[INFO ] 2022-08-13 11:03:00.965 [cluster-ClusterId{value='62f784e43c2db20a2addfdbe', description='null'}-localhost:27017] cluster - Exception in monitor thread while connecting to server localhost:27017
logstash_1 | com.mongodb.MongoSocketOpenException: Exception opening socket
logstash_1 | at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:70) ~[mongodb-driver-core-4.7.1.jar:?]
logstash_1 | at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:180) ~[mongodb-driver-core-4.7.1.jar:?]
logstash_1 | at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:193) [mongodb-driver-core-4.7.1.jar:?]
logstash_1 | at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:157) [mongodb-driver-core-4.7.1.jar:?]
logstash_1 | at java.lang.Thread.run(Thread.java:834) [?:?]
logstash_1 | Caused by: java.net.ConnectException: Connection refused (Connection refused)
logstash_1 | at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]
logstash_1 | at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[?:?]
logstash_1 | at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[?:?]
logstash_1 | at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[?:?]
logstash_1 | at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) ~[?:?]
logstash_1 | at java.net.Socket.connect(Socket.java:609) ~[?:?]
logstash_1 | at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:107) ~[mongodb-driver-core-4.7.1.jar:?]
logstash_1 | at com.mongodb.internal.connection.SocketStream.initializeSocket(SocketStream.java:79) ~[mongodb-driver-core-4.7.1.jar:?]
logstash_1 | at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:65) ~[mongodb-driver-core-4.7.1.jar:?]
It seems that Logstash can't read the database, I don't know if it's missing, I've tested many configurations and versions but without success. A little help would be nice and thanks in advance.