Installing Mongodb input plugin in Logstash

Elastic Stack Logstash
1

Hello,

I have a MongoDB database and I want to create a pipeline in Logstash to transfer data from MongoDb to Elasticsearch using Logstash.

I have installed Elasticsearch latest version 6.1 and I checked official website and found that there is no input plugin for MongoDB.

But there are two GitHub repo available for MongoDB input plugin for logstash

  1. GitHub - kurkop/logstash-input-mongodb: MongoDB input plugin for Logstash (last commited on Dec 18,2015 and have .gem file)
  2. GitHub - phutchins/logstash-input-mongodb: MongoDB input plugin for Logstash (last commited on May 4,2017 but no .gem file included)

Now If I install 1st one then I am getting error of compatibility. Here is error I am getting

ERROR: An error occured when installing the: file:///E:/Office/mongodb-old/logstash-input-mongodb-master.zip, to have more information about the error add a DEBUG=1 before running the command., message: Bundler could not find compatible versions for gem "logstash-core":
In snapshot (Gemfile.lock):
logstash-core (= 6.1.0)

In Gemfile:
logstash-core-plugin-api (>= 0) java depends on
logstash-core (= 6.1.0) java

logstash-input-syslog (>= 0) java depends on
  logstash-filter-grok (>= 0) java depends on
    logstash-core (>= 5.6.0) java

logstash-input-mongodb (= 0.3.3) java depends on
  logstash-core (< 3.0.0, >= 2.0.0.beta2) java

logstash-core (>= 0) java

Running bundle update will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

So its not compatible with latest version.

  1. If I install 2nd one, I am getting error below

ERROR: Invalid pack for: file:///E:/Office/logstash-input-mongodb-master.zip, reason: The pack must contains at least one plugin, message: The pack must contains at least one plugin

may be because it doesn't contain any .gem file ?

So how I create logstash pipeline to transfer data from MongoDb to Elasticsearch ?

2

no one interested to reply ?

3

You can investigate using JBDC. There are some non-free 3rd party jars from some integrations vendors.

Then you might be able to use the JDBC input.

Other alternatives:

  • Create a MongoDB http server and use the http_poller input to fetch records over http.
  • Use mongoexport to a JSON or CSV file.
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.