I have developed a custom logstash output plugin for cassandra. Gem gets built successfully and plugin gets installed successfully too. However When I start the logstash, it crashes with the below message:
:message=>"The error reported is: \n Couldn't find any output plugin named 'cassandra'. Are you sure this is correct? Trying to load the cassandra output plugin resulted in this error: no such file to load -- cassandra"}
My plugin gemspec is as below:
Gem::Specification.new do |s|
s.name = 'logstash-output-cassandra'
s.version = "0.0.1"
s.licenses = ["Apache License (2.0)"]
s.summary = "This example output lets you store your data to Cassandra"
s.description = "This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program"
s.authors = ["Elastic"]
s.email = "info@elastic.co"
s.homepage = "http://www.elastic.co/guide/en/logstash/current/index.html"
s.require_paths = ["lib"]
# Files
s.files = ["lib/logstash/outputs/cassandra.rb"]
# Tests
s.test_files = s.files.grep(%r{^(test|spec|features)/})
# Special flag to let us know this is actually a logstash plugin
#s.metadata = { "logstash_plugin" => "true", "logstash_group" => "output" }
# Gem dependencies
s.add_runtime_dependency "logstash-core", ">= 1.4.2", "<= 2.1.0"
s.add_runtime_dependency "logstash-codec-plain", ">=0", "<= 3.0.0"
s.add_runtime_dependency "cassandra-driver", ">=0", "<= 3.0.0"
s.add_runtime_dependency "ione", "~>1.2"
s.add_development_dependency "bundler", "~>1.6"
s.add_development_dependency "rake", "~>10.0"
s.add_development_dependency "logstash-devutils"
end
My plugin code is as below:
# encoding: utf-8
require "logstash/outputs/base"
require "logstash/namespace"
require "cassandra"
class LogStash::Outputs::Cassandra < LogStash::Outputs::Base
config_name "cassandra"
# The keyspace to write to
config :keyspace, :validate => :string, :required => true
# Credentials for Cassandra DB:
config :username, :validate => :string, :default => "sonus", :required => true
config :password, :validate => :string, :default => "Sonus@123", :required => true
# The column family to write to
config :table, :validate => :string
# Cluster connection options
# If no connection options provided, connects to localhost by default
# (See http://datastax.github.io/ruby-driver/api/#cluster-class_method
# for the full list of options)
config :connection_options, :validate => :hash
# A mapping from event fields to column names
# config :event_schema, :validate => :hash, :required => true
# Index tables for efficient searching of the data
# The index table should include only two fields the event_id (by default)
# and an additional field and they should form a composite key
# e.g CREATE TABLE search_by_host (id uuid, host text) PRIMARY KEY (id, host)
# config :index_tables, :validate => :hash
public
def register
@connection_options ||= {}
cluster = Cassandra.cluster(@connection_options)
@session = cluster.connect(@keyspace)
end # def register
public
def receive(event)
return unless output?(event)
begin
recordsMapFromEvent = event.to_hash
event_id = nil;
if(recordsMapFromEvent.has_key? "uuid")
event_id = recordsMapFromEvent["uuid"]
end
if(event_id == nil)
event_id = (Cassandra::Uuid::Generator.new.uuid).to_s
end
if(recordsMapFromEvent.has_key? "type" and recordsMapFromEvent["type"]=="stats")
statType = recordsMapFromEvent["statType"]
hostName = recordsMapFromEvent["host"]
timeStamp = recordsMapFromEvent["TIMESTAMP"]
rejectKeys = ['beat','count','input_type','offset','version','source','type','statType','host']
mapCopy = recordsMapFromEvent
rejectKeys.each { |k| mapCopy.delete k}
insert = @session.prepare("INSERT INTO statistics(uid ,host ,statstype ,datetimestamp ,statistics) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)")
@session.execute(insert, arguments: [event_id, hostName, statType, timeStamp, mapCopy])
end #end top if
end # end begin
end # def receive
end # class LogStash::Outputs::Cassandra
Please suggest how to proceed further. I am struck here for the past couple of days. Please revert if you require any further info.