Sqlite input plugin not working

Elastic Stack Logstash
1

Not sure what is wrong, but I am getting this output from the sqlite input plugin.

Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]: [2018-08-05T19:54:55,346][ERROR][logstash.pipeline        ] A plugin had an unr
ecoverable error. Will restart this plugin.                                                                                      
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]:   Pipeline_id:main
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]:   Plugin: <LogStash::Inputs::Sqlite path=>"/home/ubuntu/project/my_database.db", id=>"f6da85740e237058c8b03f94e561a31dec6e556744dd7affce05e471d7f5e9f0", enable_metric=>true, codec=><LogStash::Codecs::P
lain id=>"plain_a6bc860f-1a90-4bca-a0c6-f695da890569", enable_metric=>true, charset=>"UTF-8">, batch=>5>                         
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]:   Error: Missing Converter handling for full class name=org.jruby.RubyObjectVar
9, simple name=RubyObjectVar9                                                                                                    
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]:   Exception: Java::OrgLogstash::MissingConverterException
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]:   Stack: org.logstash.Valuefier.fallbackConvert(Valuefier.java:97)
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]: org.logstash.Valuefier.convert(Valuefier.java:75)
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]: org.logstash.ConvertedMap$1.visit(ConvertedMap.java:35)
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]: org.logstash.ConvertedMap$1.visit(ConvertedMap.java:29)
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]: org.jruby.RubyHash.visitLimited(RubyHash.java:662)
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]: org.jruby.RubyHash.visitAll(RubyHash.java:647)
Aug 05 19:54:55 ip-10-221-145-243 logstash[22850]: org.logstash.ConvertedMap.newFromRubyHash(ConvertedMap.java:69)

My config:

input {
  sqlite {
    path => "/home/ubuntu/project/my_database.db"
  }
}

output {
  stdout {}
}
2

I think it is getting a datatype in one of the columns that it does not know how to interpret. If you can figure out which column it is you may be able to cast it.

3

Using a filter like mutate and convert?

Edit: tried using mutate and convert and am getting this now:

[ERROR][logstash.pipeline        ] Error registering p
lugin {:pipeline_id=>"main", :plugin=>"#<LogStash::FilterDelegator:0x41dd60b5 @metric_events_out=org.jruby.proxy.org.logstash.inst
rument.metrics.counter.LongCounter$Proxy2 -  name: out value:0, @metric_events_in=org.jruby.proxy.org.logstash.instrument.metrics.
counter.LongCounter$Proxy2 -  name: in value:0, @metric_events_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCo
unter$Proxy2 -  name: duration_in_millis value:0, @id=\"46c8b0710b4b3e2c9b7c901e235f7d182b66d5c0411c5432a0b55b6c6a64ff15\", @klass
=LogStash::Filters::Mutate, @metric_events=#<LogStash::Instrument::NamespacedMetric:0x45e2ec56 @metric=#<LogStash::Instrument::Met
ric:0x2e3dd94 @collector=#<LogStash::Instrument::Collector:0x316973eb @agent=nil, @metric_store=#<LogStash::Instrument::MetricStor
e:0x1fe62b06 @store=#<Concurrent::Map:0x00000000000fac entries=4 default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x95433df>, @
fast_lookup=#<Concurrent::Map:0x00000000000fb0 entries=72 default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugi
ns, :filters, :\"46c8b0710b4b3e2c9b7c901e235f7d182b66d5c0411c5432a0b55b6c6a64ff15\", :events]>, @filter=<LogStash::Filters::Mutate
 convert=>{ enable_metric=>true, periodic_flush=>false>>", :error=>"translation missing: en.logstash.agent.configura
tion.invalid_plugin_register", :thread=>"#<Thread:0x1bd8fe03 run>"}
4

No, a CAST in the SQL. I think you need logstash to see the data as a different type as it first receives it.

5

Okay, I looked at the schema and there were some columns that didn't have a set type, so I deleted the table and created it again with those types explicitly set, but I'm still getting the original error message. Everything is just an integer, timestamp, or string.