X-Pack Machine Learning Lab 1 Logstash Error

Elastic Community and Ecosystem Elastic Training
1

I'm having an issue getting the stocks data into kibana using logstash.

Here is my log file:

[2018-01-11T22:17:04,976][INFO ][logstash.inputs.metrics ] Monitoring License OK
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................[2018-01-11T22:17:05,631][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<NoMethodError: undefined method <' for nil:NilClass>, :backtrace=>["/Users/<USER>/Downloads/logstash-6.1.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:213:inget_event_type'", "/Users//Downloads/logstash-6.1.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:165:in event_action_params'", "/Users/<USER>/Downloads/logstash-6.1.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:39:inevent_action_tuple'", "/Users//Downloads/logstash-6.1.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:34:in block in multi_receive'", "org/jruby/RubyArray.java:2486:inmap'", "/Users//Downloads/logstash-6.1.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:34:in multi_receive'", "/Users/<USER>/Downloads/logstash-6.1.1/logstash-core/lib/logstash/output_delegator_strategies/shared.rb:13:inmulti_receive'", "/Users//Downloads/logstash-6.1.1/logstash-core/lib/logstash/output_delegator.rb:50:in multi_receive'", "/Users/<USER>/Downloads/logstash-6.1.1/logstash-core/lib/logstash/pipeline.rb:487:inblock in output_batch'", "org/jruby/RubyHash.java:1343:in each'", "/Users/<USER>/Downloads/logstash-6.1.1/logstash-core/lib/logstash/pipeline.rb:486:inoutput_batch'", "/Users//Downloads/logstash-6.1.1/logstash-core/lib/logstash/pipeline.rb:438:in worker_loop'", "/Users/<USER>/Downloads/logstash-6.1.1/logstash-core/lib/logstash/pipeline.rb:393:inblock in start_workers'"]}`

2

Sorry for the slow response, but it took me some time to reproduce the error you are seeing. What is probably happening is that you set up a different password for the elastic user than the password that is configured in the stocks.conf Logstash configuration file.

If you open stocks.conf with a text editor, you can change the changeme password near the bottom of the file into the password that you have chosen when you set up X-Pack.

You will have to do the same for the other .conf Logstash files that you will be using in the labs.

2 Likes
3

Thanks I reinstalled everything and was able to import the stocks data. The only issue I'm seeing is that it's only importing 1625 of the 1859 records in the CSV. I'm using the latest version of ELK; are there any known issues getting the data imported with that version?

4

Unfortunately, it seems that Logstash 6.1.0 has introduced a bug which causes it to drop some documents. The Logstash team is looking into the issue right now. As a workaround, you could use Logstash 6.0.1, which should work fine with the latest version of Elasticsearch, and which does not seem to have the problem.

5

Hello there, so my test ELK is at 6.1.2 and recently 6.1.3 was released, do you know if .3 addresses this error because I'm seeing it in all the .csv files I'm putting in as well.

6

Hey T,
sorry but I don't think it was fixed. Here is the main issue which is only included in 6.2.0 which will be released soon (https://github.com/elastic/logstash/pull/9024). I know the bug is annoying, but the labs are still valid and all the experience is pretty much the same. I would not be too worried about those few hundred documents that could not be indexed.

Please, let us know if you have other issues.

Cheers,
Pablo

7

Actually, 6.1.3 has fixed the issue. :slightly_smiling_face: Please let us know if you still run into problems with 6.1.3.

8

Oh great! I'll install 6.1.3 for my test environment, I'll let you know if its still a problem, thank you!

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.