Sending data from LS to ES - Looks like a local machine config issue

(Leigh) #1

Hi can anyone advise if there are any specific settings required on a machine to allow LS to talk to ES. I've got everything installed and they both appear to startup ok however when I run a log file through LS I get errors. I got a colleague to run the same on his machine and it worked fine so I know its not the config file or the input file.

Logstash startup completed
←[33m {:exception=>Twitter::Error::Unauthorized, :backtrace=>["C:/logstash-1.5.1
1:in on_headers_complete'", "org/ruby_http_parser/<
<'", "C:/logstash-1.5.1/vendor/bundle/jruby/1.9/gems/twitter-5.12.0/lib/twitter/
streaming/response.rb:16:in <<'", "C:/logstash-1.5.1/vendor/bundle/jruby/1.9/ge
ms/twitter-5.12.0/lib/twitter/streaming/connection.rb:22:instream'", "C:/logst
nt.rb:116:in request'", "C:/logstash-1.5.1/vendor/bundle/jruby/1.9/gems/twitter
-5.12.0/lib/twitter/streaming/client.rb:36:infilter'", "C:/logstash-1.5.1/vend
r.rb:88:in run'", "C:/logstash-1.5.1/vendor/bundle/jruby/1.9/gems/logstash-core
-1.5.1-java/lib/logstash/pipeline.rb:176:ininputworker'", "C:/logstash-1.5.1/v
0:in `start_input'"], :level=>:warn}←[0m

Config file

input { 
  file {
    start_position => "beginning"
    type => "onboard"
    path => "C:\logstash-1.5.1\bin\OnboardServer03-20_May_logs\onboard_app.2015-04-22.txt"
    sincedb_path => "/dev/null"

output {
  elasticsearch {
    index => "onboard"
    host => localhost
  stdout { codec => rubydebug }

Error I'm seeing

←[33mConnection refused - Connection refused {:exception=># Connection refused - Connection refused>, :backtrace=>["org/jruby/ext/socket/Ru initialize'", "org/jruby/'", "C:
g/connection.rb:16:in stream'", "C:/logstash-1.5.1/vendor/bundle/jruby/1.9/gems
/twitter-5.12.0/lib/twitter/streaming/client.rb:116:inrequest'", "C:/logstash-
b:36:in filter'", "C:/logstash-1.5.1/vendor/bundle/jruby/1.9/gems/logstash-inpu
t-twitter-0.1.6/lib/logstash/inputs/twitter.rb:88:inrun'", "C:/logstash-1.5.1/
76:in inputworker'", "C:/logstash-1.5.1/vendor/bundle/jruby/1.9/gems/logstash-c
ore-1.5.1-java/lib/logstash/pipeline.rb:170:instart_input'"], :level=>:warn}←[

If this same configuration is run on another developers machine it works fine so there is a config on my desktop that isn't set correctly (both machines are running the same setup windows 8.1 running ES 1.7.1 and LS 1.5.1).

Any advice?


(Magnus Bäck) #2

The error message has nothing to do with the elasticsearch output. It's the twitter input that's complaining. I'm guessing you have an old configuration file laying around in /etc/logstash/conf.d.

(Leigh) #3

Thanks but I tried a completely fresh install of LS and ES and a new config file and still no luck. The same LS/ES setup and config file works perfectly fine on my colleagues machine.

Both machines are the same apart from one is on Windows 7 and the other on Windows 8.

(system) #4