# Configuring filebeat to send specified logs to E.L.K server's logstash HELP! **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 18, 2016, 10:53am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830 "2016-03-18T10:53:10Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![brayndasilva](https://avatars.discourse-cdn.com/v4/letter/b/2bfe46/32.png) [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)\ **Post date:** [March 18, 2016, 10:53am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/1 "2016-03-18T10:53:10Z")
Hi there, About two days ago I started looking into ELK and decided to try it out on a few machines. My set-up is the following: 1 E.L.K server, 2 Client servers and 1 Laptop to access Kibana. I have followed the official guide of installation on Ubuntu 14.04 and reached the part where I start configuring my client servers to send out the logs to the logstash. I have created the following filebeat.yml stored in /etc/filebeat/ ``` filebeat: prospectors: - paths: - /var/log/auth.log - /var/log/syslog # - /var/log/*.log input_type: log document_type: syslog registry_file: /var/lib/filebeat/registry output: logstash: hosts: ["192.168.71.104:5044"] bulk_max_size: 1024 tls: certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"] shipper: logging: files: rotateeverybytes: 10485760 # = 10MB ``` I think it's fine and should work out, but when I try to restart the service by executing `sudo service filebeat restart` it gives me the following message: `* Restarting Sends log files to Logstash or directly to Elasticsearch. filebeat 2016/03/18 10:28:35.803779 transport.go:125: ERR SSL client failed to connect with: dial tcp 192.168.71.104:5044: getsockopt: connection refused` First I thought something was wrong with the certificate that I created on the E.L.K server and transferred over, but I triple checked it and even created new ones and it still gave me that error. What is going on? My logstash should be properly configured over at the E.L.K server, if necessary I can show the config files. There's 3. I also tried to troubleshoot a little further and noticed that the logstash service on the E.L.K server was not running. I tried to restart it and it successfully started, but after a while I checked the status again it told me that filebeat was not running. Shouldn't it be up at all times? Hopefully someone can help me out, thanks. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [March 18, 2016, 12:20pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/2 "2016-03-18T12:20:09Z")
> [@brayndasilva](#): > > if necessary I can show the config files. There's 3 Please do. Surround it with `
`tags so we don't lose the formatting. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [March 18, 2016, 2:13pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/3 "2016-03-18T14:13:51Z")
> [@brayndasilva](#): > > - Restarting Sends log files to Logstash or directly to Elasticsearch. filebeat 2016/03/18 10:28:35.803779 transport.go:125: ERR SSL client failed to connect with: dial tcp 192.168.71.104:5044: getsockopt: connection refused Is logstash running? Can you ping target machine? Can you telnet into logstash? What's your logstash config? ---
**Author:** ![brayndasilva](https://avatars.discourse-cdn.com/v4/letter/b/2bfe46/32.png) [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)\ **Post date:** [March 18, 2016, 2:21pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/4 "2016-03-18T14:21:44Z")
Logstash config: /etc/logstash/conf.d/02-beats-input.conf ``` input { beats { port => 5044 ssl => true ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt" ssl_key => "/etc/pki/tls/private/logstash-forwarder.key" } } ``` /etc/logstash/conf.d/10-syslog-filter.conf ``` filter { if [type] == "syslog" { grok { match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" } add_field => ["received_at", "%{@timestamp}"] add_field => ["received_from", "%{host}"] } syslog_pri { } date { match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } } ``` /etc/logstash/conf.d/30-elasticsearch-output.conf ``` output { elasticsearch { hosts => ["localhost:9200"] sniffing => true manage_template => false index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } } ``` @steffens I can ping the target machine just fine, haven't tried to telnet into logstash. How do I do that? Logstash is not running, when I try to start it manually it will go back to not running status shortly after. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [March 18, 2016, 3:58pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/5 "2016-03-18T15:58:08Z")
Can you look for logstash log output why it's not running? for telnet use `telnet `. ---
**Author:** ![brayndasilva](https://avatars.discourse-cdn.com/v4/letter/b/2bfe46/32.png) [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)\ **Post date:** [March 21, 2016, 9:27am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/6 "2016-03-21T09:27:28Z")
Hi, Thank you and sorry for the late reply. I can not telnet into logstash or into the server at all. ELK Server \> Client Server and the other way around. It tells me that the connection is refused. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [March 21, 2016, 11:32am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/7 "2016-03-21T11:32:06Z")
cause logstash is not running. Check for logs from logstash why it's not running. ---
**Author:** ![brayndasilva](https://avatars.discourse-cdn.com/v4/letter/b/2bfe46/32.png) [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)\ **Post date:** [March 21, 2016, 12:07pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/8 "2016-03-21T12:07:08Z")
Hi, thank you for your fast reply. I'm new to this and and it's my first experience with Ubuntu as well, but am very interested and appreciate your help. How would I be able to see my logstash logs? Are they saved in a specific directory? Sorry! ---
**Author:** ![mourlos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mourlos/32/10788_2.png) [@mourlos](https://discuss.elastic.co/u/mourlos)\ **Post date:** [March 21, 2016, 1:45pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/9 "2016-03-21T13:45:41Z")
If you want to see possible error messages or log information on the logstash server you can do: tail -100 /var/log/logstash/logstash.log tail -100 /var/log/logstash/logstash.err From the error message you get, it could also be a firewall rule that block the communication between the two servers ---
**Author:** ![brayndasilva](https://avatars.discourse-cdn.com/v4/letter/b/2bfe46/32.png) [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)\ **Post date:** [March 21, 2016, 3:00pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/10 "2016-03-21T15:00:22Z")
Damn, the log told me a lot. I can elaborate on this if necessary. In short: I had to wrong cert location set up + a few typos on the input configuration file. I fixed those and everything is working! Thanks a lot, I really appreciate it. ---
**Author:** ![frengki](https://avatars.discourse-cdn.com/v4/letter/f/65b543/32.png) [@frengki](https://discuss.elastic.co/u/frengki)\ **Post date:** [April 6, 2016, 3:55am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/11 "2016-04-06T03:55:42Z")
i want to ask how about my error after i do the command in my server: tail -100 /var/log/logstash/logstash.log {:timestamp=\>"2016-04-06T09:02:54.238000+0700", :message=\>"The error reported is: \n pattern %{HOST:service} not defined"} please help me. Thanks ---
**Author:** ![mourlos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mourlos/32/10788_2.png) [@mourlos](https://discuss.elastic.co/u/mourlos)\ **Post date:** [April 6, 2016, 6:05am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/12 "2016-04-06T06:05:10Z")
This means that although you have defined a pattern "HOST" in your logstash config, the pattern is not defined in the pattern directory. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [April 6, 2016, 10:52am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/13 "2016-04-06T10:52:15Z")
your config (I guess for elasticsearch output) is faulty? ---
**Author:** ![bv4wolf](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bv4wolf](https://discuss.elastic.co/u/bv4wolf)\ **Post date:** [September 17, 2016, 12:16am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/14 "2016-09-17T00:16:41Z")
Hi @brayndasilva, can you help me how to fix the "host" ? I got same error. -( ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [September 19, 2016, 12:53pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/15 "2016-09-19T12:53:28Z")
@bv4wolf this topic is rather old. please start a new discussion including logs + config files if possible (use `` button to format file content). ---
**Author:** ![bv4wolf](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bv4wolf](https://discuss.elastic.co/u/bv4wolf)\ **Post date:** [September 20, 2016, 12:59am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/16 "2016-09-20T00:59:11Z")
Thanks @steffens. I've know how to fix it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 9:50pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-send-specified-logs-to-e-l-k-servers-logstash-help/44830/17 "2017-07-05T21:50:36Z")