# Logserver in AWS not receiving any logs **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [February 23, 2016, 7:49am UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463 "2016-02-23T07:49:48Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [February 23, 2016, 7:49am UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/1 "2016-02-23T07:49:48Z")
I have setup logserver using ELK in aws ubuntu instance. And I need to send syslogs from my production server to logserver. For that filebeat is installed in production server. Contents of configuration files are given below. elasticsearch.yml ``` bootstrap.mlockall: true network.host: 0.0.0.0 http.port: 9200 ``` kibana.yml ``` server.port: 5601 server.host: "172.30.0.28" ``` I have used nginx proxy server , and here is its 'default' server block. ``` server { listen 80; server_name aws_public_address; auth_basic "Restricted Access"; auth_basic_user_file /etc/nginx/htpasswd.users; location / { proxy_pass http://172.30.0.28:5601; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade ; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; allow all; } } ``` Logstash config file logstash\_syslog.conf ``` input { tcp { port => 5040 type => 'syslog' } } filter { if [type == 'syslog'] { grok { match => ['TimeCreated', "Date\(%{NUMBER:timestamp}\)"] } date { match => ["timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } } output { if [type == 'syslog'] { elasticsearch { hosts => ["localhost:9200"] } } } ``` filebeat.yml in production server ``` prospectors: - paths: - /var/log/syslog input_type: log document_type: syslog registry_file: "C:/ProgramData/filebeat/registry" output: logstash: hosts: ["aws_public_ip:5040"] ``` I couldn't see any logs in kibana view of logserver. Please help me to solve this problem. Thanks in advance ---
**Author:** ![ngv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ngv/32/6809_2.png) [@ngv](https://discuss.elastic.co/u/ngv)\ **Post date:** [February 23, 2016, 10:47am UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/2 "2016-02-23T10:47:39Z")
Hello, Did you allowed connections to this instance from your server/network? Also in your filebeat set logging on and set it to debug so you can see what happens. Here's an example of logging configuration for filebeat: logging: to\_files: true files: path: /var/log/mybeat/ name: filebeat.log level: debug ---
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [February 23, 2016, 11:19am UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/3 "2016-02-23T11:19:30Z")
@ngv I have allowed HTTP , SSH and ICMP (Echo Request and Echo Reply) protocols for the instance. And in my filebeat log I am getting.. ``` 2016-02-23T05:31:13-05:00 DBG Flushing spooler because of timeout. Events flushed: 0 2016-02-23T05:31:15-05:00 DBG End of file reached: /var/log/mysql.log; Backoff now. 2016-02-23T05:31:20-05:00 DBG Flushing spooler because of timeout. Events flushed: 0 2016-02-23T05:31:20-05:00 DBG Start next scan 2016-02-23T05:31:20-05:00 DBG scan path /var/log/mysql.log 2016-02-23T05:31:20-05:00 DBG Check file for harvesting: /var/log/mysql.log 2016-02-23T05:31:20-05:00 DBG Update existing file for harvesting: /var/log/mysql.log ``` ---
**Author:** ![ngv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ngv/32/6809_2.png) [@ngv](https://discuss.elastic.co/u/ngv)\ **Post date:** [February 23, 2016, 11:29am UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/4 "2016-02-23T11:29:03Z")
Hey, You need to create a rule that allows your production server/network external ip addresse(s) to connect to port 5040 on this instance, because none of the protocols listed by you is not used in this case. ---
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [February 23, 2016, 12:04pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/5 "2016-02-23T12:04:52Z")
Now I allowed All TCP for all port range for any address. Still no logs were sent. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [February 23, 2016, 5:33pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/6 "2016-02-23T17:33:59Z")
Can you share a little bit more of your debug log (especially the startup part) to see what happens? ---
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [February 24, 2016, 4:08am UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/7 "2016-02-24T04:08:03Z")
Ok .. Here is the starting part of filebeat.log ``` 2016-02-23T22:23:18-05:00 DBG Disable stderr logging 2016-02-23T22:23:18-05:00 DBG Initializing output plugins 2016-02-23T22:23:18-05:00 INFO GeoIP disabled: No paths were set under output.geoip.paths 2016-02-23T22:23:18-05:00 INFO Max Retries set to: 3 2016-02-23T22:23:18-05:00 DBG connect 2016-02-23T22:23:18-05:00 INFO Activated logstash as output plugin. 2016-02-23T22:23:18-05:00 DBG Create output worker 2016-02-23T22:23:18-05:00 DBG No output is defined to store the topology. The server fields might not be filled. 2016-02-23T22:23:18-05:00 INFO Publisher name: xdrgprod.ontash.local 2016-02-23T22:23:18-05:00 INFO Flush Interval set to: 1s 2016-02-23T22:23:18-05:00 INFO Max Bulk Size set to: 2048 2016-02-23T22:23:18-05:00 DBG create bulk processing worker (interval=1s, bulk size=2048) 2016-02-23T22:23:18-05:00 INFO Init Beat: filebeat; Version: 1.1.1 2016-02-23T22:23:18-05:00 INFO filebeat sucessfully setup. Start running. 2016-02-23T22:23:18-05:00 INFO Registry file set to: /var/lib/filebeat/registry 2016-02-23T22:23:18-05:00 INFO Loading registrar data from /var/lib/filebeat/registry 2016-02-23T22:23:19-05:00 DBG Set idleTimeoutDuration to 5s 2016-02-23T22:23:19-05:00 DBG File Configs: [/var/log/syslog] 2016-02-23T22:23:19-05:00 INFO Set ignore_older duration to 24h0m0s 2016-02-23T22:23:19-05:00 INFO Set scan_frequency duration to 10s 2016-02-23T22:23:19-05:00 INFO Input type set to: log 2016-02-23T22:23:19-05:00 INFO Set backoff duration to 1s 2016-02-23T22:23:19-05:00 INFO Set max_backoff duration to 10s 2016-02-23T22:23:19-05:00 INFO force_close_file is disabled 2016-02-23T22:23:19-05:00 DBG Waiting for 1 prospectors to initialise 2016-02-23T22:23:19-05:00 INFO Starting prospector of type: log 2016-02-23T22:23:19-05:00 DBG exclude_files: [] 2016-02-23T22:23:19-05:00 DBG scan path /var/log/syslog 2016-02-23T22:23:19-05:00 DBG Check file for harvesting: /var/log/syslog 2016-02-23T22:23:19-05:00 DBG Start harvesting unknown file: /var/log/syslog 2016-02-23T22:23:19-05:00 DBG Launching harvester on new file: /var/log/syslog 2016-02-23T22:23:19-05:00 DBG scan path /var/log/syslog 2016-02-23T22:23:19-05:00 DBG Check file for harvesting: /var/log/syslog 2016-02-23T22:23:19-05:00 DBG Update existing file for harvesting: /var/log/syslog 2016-02-23T22:23:19-05:00 DBG Not harvesting, file didn't change: /var/log/syslog 2016-02-23T22:23:19-05:00 DBG No pending prospectors. Finishing setup 2016-02-23T22:23:19-05:00 INFO All prospectors initialised with 0 states to persist 2016-02-23T22:23:19-05:00 INFO Starting Registrar 2016-02-23T22:23:19-05:00 INFO Start sending events to output 2016-02-23T22:23:19-05:00 DBG harvest: "/var/log/syslog" (offset snapshot:0) 2016-02-23T22:23:19-05:00 INFO Harvester started for file: /var/log/syslog 2016-02-23T22:23:19-05:00 DBG full line read 2016-02-23T22:23:19-05:00 DBG full line read ``` ---
**Author:** ![ngv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ngv/32/6809_2.png) [@ngv](https://discuss.elastic.co/u/ngv)\ **Post date:** [February 25, 2016, 12:29pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/8 "2016-02-25T12:29:20Z")
Can you try telnet the elastic server on port 5040 and see if you can connect from the server you are sending logs from. Also i've just noticed in your config file that you are NOT using beats as input: input { **tcp {** port =\> 5040 type =\> 'syslog' } } if you are sending logs via filebeat this should look like this: input { **beats {** port =\> 5040 type =\> 'syslog' } } ---
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [February 25, 2016, 12:47pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/9 "2016-02-25T12:47:02Z")
Sorry, I got a new issue with kibana. kibana is not responding now. I can start kibana and its showing ' **started**'. But when checked the status, it is showing **kibana is not running**. I checked kibana log, but there is nothing special for this case. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [February 28, 2016, 7:34pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/10 "2016-02-28T19:34:57Z")
@babeesh Does that mean your previous issue is resolved? In case of a Kibana issue, it is best to post it in the kibana forum: [https://discuss.elastic.co/c/kibana](https://discuss.elastic.co/c/kibana) ---
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [February 29, 2016, 9:16am UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/11 "2016-02-29T09:16:07Z")
@ruflin I already posted the kibana issue in kibana forum. That was a disk full error in aws EC2. I reinistalled elasticsearch and now kibana running. But the issue in this thread still exists. I can't see any logs in kibana view. I made some modifications in logstash configuration. ``` input { beats { type => 'xdrgprod' port => 5044 codec=> json{ charset => 'UTF-8' } } } filter { if [type == 'xdrgprod'] { grok { match => ['TimeCreated', "Date\(%{NUMBER:timestamp}\)"] } date { match => [timestamp, "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } } output { if [type == 'xdrgprod']{ elasticsearch { hosts => ["172.30.0.28:9200"] } stdout {codec => rubydebug} } } ``` And **logstash.log** ``` {:timestamp=>"2016-02-29T09:12:39.156000+0000", :message=>"JSON parse failure. Falling back to plain-text", :error=>#, :data=>"Feb 28 16:47:21 xdrgprod /usr/bin/filebeat[19210]: reader.go:138: End of file reached: /var/log/mysql.log; Backoff now.", :level=>:error} {:timestamp=>"2016-02-29T09:12:39.156000+0000", :message=>"JSON parse failure. Falling back to plain-text", :error=>#, :data=>"Feb 28 16:47:24 xdrgprod /usr/bin/filebeat[19210]: prospector.go:179: Start next scan", :level=>:error} ``` Edit: Adding a little bit more **logstash.log** ``` {:timestamp=>"2016-02-29T09:49:11.242000+0000", :message=>"Beats Input: Remote connection closed", :peer=>"172.30.0.28:36463", :exception=>#, :level=>:warn} {:timestamp=>"2016-02-29T10:20:13.923000+0000", :message=>"SIGTERM received. Shutting down the pipeline.", :level=>:warn} ``` ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [February 29, 2016, 12:11pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/12 "2016-02-29T12:11:17Z")
Which version of the beats-input-plugin do you use? Which version of LS? I saw in the logs above you use filebeat 1.1.1. There seem to be some issue with the json code. Can you remove it to test if it works without it? ---
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [February 29, 2016, 12:18pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/13 "2016-02-29T12:18:42Z")
How to check beats-input -plugin version? LS 2.2.x I checked removing json code. But no luck. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [February 29, 2016, 12:38pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/14 "2016-02-29T12:38:23Z")
Check the docs here: [https://www.elastic.co/guide/en/beats/libbeat/1.1/logstash-installation.html#logstash-input-update](https://www.elastic.co/guide/en/beats/libbeat/1.1/logstash-installation.html#logstash-input-update) But to be honest if you are on 2.2.x you should have the most recent version. If you remove the codec, do you still get the unsupported protocol for the beats input in the LS protocol? ---
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [February 29, 2016, 1:47pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/15 "2016-02-29T13:47:40Z")
Input beats version : logstash-input-beats (2.1.3) And now **logstash.log** shows the error ``` {:timestamp=>"2016-02-29T12:46:53.806000+0000", :message=>"SIGTERM received. Shutting down the pipeline.", :level=>:warn} {:timestamp=>"2016-02-29T12:47:09.303000+0000", :message=>"Beats Input: Remote connection closed", :peer=>"68.236.192.171:42428", :exception=>#, :level=>:warn} {:timestamp=>"2016-02-29T12:47:09.500000+0000", :message=>"Beats Input: Remote connection closed", :peer=>"68.236.192.171:42427", :exception=>#, :level=>:warn} ``` I used [This](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04) tutorial to setup ELK. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [February 29, 2016, 2:31pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/16 "2016-02-29T14:31:45Z")
This last message is about logstash being shut down due to SIGTERM. Server being shutdown/rebooted? Anyone killing or restarting logstash? Let's start with a very minimal config having filebeat forward to logstash and logstash print to console. We reconfigure the filebeat registry file for testing purpose. The registry\_test file should be deleted after every single run of filebeat to have reproducible tests. For filebeat: ```auto filebeat prospectors: - paths: - /var/log/syslog input_type: log document_type: syslog registry_file: "C:/ProgramData/filebeat/registry_test" output: logstash: hosts: ["aws_public_ip:5044"] ``` and for logstash (to be run from command line as we print events to console for testing): ```auto input { beats { port => 5044 } } output { stdout { codec => rubydebug } } ``` Is this port (input plugin) only used for input by filebeat? Can you see events being send to logstash? ---
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [March 1, 2016, 7:39am UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/17 "2016-03-01T07:39:55Z")
I tried your way with different port number . Regisrty file is **registry\_file: /var/lib/filebeat/registry** But still no luck. logstash log files... **logstash.log** `{:timestamp=>"2016-03-01T07:25:11.879000+0000", :message=>"Beats Input: Remote connection closed", :peer=>"68.236.192.171:35429", :exception=>#, :level=>:warn}` **logstash.log.1** ``` {:timestamp=>"2016-03-01T05:14:38.264000+0000", :message=>"Beats input: The circuit breaker has detected a slowdown or stall in the pipeline, the input is closing the current connection and rejecting new connection until the pipeline recover.", :exception=>LogStash::Inputs::BeatsSupport::CircuitBreaker::HalfOpenBreaker, :level=>:warn} {:timestamp=>"2016-03-01T05:14:44.721000+0000", :message=>"SIGTERM received. Shutting down the pipeline.", :level=>:warn} ``` ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [March 1, 2016, 11:09am UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/18 "2016-03-01T11:09:26Z")
need logs from filebeat and logstash. These logs are just telling me the TCP connection filebeat-\>logstash has been closed. Have any message been send from filebeat to logstash? ---
**Author:** ![babeesh](https://avatars.discourse-cdn.com/v4/letter/b/e47c2d/32.png) [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Post date:** [March 1, 2016, 12:48pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/19 "2016-03-01T12:48:50Z")
It works now with basic syslog configuration . I disabled firewall in client pc. But I didn't get an immediate result. Logs were sent after 30 minutes approximately. Thank you all ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 9:55pm UTC](https://discuss.elastic.co/t/logserver-in-aws-not-receiving-any-logs/42463/20 "2017-07-05T21:55:14Z")