# Filebeat vs Logstash **URL:** \ **Category:** Logstash\ **Created:** [April 10, 2018, 12:57pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485 "2018-04-10T12:57:05Z")\ **Posts on this page:** 20\ **Page:** 3
**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\ **Post date:** [April 12, 2018, 9:31am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/41 "2018-04-12T09:31:32Z")
So what's the output of `netstat -anutp | grep 5044` now? What do the logstash logs say? @harshbajaj16 Don't think filebeat needs a restart unless there is an error that needs to be fixed on that side. It will connect on it's own after the retry timer. ---
**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Post date:** [April 12, 2018, 9:32am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/42 "2018-04-12T09:32:08Z")
sorry guys, same issue i tried :-- filebeat -e -c filebeat.yml -d "publish" in filebeat instance: | 2018-04-12T09:30:34.195Z | ERROR | pipeline/output.go:74 | Failed to connect: dial tcp 192.168.2.79:5044: getsockopt: connection refused | | --- | --- | --- | --- | | 2018-04-12T09:30:38.201Z | ERROR | pipeline/output.go:74 | Failed to connect: dial tcp 192.168.2.79:5044: getsockopt: connection refused | | 2018-04-12T09:30:46.210Z | ERROR | pipeline/output.go:74 | Failed to connect: dial tcp 192.168.2.79:5044: getsockopt: connection refused | logstash is not still generated ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [April 12, 2018, 9:32am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/43 "2018-04-12T09:32:49Z")
> [@jawad846](#): > > done, and restarted the logstash assuming he is saying done and restart means he is not getting any error. 😄 ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [April 12, 2018, 9:33am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/44 "2018-04-12T09:33:39Z")
please forgot the filebeat first start logstash and check each and everything is fine in logstash or not...?? ---
**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\ **Post date:** [April 12, 2018, 9:33am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/45 "2018-04-12T09:33:45Z")
Please share the logs and netstat output can't help with the filebeat messages that say can't connect to something dead. Need the dead guys info 🙂 @harshbajaj16 Well... 😛 ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [April 12, 2018, 9:35am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/46 "2018-04-12T09:35:36Z")
> [@jawad846](#): > > bin/logstash -f /usr/share/logstash/logstash.conf after run this what messages are coming pls paste here??? ---
**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Post date:** [April 12, 2018, 9:36am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/47 "2018-04-12T09:36:09Z")
netstat -anutp | grep 5044 there is no result for this root@ip-192-168-2-79:~# service logstash status ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset Active: active (running) since Thu 2018-04-12 09:28:22 UTC; 6min ago Main PID: 17391 (java) Tasks: 14 Memory: 311.9M CPU: 3min 26.780s CGroup: /system.slice/logstash.service └─17391 /usr/bin/java -Xms256m -Xmx1g -XX:+UseParNewGC -XX:+Use root@ip-192-168-2-79:~# netstat -plnt Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:\* LISTEN 1331/sshd tcp6 0 0 :::22 :::\* LISTEN 1331/sshd root@ip-192-168-2-79:~# ---
**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\ **Post date:** [April 12, 2018, 9:38am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/48 "2018-04-12T09:38:06Z")
Ok what about logstash logs? Logstash will be shown as loaded and running but newer versions will not kill the process. They will start a loop that expects the error to be fixed. If you check the logs you will see a pipeline starting then error logs then pipeline shutdown. Logstash is still not working. We need to know why. We need it's logs. ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [April 12, 2018, 9:38am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/49 "2018-04-12T09:38:43Z")
> [@jawad846](#): > > netstat -anutp | grep 5044 your o/p should like this. tcp 0 0 x.x.x.x:51288 x.x.x.x:5044 ESTABLISHED 12545/filebeat ---
**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Post date:** [April 12, 2018, 9:42am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/50 "2018-04-12T09:42:52Z")
root@ip-192-168-2-79:/usr/share/logstash# bin/logstash -f /usr/share/logstash/logstash.conf WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console [INFO] 2018-04-12 09:38:35.889 [main] scaffold - Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"} [INFO] 2018-04-12 09:38:36.138 [main] scaffold - Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"} [WARN] 2018-04-12 09:38:48.662 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified [INFO] 2018-04-12 09:38:53.588 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=\>"6.2.3"} [INFO] 2018-04-12 09:38:58.110 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=\>9600} [INFO] 2018-04-12 09:39:27.175 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>1, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50} [INFO] 2018-04-12 09:39:45.998 [[main]-pipeline-manager] beats - Beats inputs: Starting input listener {:address=\>"0.0.0.0:5044"} [INFO] 2018-04-12 09:39:49.111 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Pipeline started succesfully {:pipeline\_id=\>"main", :thread=\>"#\"} [INFO] 2018-04-12 09:39:49.928 [[main]\1, :pipelines=\>["main"]} link from '/var/lib/cloud/instance' =\> '/var/lib/cloud/instances/i-069c32c16c4732237'", "offset" =\> 53759, "host" =\> "ip-192-168-2-223", "tags" =\> [ [0] "beats\_input\_codec\_plain\_applied", [1] "\_grokparsefailure" ] } { "@version" =\> "1", "prospector" =\> { "type" =\> "log" }, "beat" =\> { "hostname" =\> "ip-192-168-2-223", "name" =\> "ip-192-168-2-223", "version" =\> "6.2.3" }, "@timestamp" =\> 2018-04-12T08:01:01.673Z, "source" =\> "/var/log/cloud-init.log", "message" =\> "2018-04-12 07:49:32,724 - util.py[DEBUG]: Reading from /var/lib/cloud/instances/i-069c32c16c4732237/datasource (quiet=False)", "offset" =\> 53884, "host" =\> "ip-192-168-2-223", "tags" =\> [ [0] "beats\_input\_codec\_plain\_applied", [1] "\_grokparsefailure" ] } { "@version" =\> "1", "prospector" =\> { "type" =\> "log" }, "beat" =\> { "hostname" =\> "ip-192-168-2-223", "name" =\> "ip-192-168-2-223", "version" =\> "6.2.3" }, "@timestamp" =\> 2018-04-12T08:01:01.673Z, "source" =\> "/var/log/cloud-init.log", "message" =\> "2018-04-12 07:49:32,724 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/i-069c32c16c4732237/datasource - wb: [644] 39 bytes", "offset" =\> 54014, "host" =\> "ip-192-168-2-223", "tags" =\> [ [0] "beats\_input\_codec\_plain\_applied", [1] "\_grokparsefailure" ] } { "@version" =\> "1", "prospector" =\> { "type" =\> "log" }, "beat" =\> { "hostname" =\> "ip-192-168-2-223", "name" =\> "ip-192-168-2-223", "version" =\> "6.2.3" }, "@timestamp" =\> 2018-04-12T08:01:01.673Z, "source" =\> "/var/log/cloud-init.log", "message" =\> "2018-04-12 07:49:32,724 - util.py[DEBUG]: Writing to /var/lib/cloud/data/previous-datasource - wb: [644] 39 bytes", "offset" =\> 54128, "host" =\> "ip-192-168-2-223", "tags" =\> [ [0] "beats\_input\_codec\_plain\_applied", [1] "\_grokparsefailure" ] } ---
**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Post date:** [April 12, 2018, 9:46am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/51 "2018-04-12T09:46:40Z")
thanks guys for helpping root@ip-192-168-2-79:/var/log/logstash# netstat -anutp | grep 5044 tcp6 0 0 :::5044 :::\* LISTEN 17430/java root@ip-192-168-2-79:/var/log/logstash# and now getting the logs ---
**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\ **Post date:** [April 12, 2018, 9:48am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/52 "2018-04-12T09:48:41Z")
And i was wondering what the heck is going on since you posted healthy info logs and actual events 😛 Good to know your system is up and working. 🙂 ---
**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Post date:** [April 13, 2018, 5:29am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/53 "2018-04-13T05:29:56Z")
hi guys, the logstash port will run only when we run the command: #bin/logsatash -f logstash.conf and am saving to one of the file but once i stop running the command ( Ctrl+C ) no logs are updating to the file, i.e. service is not running. is there any command, to run this logstash every time, like the apache and other service are running, the log file is need to be updated every time automatically please help me out. and other requirement is ; I had 2 instances running with apache and i had configured filebeat on both, and i need to filter only apache and mysql logs separately from both the instances, and save to other instance (where logstash is running) separately in a different file. eg: apache logs from inst. 1 ====\> ---------------------------------------------save both logs to logstash inst. in a single file-- apache.log apache logs from inst. 2 ====\> @sancroth @harshbajaj16 @marcelo @magnusbaeck ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [April 13, 2018, 5:32am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/54 "2018-04-13T05:32:02Z")
> [@jawad846](#): > > #bin/logsatash -f logstash.conf Yes run below command : > #bin/logsatash -f logstash.conf & Thanks, Harsh Bajaj ---
**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Post date:** [April 13, 2018, 5:53am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/55 "2018-04-13T05:53:53Z")
its for background process..? right...! ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [April 13, 2018, 5:54am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/56 "2018-04-13T05:54:46Z")
yes right....👍 ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [April 13, 2018, 6:01am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/57 "2018-04-13T06:01:36Z")
> [@jawad846](#): > > I had 2 instances running with apache and i had configured filebeat on both, and i need to filter only apache and mysql logs separately from both the instances, and save to other instance (where logstash is running) separately in a different file. > > eg: > > apache logs from inst. 1 ====\> > > ---------------------------------------------save both logs to logstash inst. in a single file-- apache.log > > apache logs from inst. 2 ====\> Hi @jawad846, Logstash can't save anything as its a pipeline not a storage device but you can use "Filter" to filter out your logs according to the requirement and pass the same into elasticsearch and elasticsearch save the data in json format in index which you can see on kibana. Thanks, Harsh Bajaj ---
**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Post date:** [April 13, 2018, 6:06am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/58 "2018-04-13T06:06:51Z")
logstash can save the logs of input to a single file, here i need to make it separate for different service, its possible by using filter, saving file is not working. let me create multiple pipeline and check, @harshbajaj16 please help me out for output filter { if [type] == "apache" { grok { match =\> { "message" =\> '%{WORD:VirtualHost} %{IPORHOST:clientip} "%{WORD:balancer\_worker}" %{WORD:remote\_log\_name} %{WORD:user} [%{HTTPDAT E:timestamp}] "%{WORD:Method} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} %{NUMBER:Response\_size} "%{WORD:referrer}" "%{WOR D:agent}" %{NUMBER:Time\_taken} %{NUMBER:bytes\_received} %{NUMBER:bytes\_sents}' } } ``` date { match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"] } geoip { source => "clientip" } ``` } } ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [April 13, 2018, 6:12am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/59 "2018-04-13T06:12:56Z")
> [@jawad846](#): > > logstash can save the logs of input to a single file, How 😉 ?? > [@jawad846](#): > > filter { > > if [type] == "apache" { > > grok { > > match =\> { "message" =\> '%{WORD:VirtualHost} %{IPORHOST:clientip} "%{WORD:balancer\_worker}" %{WORD:remote\_log\_name} %{WORD:user} [%{HTTPDAT > > E:timestamp}] "%{WORD:Method} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} %{NUMBER:Response\_size} "%{WORD:referrer}" "%{WOR > > D:agent}" %{NUMBER:Time\_taken} %{NUMBER:bytes\_received} %{NUMBER:bytes\_sents}' > > } it seems ok and i'm not aware what exactly o/p you want this filter pass only apache logs and give to the elasticsearch. ---
**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Post date:** [April 13, 2018, 6:20am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/60 "2018-04-13T06:20:41Z")
output { stdout {codec =\> rubydebug} file { path =\> "/home/ubuntu/logstash.log" } } @harshbajaj16 am not using elasticsearch [Previous page](https://discuss.elastic.co/t/filebeat-vs-logstash/127485.md?page=2) [Next page](https://discuss.elastic.co/t/filebeat-vs-logstash/127485.md?page=4)