# Filebeat vs Logstash

**URL:** <https://discuss.elastic.co/t/filebeat-vs-logstash/127485>\
**Category:** Logstash\
**Created:** [April 10, 2018, 12:57pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485 "2018-04-10T12:57:05Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 10, 2018, 12:57pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/1 "2018-04-10T12:57:06Z")

</div>

Please give me the solution for forwarding the log to logstash from filebeat, were the filebeat and logstash are in different VM

am getting the error of :  
2018-04-10T11:39:05.750Z ERROR pipeline/output.go:74 Failed to connect: dial tcp 192.168.2.5:5044: getsockopt: connection refused

I just want to use the only filebeat and logstash.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 10, 2018, 4:52pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/2 "2018-04-10T16:52:34Z")

</div>

Is logstash running?

---

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 11, 2018, 5:00am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/3 "2018-04-11T05:00:21Z")

</div>

yes, its running

---

<div class="post-metadata">

**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\
**Post date:** [April 11, 2018, 10:37am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/4 "2018-04-11T10:37:27Z")

</div>

Any chance logstash is only listening on localhost?  
Can you ping from the filebeat vm to the logstash vm and vise versa?

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [April 11, 2018, 10:40am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/5 "2018-04-11T10:40:13Z")

</div>

please share your logstash pipeline configuration here..??

---

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 11, 2018, 11:53am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/6 "2018-04-11T11:53:57Z")

</div>

its doing well

---

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 11, 2018, 12:43pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/7 "2018-04-11T12:43:13Z")

</div>

it is in /etc/logstash/conf.d/first.conf

# The # character at the beginning of a line indicates a comment. Use

# comments to describe your configuration.

input {  
beats {  
port =\> "5044"  
}  
}

# The filter part of this file is commented out to indicate that it is

# optional.

# filter {

# 

# }

output {  
stdout { codec =\> rubydebug }  
}  
~

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 11, 2018, 2:11pm UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/8 "2018-04-11T14:11:35Z")

</div>

Please format logs, configs and terminal input/output using the `</>`-Button or [markdown code fences](https://help.github.com/articles/creating-and-highlighting-code-blocks/#fenced-code-blocks). This forum uses Markdown to format posts. Without proper formatting, it can be very hard to read your posts. Proper formatting helps us to help you.

The `conection refused` normally appears because the remote host did refuse the connection. Service not running, port not available, firewall blocking connection.

Running from the filebeat host can you show us the output of:

```auto
ping 192.168.2.5

```

Also check with telnet:

```auto
telnet 192.168.2.5 5044

```

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [April 12, 2018, 3:21am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/9 "2018-04-12T03:21:58Z")

</div>

Also share the filebeat.yml file here..?

---

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 12, 2018, 7:06am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/10 "2018-04-12T07:06:03Z")

</div>

Now getting:  
018-04-12T06:32:46.770Z ERROR pipeline/output.go:74 Failed to connect: dial tcp 192.168.2.81:5044: getsockopt: connection refused

**filebeat.yml**

filebeat.prospectors:

- type: log

output.logstash:

hosts: ["192.168.2.81:5044"]

**file: /etc/logsatash/logstash.yml**

path.data: /var/lib/logstash

path.logs: /var/log/logstash

**file: /usr/share/logstash/logstash.conf**

input {  
beats {  
hosts =\> "127.0.0.1"  
port =\> 5044  
}  
}  
filter  
grok {  
match =\> { "message" =\> "%{COMBINEDAPACHELOG}"}  
}  
}  
output {  
stdout { codec =\> rubydebug }  
}

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [April 12, 2018, 7:13am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/11 "2018-04-12T07:13:27Z")

</div>

Hi @jawad846,

Lets go step by step.

In your logstash.conf you have mentioned "hosts". please comment this or remove from the file. it should be like :

> beats {  
> port =\> 5044  
> }

Please do this and restart the service and let me know the error if you are getting.

Thanks,  
Harsh Bajaj

---

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 12, 2018, 7:23am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/12 "2018-04-12T07:23:24Z")

</div>

the time of restart:  
[logstash@ip-192-168-2-81 logstash]$ sudo service logstash start  
logstash: unrecognized service  
[logstash@ip-192-168-2-81 logstash]$

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [April 12, 2018, 7:26am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/13 "2018-04-12T07:26:00Z")

</div>

Hi @jawad846,

please try to start with "Systemctl" or check the status if already start then restart it not start.

---

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 12, 2018, 7:27am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/14 "2018-04-12T07:27:24Z")

</div>

am using aws linux.

it also not working

and thanks for giving the support

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [April 12, 2018, 7:28am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/15 "2018-04-12T07:28:53Z")

</div>

were you able to start the service earlier with that command which you mentioned.

---

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 12, 2018, 7:29am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/16 "2018-04-12T07:29:25Z")

</div>

/bin/logstash -f logstash.conf

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [April 12, 2018, 7:31am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/17 "2018-04-12T07:31:25Z")

</div>

Please try to start with same command as you were using earlier and check.

---

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 12, 2018, 7:31am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/18 "2018-04-12T07:31:40Z")

</div>

WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults  
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console  
[INFO] 2018-04-12 07:30:05.634 [main] scaffold - Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"}  
[INFO] 2018-04-12 07:30:05.647 [main] scaffold - Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"}  
[WARN] 2018-04-12 07:30:06.196 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified  
[INFO] 2018-04-12 07:30:06.390 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=\>"6.2.3"}  
[INFO] 2018-04-12 07:30:06.546 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=\>9600}  
[ERROR] 2018-04-12 07:30:06.597 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, { at line 7, column 4 (byte 54) after filter \n ", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in`compile\_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in`map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in`initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in`execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in`with\_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in`block in converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in`converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in`block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

**this is RESULT**

---

<div class="post-metadata">

**Author:** ![jawad846](https://avatars.discourse-cdn.com/v4/letter/j/edb3f5/32.png) [@jawad846](https://discuss.elastic.co/u/jawad846)\
**Post date:** [April 12, 2018, 7:35am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/19 "2018-04-12T07:35:17Z")

</div>

this is the architecture

![00%20PM](https://us1.discourse-cdn.com/elastic/original/3X/f/1/f13fd254be1afd0a1b305569b0f3d11c89bee1c1.png)

all the logs need to be save in a single file

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [April 12, 2018, 8:00am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485/20 "2018-04-12T08:00:58Z")

</div>

As i can see your issue is with Logstash is not working your issue is not related to filebeat right?

> [@jawad846](#):
>
> WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
> 
> Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console

and in above logs i can see conf file path incorrect.

please run below command with conf file path like below.

> /bin/logstash -f /path/logstash.conf

Thanks,  
Harsh Bajaj

[Next page](https://discuss.elastic.co/t/filebeat-vs-logstash/127485.md?page=2)