# Rsyslog to send logs to logstash

**URL:** <https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973>\
**Category:** Logstash\
**Created:** [February 21, 2021, 5:54am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973 "2021-02-21T05:54:49Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Prabhath\_samarasingh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prabhath_samarasingh/32/84203_2.png) [@Prabhath\_samarasingh](https://discuss.elastic.co/u/Prabhath_samarasingh)\
**Post date:** [February 21, 2021, 5:54am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/1 "2021-02-21T05:54:49Z")

</div>

Hi,  
Configured rsyslog to send logs to logstash.  
But kibana dosen't receive.  
Please help.

OS : Ubuntu 20.04  
`</>`  
**cat /etc/elasticsearch/elasticsearch.yml**

# 

# Elasticsearch performs poorly when the system is swapping the memory.

# 

# ---------------------------------- Network -----------------------------------

# 

# Set the bind address to a specific IP (IPv4 or IPv6):

# 

#network.host: localhost  
network.host: 172.20.111.199

# 

# Set a custom port for HTTP:

# 

http.port: 9200

# 

# For more information, consult the network module documentation.

# 

**cat /etc/logstash/conf.d/02-beats-input.conf**

# This input block will listen on port 10514 for logs to come in.

# host should be an IP on the Logstash server.

# codec =\> "json" indicates that we expect the lines we're receiving to be in JSON format

# type =\> "rsyslog" is an optional identifier to help identify messaging streams in the pipeline.

input {  
udp {  
host =\> "172.20.111.199"  
port =\> 10514  
codec =\> "json"  
type =\> "rsyslog"  
}  
}

**# cat /etc/logstash/conf.d/30-elasticsearch-output.conf**

# This is an empty filter block. You can later add other filters here to further process

# your log lines

filter { }

# This output block will send all events of type "rsyslog" to Elasticsearch at the configured

# host and port into daily indices of the pattern, "rsyslog-YYYY.MM.DD"

output {  
if [type] == "rsyslog" {  
elasticsearch {  
hosts =\> ["172.20.111.199:9200"]  
}  
}  
}

**cat /etc/filebeat/filebeat.yml**

# ------------------------------ Logstash Output -------------------------------

output.logstash:

# The Logstash hosts

#hosts: ["172.20.111.199:10514"]  
hosts: ["172.20.111.199:5044"]

**cat /etc/kibana/kibana.yml**

# The URLs of the Elasticsearch instances to use for all your queries.

elasticsearch.hosts: ["[http://172.20.111.199:9200](http://172.20.111.199:9200)"]  
#elasticsearch.hosts: ["[http://localhost:9200](http://localhost:9200)"]

**root@elk:/etc/rsyslog.d# ls**  
01-json-template.conf 20-ufw.conf 21-cloudinit.conf 50-default.conf 60-output.conf

**# cat 01-json-template.conf**

template(name="json-template"  
type="list") {  
constant(value="{")  
constant(value=""@timestamp":"") property(name="timereported" dateFormat="rfc3339")  
constant(value="","@version":"1")  
constant(value="","message":"") property(name="msg" format="json")  
constant(value="","sysloghost":"") property(name="hostname")  
constant(value="","severity":"") property(name="syslogseverity-text")  
constant(value="","facility":"") property(name="syslogfacility-text")  
constant(value="","programname":"") property(name="programname")  
constant(value="","procid":"") property(name="procid")  
constant(value=""}\n")  
}

**# cat 60-output.conf**

# This line sends all lines to defined IP address at port 10514,

# using the "json-template" format template

_._@172.20.111.199:10514;json-template

**cat /var/log/syslog**

Feb 21 11:04:02 elk filebeat[4471]: 2021-02-21T11:04:02.296+0530#011ERROR#011[publisher\_pipeline\_output]#011pipeline/output.go:154#011Failed to connect to backoff(async(tcp://172.20.111.199:5044)): dial tcp 172.20.111.199:5044: connect: connection refused  
Feb 21 11:04:02 elk filebeat[4471]: 2021-02-21T11:04:02.296+0530#011INFO#011[publisher\_pipeline\_output]#011pipeline/output.go:145#011Attempting to reconnect to backoff(async(tcp://172.20.111.199:5044)) with 1168 reconnect attempt(s)

```
indent preformatted text by 4 spaces

```

Kibana didnt received any log.

Please help to rectify this issue.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 21, 2021, 6:14am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/2 "2021-02-21T06:14:29Z")

</div>

Hi @Prabhath_samarasingh it is very hard to read the yml code

Could you please edit and format all the code in your post above using by selecting it all and using the format button `</>` above , then perhaps we can help

One thing I do already notice is that you did not define the index in the Logstash output section see [here](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-index) so the data is probably going to

`logstash-%{+yyyy.MM.dd}`

---

<div class="post-metadata">

**Author:** ![Prabhath\_samarasingh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prabhath_samarasingh/32/84203_2.png) [@Prabhath\_samarasingh](https://discuss.elastic.co/u/Prabhath_samarasingh)\
**Post date:** [February 21, 2021, 6:51am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/3 "2021-02-21T06:51:55Z")

</div>

Sorry ..  
Can you give some idea to create index for this configuration.  
I'm really new to this system.

```
root@elk:/etc/rsyslog.d# curl '172.20.111.199:9200/_cat/indices?v'

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open filebeat-7.11.0-2021.02.19 IQWbdWcrR8ONaHOAawN7jA 1 1 17890 0 4.6mb 4.6mb
yellow open filebeat-7.11.0-2021.02.17 WBpjRiN9TKmqcci3i2VFEg 1 1 78584 0 15.7mb 15.7mb
yellow open logstash-2021.02.19-000001 fmYY0L9lSC-TzhI64CSKAA 1 1 378891728 0 51.1gb 51.1gb
yellow open filebeat-7.11.0-2021.02.18 CpKNzF91S2C2Gp48is5KdA 1 1 24552 0 6.5mb 6.5mb
green open .apm-agent-configuration ca-FRHwdTUqasVeWwHkHnw 1 0 0 0 208b 208b
yellow open filebeat-7.11.0-2021.02.17-000001 RON3HsOhSL2g9luXu04dlQ 1 1 0 0 208b 208b
green open .kibana_1 4M_sIPucR7C0PR0DoPaU-g 1 0 1608 27 2.5mb 2.5mb
green open .kibana-event-log-7.11.0-000001 6jbVzcKzQpeHi7KplCjkgA 1 0 13 0 13.5kb 13.5kb
green open .apm-custom-link iSYao0AUTouxuFFy73N08g 1 0 0 0 208b 208b
green open .kibana_task_manager_1 PUfBaYswS1SzmsVwggX9xg 1 0 8 3991 640.8kb 640.8kb
green open .async-search 26Qpix7XR2Gq8DZ-02ik8A 1 0 0 0 3.7kb 3.7kb
yellow open tutorial Iy_3a_QqRiuGxCRdh6bhdA 1 1 1 0 3.8kb 3.8kb
yellow open logstash-2021.02.20-000002 G9yofbfOQy250jI9UGgIYA 1 1 84345626 0 8.3gb 8.3gb
root@elk:/etc/rsyslog.d#
```

---

<div class="post-metadata">

**Author:** ![Prabhath\_samarasingh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prabhath_samarasingh/32/84203_2.png) [@Prabhath\_samarasingh](https://discuss.elastic.co/u/Prabhath_samarasingh)\
**Post date:** [February 21, 2021, 7:24am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/4 "2021-02-21T07:24:09Z")

</div>

cat /etc/logstash/conf.d/30-elasticsearch-output.conf

```
output {
  if [type] == "rsyslog" {
    elasticsearch {
      hosts => ["172.20.111.199:9200"]
      index => "rsyslog-%{+YYYY.MM.dd}"
    }
  }
}

```

cat /var/log/syslog

```
 Feb 21 12:49:59 elk logstash[12530]: Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
Feb 21 12:49:59 elk logstash[12530]: [2021-02-21T12:49:59,813][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.11.0", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.8+10 on 11.0.8+10 +indy +jit [linux-x86_64]"}
Feb 21 12:50:00 elk logstash[12530]: [2021-02-21T12:50:00,411][FATAL][logstash.runner] Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting.
Feb 21 12:50:00 elk logstash[12530]: [2021-02-21T12:50:00,418][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
Feb 21 12:50:00 elk logstash[12530]: org.jruby.exceptions.SystemExit: (SystemExit) exit
Feb 21 12:50:00 elk logstash[12530]: #011at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.13.0.jar:?]
Feb 21 12:50:00 elk logstash[12530]: #011at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.13.0.jar:?]
Feb 21 12:50:00 elk logstash[12530]: #011at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]
Feb 21 12:50:00 elk systemd[1]: logstash.service: Main process exited, code=exited, status=1/FAILURE
Feb 21 12:50:00 elk systemd[1]: logstash.service: Failed with result 'exit-code'.
Feb 21 12:50:00 elk systemd[1]: logstash.service: Scheduled restart job, restart counter is at 10.

root@elk:~#
root@elk:~# ps -ef | grep logstash
root 2903 2579 0 13:19 pts/0 00:00:00 grep --color=auto logstash
root@elk:~#
root@elk:~#
root@elk:~#
root@elk:~#
root@elk:~# /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/logstash.conf
Using JAVA_HOME defined java: /usr/lib/jvm/java-11-openjdk-amd64/
WARNING, using JAVA_HOME while Logstash distribution comes with a bundled JDK
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2021-02-21T13:19:40,192][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.11.0", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.10+9-Ubuntu-0ubuntu1.20.04 on 11.0.10+9-Ubuntu-0ubuntu1.20.04 +indy +jit [linux-x86_64]"}
[2021-02-21T13:19:40,721][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-02-21T13:19:41,322][INFO][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/logstash.conf"}
[2021-02-21T13:19:41,349][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2021-02-21T13:19:41,606][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2021-02-21T13:19:46,496][INFO][logstash.runner] Logstash shut down.
[2021-02-21T13:19:46,515][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.13.0.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.13.0.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]
root@elk:~#
```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 21, 2021, 5:11pm UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/5 "2021-02-21T17:11:47Z")

</div>

Hi @Prabhath_samarasingh

It very hard to tell what you are trying to do ... first if you want our help please format your code with the by selecting the text and the pressing the format button. I did some of that to your last post see how much easier it is to read.

 ![Screen Shot 2021-02-21 at 8.52.25 AM](https://us1.discourse-cdn.com/elastic/original/3X/1/f/1f3ac944ef9da8dc3fc936ed4b5f0e969dfff41b.png)

2nd you are just running a bunch of commands ...

You cat this file

cat /etc/logstash/conf.d/30-elasticsearch-output.conf

```auto
output {
  if [type] == "rsyslog" {
    elasticsearch {
      hosts => ["172.20.111.199:9200"]
      index => "rsyslog-%{+YYYY.MM.dd}"
    }
  }
}

```

Then you run this command

```auto
/usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/logstash.conf

```

and it says

```auto
[2021-02-21T13:19:41,322][INFO][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/logstash.conf"}

```

which means that file does not exist or is not a valid logstash configurations file.

I think you are close, but I can not tell where the problem

This looks correct....

```auto
output {
  if [type] == "rsyslog" {
    elasticsearch {
      hosts => ["172.20.111.199:9200"]
      index => "rsyslog-%{+YYYY.MM.dd}"
    }
  }
}

```

you can try running this to check this will load the configurationss from the pipelines.yml

` /usr/share/logstash/bin/logstash --path.settings /etc/logstash`

It also looks like Logstash was already writing to...

`logstash-2021.02.20-000002`

**If not, lets start back at the very beginning...**

Please confirm you are trying to run this architecture

`rsyslog -> filebeat -> logstash -> elasticsearch`

Please run the following **and format the result** s and I will take a look.

`cat /etc/logstash/logstash.yml`

`cat /etc/logstash/pipelines.yml`

`ls -l /etc/logstash/conf.d`

---

<div class="post-metadata">

**Author:** ![Prabhath\_samarasingh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prabhath_samarasingh/32/84203_2.png) [@Prabhath\_samarasingh](https://discuss.elastic.co/u/Prabhath_samarasingh)\
**Post date:** [February 22, 2021, 4:32am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/6 "2021-02-22T04:32:52Z")

</div>

Thanks for your kind help.  
Yes I need to run " `rsyslog -> filebeat -> logstash -> elasticsearch`" architechture.

```
root@elk:~# /usr/share/logstash/bin/logstash --path.settings /etc/logstash
Using JAVA_HOME defined java: /usr/lib/jvm/java-11-openjdk-amd64/
WARNING, using JAVA_HOME while Logstash distribution comes with a bundled JDK
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2021-02-22T09:51:01,745][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.11.0", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.10+9-Ubuntu-0ubuntu1.20.04 on 11.0.10+9-Ubuntu-0ubuntu1.20.04 +indy +jit [linux-x86_64]"}
[2021-02-22T09:51:04,818][INFO][org.reflections.Reflections] Reflections took 59 ms to scan 1 urls, producing 23 keys and 47 values 
[2021-02-22T09:51:05,747][INFO][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://172.20.111.199:9200/]}}
[2021-02-22T09:51:05,959][WARN][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://172.20.111.199:9200/"}
[2021-02-22T09:51:06,012][INFO][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2021-02-22T09:51:06,018][WARN][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2021-02-22T09:51:06,097][INFO][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//172.20.111.199:9200"]}
[2021-02-22T09:51:06,154][INFO][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
[2021-02-22T09:51:06,195][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>32, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>4000, "pipeline.sources"=>["/etc/logstash/conf.d/02-beats-input.conf", "/etc/logstash/conf.d/30-elasticsearch-output.conf"], :thread=>"#<Thread:0x79ca1f67 run>"}
[2021-02-22T09:51:06,257][INFO][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2021-02-22T09:51:07,774][INFO][logstash.javapipeline][main] Pipeline Java execution initialization time {"seconds"=>1.57}
[2021-02-22T09:51:07,814][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"}
[2021-02-22T09:51:07,908][INFO][logstash.inputs.udp][main][a6c77bc59fe85849e4ded46b15bb40ebe99fe2061050875e32b231be0d1e6c94] Starting UDP listener {:address=>"172.20.111.199:10514"}
[2021-02-22T09:51:07,920][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2021-02-22T09:51:07,981][INFO][logstash.inputs.udp][main][a6c77bc59fe85849e4ded46b15bb40ebe99fe2061050875e32b231be0d1e6c94] UDP listener started {:address=>"172.20.111.199:10514", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
[2021-02-22T09:51:08,168][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
```

---

<div class="post-metadata">

**Author:** ![Prabhath\_samarasingh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prabhath_samarasingh/32/84203_2.png) [@Prabhath\_samarasingh](https://discuss.elastic.co/u/Prabhath_samarasingh)\
**Post date:** [February 22, 2021, 4:47am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/7 "2021-02-22T04:47:13Z")

</div>

> [@stephenb](#):
>
> cat /etc/logstash/pipelines.yml

```
oot@elk:~# cat /etc/logstash/logstash.yml

# ------------ Data path ------------------
path.data: /var/lib/logstash
#
# ------------ Pipeline Settings --------------
pipeline.ordered: auto
#
# ------------ Pipeline Configuration Settings --------------

# ------------ HTTP API Settings -------------

# ------------ Cloud Settings ---------------

# ------------ Queuing Settings --------------

#
# ------------ Dead-Letter Queue Settings --------------
#

# ------------ Metrics Settings --------------
0
#
# ------------ Debugging Settings --------------

# ------------ Other Settings --------------

# ------------ X-Pack Settings (not applicable for OSS build)--------------

#####

cat /etc/logstash/pipelines.yml

root@elk:~# cat /etc/logstash/pipelines.yml
# This file is where you define your pipelines. You can define multiple.
# For more information on multiple pipelines, see the documentation:
# https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html

- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"
root@elk:~# 

root@elk:~# ls -l /etc/logstash/conf.d
total 16
-rw-r--r-- 1 root root 412 Feb 20 18:47 02-beats-input.conf
-rw-r--r-- 1 root root 61 Feb 20 12:35 02-beats-input.conf-Org
-rw-r--r-- 1 root root 433 Feb 21 12:46 30-elasticsearch-output.conf
-rw-r--r-- 1 root root 422 Feb 19 18:22 30-elasticsearch-output.conf-Org
root@elk:~#

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 22, 2021, 5:11am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/8 "2021-02-22T05:11:09Z")

</div>

OK the logstash.yml looks incomplete / wrong.... that can not be the whole file, we can come back to that later..

What happened when you ran logstash?

Can you please now share

`cat /etc/logstash/conf.d/02-beats-input.conf`

and

`cat /etc/logstash/conf.d/30-elasticsearch-output.conf`

It looks like logstash is listening on port `10514` but for beats it should usually be listening on `5044` and not UDP so we need to look at the `02-beats-input.conf`

```auto
[2021-02-22T09:51:07,908][INFO][logstash.inputs.udp][main][a6c77bc59fe85849e4ded46b15bb40ebe99fe2061050875e32b231be0d1e6c94] Starting UDP listener {:address=>"172.20.111.199:10514"}

```

The input looks like logstash is trying to listen to rsyslog directly not through beats...

So you need to decide if you want

`rsyslog -> filebeat -> logstash -> elasticsearch`

or what you appear to have configured, there is no reason you have to use filebeat unless you want to, logstash can directly listen to syslog as well.

`rsyslog -> logstash -> elasticsearch`

Which one?

---

<div class="post-metadata">

**Author:** ![Prabhath\_samarasingh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prabhath_samarasingh/32/84203_2.png) [@Prabhath\_samarasingh](https://discuss.elastic.co/u/Prabhath_samarasingh)\
**Post date:** [February 22, 2021, 5:56am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/9 "2021-02-22T05:56:23Z")

</div>

I used filebeat, because it use as a filter on kibana.Is it not the way to analysis my logs on kibana?

Yes I planned to configure as this...  
" `rsyslog -> filebeat -> logstash -> elasticsearch`"

[\*\*\* Before integrate the rsyslog,kibana got te graphs.]

 ![Screenshot from 2021-02-22 11-25-10](https://us1.discourse-cdn.com/elastic/original/3X/c/3/c389a3ff1c36df3fae35cddb8c06f7b9d6f0a1f1.png)

```
root@elk:~# cat /etc/logstash/conf.d/02-beats-input.conf
# This input block will listen on port 10514 for logs to come in.
# host should be an IP on the Logstash server.
# codec => "json" indicates that we expect the lines we're receiving to be in JSON format
# type => "rsyslog" is an optional identifier to help identify messaging streams in the pipeline.
input {
  udp {
    host => "172.20.111.199"
    port => 10514
    codec => "json"
    type => "rsyslog"
  }
}

root@elk:~# 
root@elk:~# cat /etc/logstash/conf.d/30-elasticsearch-output.conf
# This is an empty filter block. You can later add other filters here to further process
# your log lines
filter { }
# This output block will send all events of type "rsyslog" to Elasticsearch at the configured
# host and port into daily indices of the pattern, "rsyslog-YYYY.MM.DD"
output {
  if [type] == "rsyslog" {
    elasticsearch {
      hosts => ["172.20.111.199:9200"]
      index => "rsyslog-%{+YYYY.MM.dd}"
    }
  }
}
root@elk:~#
```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 22, 2021, 6:20am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/10 "2021-02-22T06:20:56Z")

</div>

If you want to use Filebeat

Your Filebeat should use the [syslog input](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-syslog.html) which should be listening to the syslog server on the correct IP and correct port.

and the [Logstash output](https://www.elastic.co/guide/en/beats/filebeat/current/logstash-output.html) it should point to the Logstash server at port 5044

```
/etc/logstash/conf.d/02-beats-input.conf

```

Should look like.

```
input {
  beats {
    port => 5044
    type => rsyslog
  }
}

```

Rsyslog -\> Filebeat syslog input (syslog server and port)/ Filebeat Logstash output (Logstash server  
and port 5044) -\> Logstash beats input 5044 / Logstash elasticsearch -\> elasticsearch

---

<div class="post-metadata">

**Author:** ![Prabhath\_samarasingh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prabhath_samarasingh/32/84203_2.png) [@Prabhath\_samarasingh](https://discuss.elastic.co/u/Prabhath_samarasingh)\
**Post date:** [February 22, 2021, 6:46am UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/11 "2021-02-22T06:46:55Z")

</div>

```
root@elk:/etc/rsyslog.d# pwd
/etc/rsyslog.d
root@elk:/etc/rsyslog.d# ls
01-json-template.conf 20-ufw.conf 21-cloudinit.conf 50-default.conf 60-output.conf
root@elk:/etc/rsyslog.d# 

root@elk:/etc/rsyslog.d# cat 60-output.conf

*.* @172.20.111.199:10514;json-template

root@elk:/etc/rsyslog.d# cd ../filebeat/
root@elk:/etc/filebeat# ls
fields.yml filebeat.reference.yml filebeat.yml filebeat.yml_2021.02.20.11.30.47 filebeat.yml-ORG modules.d

root@elk:/etc/filebeat# cat filebeat.yml

# ============================== Filebeat inputs ===============================

filebeat.inputs:

- type: log

  
    - /var/log/*.log
  

# filestream is an experimental input. It is going to replace log input in the future.
- type: filestream

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/*.log
  

# ============================== Filebeat modules ==============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

  
# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 1

# ================================== General ===================================

# ================================= Dashboards =================================

# =================================== Kibana ===================================

setup.kibana:

  

# =============================== Elastic Cloud ================================

# ================================== Outputs ===================================

# Configure what output to use when sending the data collected by the beat.

# ---------------------------- Elasticsearch Output ----------------------------

# ------------------------------ Logstash Output -------------------------------
output.logstash:
  # The Logstash hosts
      hosts: ["172.20.111.199:5044"]

# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

# ================================== Logging ===================================

# ============================= X-Pack Monitoring ==============================

# ============================== Instrumentation ===============================

# ================================= Migration ==================================

# This allows to enable 6.7 migration aliases
#migration.6_to_7.enabled: true

```

Is this configuration correct ?  
In 60-output.conf will ships logs to logstash on port 10514?  
And is there any changers on filebeat.yml ?

```auto

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 22, 2021, 2:41pm UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/12 "2021-02-22T14:41:57Z")

</div>

> [@Prabhath\_samarasingh](#):
>
> In 60-output.conf will ships logs to logstash on port 10514?

I am not a rsyslog expert.

Again I thought you want to ship to rsyslog to Filebeat first so why do you keep saying ship to Logstash.

I think you need to slow down and decide what you want to do pretty much everything you need I gave put in these posts.

Personally I would start with simple

rsyslog -\> logstash -\> elasticsearch

which is basically what you had working in the very beginning. Filebeat adds another layer which seems to be confusing you. I am not sure where you downloaded all these configs from but I would go back to a working configuration and iterate from there.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 22, 2021, 2:42pm UTC](https://discuss.elastic.co/t/rsyslog-to-send-logs-to-logstash/264973/13 "2021-03-22T14:42:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.