# Rsyslog is not forwarding logs to elasticsearch **URL:** \ **Category:** Logstash\ **Created:** [January 1, 2023, 3:05pm UTC](https://discuss.elastic.co/t/rsyslog-is-not-forwarding-logs-to-elasticsearch/322266 "2023-01-01T15:05:59Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Aksel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aksel/32/115432_2.png) [@Aksel](https://discuss.elastic.co/u/Aksel)\ **Post date:** [January 1, 2023, 3:05pm UTC](https://discuss.elastic.co/t/rsyslog-is-not-forwarding-logs-to-elasticsearch/322266/1 "2023-01-01T15:05:59Z")
I'm trying to configure rsyslog to send logs to logstash and then forward them to elasticsearch. I have create a config file `/etc/rsyslog.d/60-output.conf` with the following content: ```auto *.* @localhost:10514;json-template ``` and a template file `/etc/rsyslog.d/01-json-template.conf` with the following content: ```auto template(name="json-template" type="list") { constant(value="{") constant(value="\"@timestamp\":\"") property(name="timereported" dateFormat="rfc3339") constant(value="\",\"@version\":\"1") constant(value="\",\"message\":\"") property(name="msg" format="json") constant(value="\",\"sysloghost\":\"") property(name="hostname") constant(value="\",\"severity\":\"") property(name="syslogseverity-text") constant(value="\",\"facility\":\"") property(name="syslogfacility-text") constant(value="\",\"programname\":\"") property(name="programname") constant(value="\",\"procid\":\"") property(name="procid") constant(value="\"}\n") } ``` Then I restarted rsyslog service. And for logstash I created a config file `/etc/logstash/conf.d/logstash.conf` with the following content: ```auto input { udp { port => 10514 codec => "json" type => "rsyslog" } } filter { } output { if [type] == "rsyslog" { elasticsearch { hosts => ["localhost:9200"] } } } ``` Then I restarted logstash. When I run `sudo netstat -tulpn | grep 10514` I get this: ```auto user@rsyslog-server:/var/log$ sudo netstat -tulpn | grep 10514 udp 0 0 0.0.0.0:10514 0.0.0.0:* 5327/java ``` so Logstash is listening on port 10514. To verify the elasticsearch input I run `curl -XGET 'http://localhost:9200/logstash-*/_search?q=*&pretty'` but this doesn't return any results: ```auto { "took" : 0, "timed_out" : false, "_shards" : { "total" : 0, "successful" : 0, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 0, "relation" : "eq" }, "max_score" : 0.0, "hits" : [] } } ``` I'm using: - rsyslogd 8.2208.0 (aka 2022.08) - logstash 7.17.8 - elastisearch 7.17.8 How can I solve this ? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 1, 2023, 5:19pm UTC](https://discuss.elastic.co/t/rsyslog-is-not-forwarding-logs-to-elasticsearch/322266/2 "2023-01-01T17:19:57Z")
Hi @Aksel For debug...Try taking this out... > [@Aksel](#): > > `codec => "json"` Also you could add `stdout{}` in the output section outside the `if` to see if you are actually getting any messages ---
**Author:** ![Aksel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aksel/32/115432_2.png) [@Aksel](https://discuss.elastic.co/u/Aksel)\ **Post date:** [January 1, 2023, 6:01pm UTC](https://discuss.elastic.co/t/rsyslog-is-not-forwarding-logs-to-elasticsearch/322266/3 "2023-01-01T18:01:29Z")
Thnaks for your response, I changed my configuration to: ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/d/0da29031c0a27bff3a00409faa18338bf9b5e30d.png) Then I started logstash with this config: `/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --verbose` This is the output: ```auto Using bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console [WARN] 2023-01-01 18:50:48.417 [main] runner - DEPRECATION WARNING: The flag ["--verbose"] has been deprecated, please use "--log.level=info" instead. [INFO] 2023-01-01 18:50:48.423 [main] runner - Starting Logstash {"logstash.version"=>"7.17.8", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.17+8 on 11.0.17+8 +indy +jit [linux-x86_64]"} [INFO] 2023-01-01 18:50:48.426 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true] [WARN] 2023-01-01 18:50:48.689 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified [INFO] 2023-01-01 18:50:49.682 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false} [INFO] 2023-01-01 18:50:50.739 [Converge PipelineAction::Create
] Reflections - Reflections took 50 ms to scan 1 urls, producing 119 keys and 419 values [WARN] 2023-01-01 18:50:51.350 [Converge PipelineAction::Create
] plain - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [WARN] 2023-01-01 18:50:51.396 [Converge PipelineAction::Create
] udp - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [WARN] 2023-01-01 18:50:51.438 [Converge PipelineAction::Create
] plain - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [WARN] 2023-01-01 18:50:51.479 [Converge PipelineAction::Create
] elasticsearch - Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode. [INFO] 2023-01-01 18:50:51.658 [[main]-pipeline-manager] elasticsearch - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]} [INFO] 2023-01-01 18:50:51.953 [[main]-pipeline-manager] elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}} [WARN] 2023-01-01 18:50:52.174 [[main]-pipeline-manager] elasticsearch - Restored connection to ES instance {:url=>"http://localhost:9200/"} [INFO] 2023-01-01 18:50:52.199 [[main]-pipeline-manager] elasticsearch - Elasticsearch version determined (7.17.8) {:es_version=>7} [WARN] 2023-01-01 18:50:52.200 [[main]-pipeline-manager] elasticsearch - Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7} [INFO] 2023-01-01 18:50:52.276 [[main]-pipeline-manager] elasticsearch - Config is not compliant with data streams. `data_stream => auto` resolved to `false` [INFO] 2023-01-01 18:50:52.362 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/logstash.conf"], :thread=>"#"} [INFO] 2023-01-01 18:50:52.380 [Ruby-0-Thread-10: :1] elasticsearch - Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled} [INFO] 2023-01-01 18:50:53.026 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>0.66} [INFO] 2023-01-01 18:50:53.093 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"} [WARN] 2023-01-01 18:50:53.158 [[main]1, :running_pipelines=>[:main], :non_running_pipelines=>[]} [INFO] 2023-01-01 18:50:53.206 [[main]"0.0.0.0:10514"} [INFO] 2023-01-01 18:50:53.244 [[main]"0.0.0.0:10514", :receive_buffer_bytes=>"106496", :queue_size=>"2000"} ``` It seems that nothing is happening ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 1, 2023, 6:08pm UTC](https://discuss.elastic.co/t/rsyslog-is-not-forwarding-logs-to-elasticsearch/322266/4 "2023-01-01T18:08:14Z")
Yup So it seems like your rsyslog is not actually sending anything because anything you receive you should show up... You could just try to netcat a line or two to the UDP port and That should show up in your logstash put. ---
**Author:** ![Aksel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aksel/32/115432_2.png) [@Aksel](https://discuss.elastic.co/u/Aksel)\ **Post date:** [January 1, 2023, 6:10pm UTC](https://discuss.elastic.co/t/rsyslog-is-not-forwarding-logs-to-elasticsearch/322266/5 "2023-01-01T18:10:53Z")
I'm sorry, but how can I do that ? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 1, 2023, 6:24pm UTC](https://discuss.elastic.co/t/rsyslog-is-not-forwarding-logs-to-elasticsearch/322266/6 "2023-01-01T18:24:41Z")
Google is your friend 🙂 > **[Sending TCP/UDP packets using Netcat](https://help.ubidots.com/en/articles/937233-sending-tcp-udp-packets-using-netcat)** > > What is Netcat? And, how to use it to send TCP/UDP packets to a remote server But make sure you use `127.0.0.1` or your IP not `localhost` ```auto $ nc -u 127.0.0.1 10514 Hello from UDP Land ``` Then type in / paste in a couple line... they should show up in the logstash logs.. and actually even in elasticsearch On the logstash side ```auto { "event" => { "original" => "Hello from UDP Land\n" }, "host" => { "ip" => "127.0.0.1" }, "@timestamp" => 2023-01-01T18:24:15.154811Z, "message" => "Hello from UDP Land\n", "@version" => "1", "type" => "rsyslog" } ``` this is the logstash config that produced that with Version 8.5.3 your with 7.17.x the index name will be different.. other than that it should be same ```auto input { udp { port => 10514 type => "rsyslog" } } filter { } output { if [type] == "rsyslog" { elasticsearch { hosts => ["localhost:9200"] } } stdout {} } ``` And the logs showed up in Elasticsearch ```auto GET logs-generic-default/_search { "took": 2, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 8, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": ".ds-logs-generic-default-2023.01.01-000001", "_id": "FRmTboUBhuzPI0SRwbCj", "_score": 1, "_source": { "event": { "original": """Hello from UDP Land """ }, "host": { "ip": "127.0.0.1" }, "@timestamp": "2023-01-01T18:24:15.154811Z", "message": """Hello from UDP Land """, "@version": "1", "type": "rsyslog", "data_stream": { "type": "logs", "dataset": "generic", "namespace": "default" } } } .... ``` ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 1, 2023, 6:37pm UTC](https://discuss.elastic.co/t/rsyslog-is-not-forwarding-logs-to-elasticsearch/322266/7 "2023-01-01T18:37:25Z")
Oh and @Aksel I forgot to welcome you to the community...so Welcome! 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 29, 2023, 6:38pm UTC](https://discuss.elastic.co/t/rsyslog-is-not-forwarding-logs-to-elasticsearch/322266/8 "2023-01-29T18:38:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.