# Filebeat pushs syslog logs to elasticsearch through Logstash

**URL:** <https://discuss.elastic.co/t/filebeat-pushs-syslog-logs-to-elasticsearch-through-logstash/205149>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 24, 2019, 8:25pm UTC](https://discuss.elastic.co/t/filebeat-pushs-syslog-logs-to-elasticsearch-through-logstash/205149 "2019-10-24T20:25:54Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vladpov](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@Vladpov](https://discuss.elastic.co/u/Vladpov)\
**Post date:** [October 24, 2019, 8:25pm UTC](https://discuss.elastic.co/t/filebeat-pushs-syslog-logs-to-elasticsearch-through-logstash/205149/1 "2019-10-24T20:25:54Z")

</div>

Hi everyone!

I'm trying to push syslog logs to elasticsearch by using Filebeat and Logstash. How should my configuration files look like?

```
#=========================== Filebeat inputs =============================

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: false

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
- /var/log/*.log
#- c:\programdata\elasticsearch\logs\*

#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

#================================ Processors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

```

My logstash conf.d file looks like

```
input {
  beats {
port => 5044
  }
}

filter {
  grok { match => { "message" => "%{COMBINEDAPACHELOG}" } }
  geoip { source => "clientip" }
}

output {
  elasticsearch {
   hosts => ["localhost:9200"]
   index => "geoip3"
  }
}

```

Everything looks like it's working but I cannot see any data coming to elasticsearch ☹

I'd like to use logger command and see the log message in elasticsearch via Kibana

Thanks for any help ))

---

<div class="post-metadata">

**Author:** ![kumarabhi](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kumarabhi](https://discuss.elastic.co/u/kumarabhi)\
**Post date:** [October 24, 2019, 10:36pm UTC](https://discuss.elastic.co/t/filebeat-pushs-syslog-logs-to-elasticsearch-through-logstash/205149/2 "2019-10-24T22:36:21Z")

</div>

To debug any issue, it is better to have a simple use case.  
You should first make sure Filebeat is reading the files and data from Filebeat is reaching Logstash.

Ensure Filebeat is reading files : --  
-------------------------------------------------  
1) Add logging (refer filebeat\_reference.yml) for Filebeat.

```
#================================ Logging ======================================
# There are four options for the log output: file, stderr, syslog, eventlog
# The file output is the default.

# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
logging.level: info

# Enable debug output for selected components. To enable all selectors use ["*"]
# Other available selectors are "beat", "publish", "service"
# Multiple selectors can be chained.
#logging.selectors: []

# Send all logging output to stderr. The default is false.
#logging.to_stderr: false

# Send all logging output to syslog. The default is false.
#logging.to_syslog: false

# Send all logging output to Windows Event Logs. The default is false.
#logging.to_eventlog: false

# If enabled, Filebeat periodically logs its internal metrics that have changed
# in the last period. For each metric that changed, the delta from the value at
# the beginning of the period is logged. Also, the total values for
# all non-zero internal metrics are logged on shutdown. The default is true.
#logging.metrics.enabled: true

# The period after which to log the internal metrics. The default is 30s.
logging.metrics.period: 30s

# Logging to rotating files. Set logging.to_files to false to disable logging to
# files.
logging.to_files: true
logging.files:
  # Configure the path where the logs are written. The default is the logs directory
  # under the home path (the binary location).
  path: /var/log/filebeat

  # The name of the files where the logs are written to.
  name: filebeat

  # Configure log file size limit. If limit is reached, log file will be
  # automatically rotated
  rotateeverybytes: 10485760 # = 10MB

  # Number of rotated log files to keep. Oldest files will be deleted first.
  keepfiles: 7

  # The permissions mask to apply when rotating log files. The default value is 0600.
  # Must be a valid Unix-style file permissions mask expressed in octal notation.
  permissions: 0600

  # Enable log file rotation on time intervals in addition to size-based rotation.
  # Intervals must be at least 1s. Values of 1m, 1h, 24h, 7*24h, 30*24h, and 365*24h
  # are boundary-aligned with minutes, hours, days, weeks, months, and years as
  # reported by the local system clock. All other intervals are calculated from the
  # Unix epoch. Defaults to disabled.
  interval: 0

  # Rotate existing logs on startup rather than appending to the existing
  # file. Defaults to true.
  # rotateonstartup: true

# Set to true to log messages in JSON format.
logging.json: false

```

1. Start filebeat by passing your config file explicitly  
/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml (alter this command based on your local machine settings)

2. After starting Filebeat, add a new file in /var/log folder

3. Check filebeat log (/var/log/filebeat/filebeat) that the harvester started for new file

## Logstash receiving data from Filebeat :-

Instead of sending Logstash output to ES please direct it to stdout.

```
Logstash.conf
------------------------

input {
  beats {
    port => 5044
  }
}

output {
	stdout { 
	     codec => rubydebug { } 
	}
}
```

---

<div class="post-metadata">

**Author:** ![Vladpov](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@Vladpov](https://discuss.elastic.co/u/Vladpov)\
**Post date:** [October 25, 2019, 7:08am UTC](https://discuss.elastic.co/t/filebeat-pushs-syslog-logs-to-elasticsearch-through-logstash/205149/3 "2019-10-25T07:08:46Z")

</div>

After I added Logging part that you suggested into filebeat.yml I cannot even start filebeat. It gives me following message when I try to find out status of filebeat

```
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
   Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2019-10-25 07:06:57 UTC; 5s ago
 Docs: https://www.elastic.co/products/beats/filebeat
  Process: 2313 ExecStart=/usr/share/filebeat/bin/filebeat $BEAT_LOG_OPTS $BEAT_CONFIG_OPTS $BEAT_PATH_OPTS (code=exited, status=1/FAILURE)
 Main PID: 2313 (code=exited, status=1/FAILURE)

Oct 25 07:06:57 bak2 systemd[1]: filebeat.service: Service hold-off time over, scheduling restart.
Oct 25 07:06:57 bak2 systemd[1]: filebeat.service: Scheduled restart job, restart counter is at 5.
Oct 25 07:06:57 bak2 systemd[1]: Stopped Filebeat sends log files to Logstash or directly to Elasticsearch..
Oct 25 07:06:57 bak2 systemd[1]: filebeat.service: Start request repeated too quickly.
Oct 25 07:06:57 bak2 systemd[1]: filebeat.service: Failed with result 'exit-code'.
Oct 25 07:06:57 bak2 systemd[1]: Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch..
```

---

<div class="post-metadata">

**Author:** ![kumarabhi](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kumarabhi](https://discuss.elastic.co/u/kumarabhi)\
**Post date:** [October 25, 2019, 8:53pm UTC](https://discuss.elastic.co/t/filebeat-pushs-syslog-logs-to-elasticsearch-through-logstash/205149/4 "2019-10-25T20:53:29Z")

</div>

YAML files can be very sensitive to space and indentation.  
Please check filebeat\_reference.yml and copy the logging part into filebeat.yml  
Make changes to settings if required and then try starting Filebeat again

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 22, 2019, 8:53pm UTC](https://discuss.elastic.co/t/filebeat-pushs-syslog-logs-to-elasticsearch-through-logstash/205149/5 "2019-11-22T20:53:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.