# Filebeat System module not sending process name when using Logstash as a output

**URL:** <https://discuss.elastic.co/t/filebeat-system-module-not-sending-process-name-when-using-logstash-as-a-output/262844>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 1, 2021, 1:59pm UTC](https://discuss.elastic.co/t/filebeat-system-module-not-sending-process-name-when-using-logstash-as-a-output/262844 "2021-02-01T13:59:03Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\
**Post date:** [February 1, 2021, 1:59pm UTC](https://discuss.elastic.co/t/filebeat-system-module-not-sending-process-name-when-using-logstash-as-a-output/262844/1 "2021-02-01T13:59:03Z")

</div>

I am using filebeat system module to send syslog data to Elasticsearch and it is working fine, even the default dashboard is also working fine. However, while I use Logstash as the output, it is not sending few fields such as process name and therefore it's not mapping with dashboard visualizations.

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [February 1, 2021, 7:30pm UTC](https://discuss.elastic.co/t/filebeat-system-module-not-sending-process-name-when-using-logstash-as-a-output/262844/2 "2021-02-01T19:30:13Z")

</div>

What versions of Filebeat, Logstash, and Elasticsearch are you running?

Also, would you mind re-running Filebeat with [`logging.level: debug`](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-logging.html#level)? This will let us see JSON representations of events that Filebeat is sending to Logstash.

Thanks,

Shaunak

---

<div class="post-metadata">

**Author:** ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\
**Post date:** [February 2, 2021, 7:43am UTC](https://discuss.elastic.co/t/filebeat-system-module-not-sending-process-name-when-using-logstash-as-a-output/262844/3 "2021-02-02T07:43:48Z")

</div>

Filebeat: 6.8  
Logstash: 7.9.2  
Elasticsearch: 7.9.2

Here is my filebeat.yml

```
###################### Filebeat Configuration Example #########################

# This file is an example configuration file highlighting only the most common
# options. The filebeat.reference.yml file from the same directory contains all the
# supported options with more comments. You can use it as a reference.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/filebeat/index.html

# For more available modules and options, please see the filebeat.reference.yml sample
# configuration file.
#display_name
# ============================== Filebeat inputs ===============================

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/syslog
    #- c:\programdata\elasticsearch\logs\*

  # Exclude lines. A list of regular expressions to match. It drops the lines that are
  # matching any regular expression from the list.
  #exclude_lines: ['^DBG']

  # Include lines. A list of regular expressions to match. It exports the lines that are
  # matching any regular expression from the list.
  #include_lines: ['^ERR', '^WARN']

  # Exclude files. A list of regular expressions to match. Filebeat drops the files that
  # are matching any regular expression from the list. By default, no files are dropped.
  #exclude_files: ['.gz$']

  # Optional additional fields. These fields can be freely picked
  # to add additional information to the crawled log files for filtering
  #fields:
  # level: debug
  # review: 1

  ### Multiline options

  # Multiline can be used for log messages spanning multiple lines. This is common
  # for Java Stack Traces or C-Line Continuation

  # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
  #multiline.pattern: ^\[

  # Defines if the pattern set under pattern should be negated or not. Default is false.
  #multiline.negate: false

  # Match can be set to "after" or "before". It is used to define if lines should be append to a pattern
  # that was (not) matched before or after or as long as a pattern is not matched based on negate.
  # Note: After is the equivalent to previous and before is the equivalent to to next in Logstash
  #multiline.match: after

# ============================== Filebeat modules ==============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

  # Period on which files under path should be checked for changes
  #reload.period: 10s

# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

# ================================== General ===================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:

# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the
# output.
#fields:
# env: staging

# ================================= Dashboards =================================
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards is disabled by default and can be enabled either by setting the
# options here or by using the `setup` command.
#setup.dashboards.enabled: false

# The URL from where to download the dashboards archive. By default this URL
# has a value which is computed based on the Beat name and version. For released
# versions, this URL points to the dashboard archive on the artifacts.elastic.co
# website.
#setup.dashboards.url:

# =================================== Kibana ===================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  host: "https://kibana.app:5601"
  username: "uname"
  password: "password"
  # Kibana Space ID
  # ID of the Kibana Space into which the dashboards should be loaded. By default,
  # the Default Space will be used.
  #space.id:

# =============================== Elastic Cloud ================================

# These settings simplify using Filebeat with the Elastic Cloud (https://cloud.elastic.co/).

# The cloud.id setting overwrites the `output.elasticsearch.hosts` and
# `setup.kibana.host` options.
# You can find the `cloud.id` in the Elastic Cloud web UI.
#cloud.id:

# The cloud.auth setting overwrites the `output.elasticsearch.username` and
# `output.elasticsearch.password` settings. The format is `<user>:<pass>`.
#cloud.auth:

# ================================== Outputs ===================================

# Configure what output to use when sending the data collected by the beat.

# ---------------------------- Elasticsearch Output ----------------------------
#output.elasticsearch:
  # Array of hosts to connect to.
  #hosts: ["localhost:9200"]

  # Protocol - either `http` (default) or `https`.
  #protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  #username: "elastic"
  #password: "changeme"

output.elasticsearch:
  # Array of hosts to connect to.
   hosts: ["ip:9200"]

  # Protocol - either `http` (default) or `https`.
   protocol: "https"
   ssl.verification_mode: none
  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
   username: "uname"
   password: "password"

# ------------------------------ Logstash Output -------------------------------
#output.logstash:
  # The Logstash hosts
  #hosts: ["ip:5044"]

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

# ================================= Processors =================================

# Configure processors to enhance or manipulate events generated by the beat.

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

# ================================== Logging ===================================

# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
logging.level: debug

# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]

# ============================= X-Pack Monitoring ==============================
# Filebeat can export internal metrics to a central Elasticsearch monitoring
# cluster. This requires xpack monitoring to be enabled in Elasticsearch. The
# reporting is disabled by default.

# Set to true to enable the monitoring reporter.
#monitoring.enabled: false

# Sets the UUID of the Elasticsearch cluster under which monitoring data for this
# Filebeat instance will appear in the Stack Monitoring UI. If output.elasticsearch
# is enabled, the UUID is derived from the Elasticsearch cluster referenced by output.elasticsearch.
#monitoring.cluster_uuid:

# Uncomment to send the metrics to Elasticsearch. Most settings from the
# Elasticsearch output are accepted here as well.
# Note that the settings should point to your Elasticsearch *monitoring* cluster.
# Any setting that is not set is automatically inherited from the Elasticsearch
# output configuration, so if you have the Elasticsearch output configured such
# that it is pointing to your Elasticsearch monitoring cluster, you can simply
# uncomment the following line.
#monitoring.elasticsearch:

# ================================= Migration ==================================

# This allows to enable 6.7 migration aliases
#migration.6_to_7.enabled: true
```

---

<div class="post-metadata">

**Author:** ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\
**Post date:** [February 2, 2021, 7:45am UTC](https://discuss.elastic.co/t/filebeat-system-module-not-sending-process-name-when-using-logstash-as-a-output/262844/4 "2021-02-02T07:45:15Z")

</div>

here is my system.yml

```
  # Module: system
            # Docs: https://www.elastic.co/guide/en/beats/filebeat/7.8/filebeat-module-system.html

            #- module: system
              # Syslog
             # syslog:
               # enabled: true

                # Set custom paths for the log files. If left empty,
                # Filebeat will choose the paths depending on your OS.
               # var.paths: ["/var/log/syslog"]
               # var.convert_timezone: true
              # Authorization logs
              #auth:
                #enabled: true

                # Set custom paths for the log files. If left empty,
                # Filebeat will choose the paths depending on your OS.
                #var.paths: ["/var/log/auth"]
                #var.convert_timezone: true
```

---

<div class="post-metadata">

**Author:** ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\
**Post date:** [February 2, 2021, 7:56am UTC](https://discuss.elastic.co/t/filebeat-system-module-not-sending-process-name-when-using-logstash-as-a-output/262844/5 "2021-02-02T07:56:47Z")

</div>

here is my filebeat logfile

```
2021-02-01T12:47:54.003Z INFO instance/beat.go:647 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2021-02-01T12:47:54.003Z INFO instance/beat.go:655 Beat ID: accaa3df-79c3-48b4-9660-58e14cbe29e1
2021-02-01T12:47:54.005Z INFO [beat] instance/beat.go:983 Beat info {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", $
2021-02-01T12:47:54.005Z INFO [beat] instance/beat.go:992 Build info {"system_info": {"build": {"commit": "f79387d32717d79f689d94fda1ec80b2cf285d30", "libbeat": "7.8.0", "time": "2020-06-14T1$
2021-02-01T12:47:54.005Z INFO [beat] instance/beat.go:995 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.13.10"}}}
2021-02-01T12:47:54.005Z INFO [beat] instance/beat.go:999 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-02-01T12:03:19Z","containerized":false,"name":"ip-172-$
2021-02-01T12:47:54.005Z INFO [beat] instance/beat.go:1028 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fo$
2021-02-01T12:47:54.005Z INFO instance/beat.go:310 Setup Beat: filebeat; Version: 7.8.0
2021-02-01T12:47:54.006Z INFO [index-management] idxmgmt/std.go:183 Set output.elasticsearch.index to 'filebeat-7.8.0' as ILM is enabled.
2021-02-01T12:47:54.006Z INFO eslegclient/connection.go:97 elasticsearch url: https://localhost:9200
2021-02-01T12:47:54.006Z WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
2021-02-01T12:47:54.006Z INFO [publisher] pipeline/module.go:113 Beat name: ip-172-21-6-100
2021-02-01T12:47:54.007Z INFO eslegclient/connection.go:97 elasticsearch url: https://localhost:9200
2021-02-01T12:47:54.007Z WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
2021-02-01T12:47:54.007Z WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
2021-02-01T12:47:54.012Z INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:93 add_cloud_metadata: hosting provider type detected as aws, metadata={"account":{"id":"6551364$
2021-02-01T12:47:54.113Z INFO [esclientleg] eslegclient/connection.go:306 Attempting to connect to Elasticsearch version 7.9.1
2021-02-01T12:47:54.132Z INFO [index-management] idxmgmt/std.go:259 Auto ILM enable success.
2021-02-01T12:47:54.133Z INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false
2021-02-01T12:47:54.133Z INFO [index-management] idxmgmt/std.go:272 ILM policy successfully loaded.
2021-02-01T12:47:54.133Z INFO [index-management] idxmgmt/std.go:405 Set setup.template.name to '{filebeat-7.8.0 {now/d}-000001}' as ILM is enabled.
2021-02-01T12:47:54.133Z INFO [index-management] idxmgmt/std.go:410 Set setup.template.pattern to 'filebeat-7.8.0-*' as ILM is enabled.
2021-02-01T12:47:54.133Z INFO [index-management] idxmgmt/std.go:444 Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.8.0 {now/d}-000001} as ILM is enabled.
2021-02-01T12:47:54.133Z INFO [index-management] idxmgmt/std.go:448 Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"3$
2021-02-01T12:47:54.143Z INFO template/load.go:169 Existing template will be overwritten, as overwrite is enabled.
2021-02-01T12:47:54.411Z INFO template/load.go:109 Try loading template filebeat-7.8.0 to Elasticsearch
2021-02-01T12:47:54.565Z INFO template/load.go:101 template with name 'filebeat-7.8.0' loaded.
2021-02-01T12:47:54.565Z INFO [index-management] idxmgmt/std.go:296 Loaded index template.
2021-02-01T12:47:54.786Z INFO [index-management] idxmgmt/std.go:307 Write alias successfully generated.
2021-02-01T12:47:54.786Z INFO kibana/client.go:118 Kibana url: https://localhost:5061
2021-02-01T12:49:24.786Z ERROR instance/beat.go:958 Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to https://localhost:5061/api/status fails: fail to exe$
```

---

<div class="post-metadata">

**Author:** ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\
**Post date:** [February 2, 2021, 8:13am UTC](https://discuss.elastic.co/t/filebeat-system-module-not-sending-process-name-when-using-logstash-as-a-output/262844/6 "2021-02-02T08:13:40Z")

</div>

While `syslog` is being sent directly to Elasticsearch it sends `process.name` field, but when I switch filebeat output to Logstash, it stops sending `process.name`. I identify this one field but there may be many fields like such which are not coming when logs are coming via Logstash.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 2, 2021, 10:13am UTC](https://discuss.elastic.co/t/filebeat-system-module-not-sending-process-name-when-using-logstash-as-a-output/262844/7 "2021-03-02T10:13:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.