# Data is not available for syslog dashboard

**URL:** <https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 3, 2018, 7:38am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026 "2018-09-03T07:38:24Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 3, 2018, 7:38am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/1 "2018-09-03T07:38:24Z")

</div>

Hello All,

I am using ELK6.4.0 and beats 6.4.0. I have enabled the filebeat system module and getting the data over dashboard for syslog and auth.log. But when i am checking the filebeat dashboard for syslog no data is available there. Please refer the below screenshot for logs coming via system module:

 ![Selection_020](https://us1.discourse-cdn.com/elastic/original/3X/d/2/d2ab96a5d2eef5fb21f05276677445158b3f2c98.png)

Screenshot for syslog dashboard:

 ![Selection_021](https://us1.discourse-cdn.com/elastic/original/3X/9/3/93ca3f38f8443aa23bbcc6a67aad7a9faa9d09c9.png)

Can anybody please assist me to troubleshoot the issue?

It little bit urgent and i am troubleshooting this issue from last 2 days.

Any lead will be appreciated.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 3, 2018, 8:29am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/2 "2018-09-03T08:29:19Z")

</div>

Are you sure you selected the correct time interval?

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 3, 2018, 8:56am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/3 "2018-09-03T08:56:02Z")

</div>

Hello Noemi,

I have checked again for the last 4 hour but still no data for syslog dashboard.

Please assist.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 3, 2018, 9:17am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/4 "2018-09-03T09:17:26Z")

</div>

Could you please share you full config formatted using `</>`?

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 3, 2018, 9:32am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/5 "2018-09-03T09:32:21Z")

</div>

Hello Noemi,

Please find all the config files:

**1) filebeat.yml**  
###################### Filebeat Configuration Example #########################

#This file is an example configuration file highlighting only the most commo

#=========================== Filebeat prospectors =============================

filebeat.prospectors:

#Each - is a prospector. Most options can be set at the prospector level, so  
#you can use different prospectors for various configurations.  
#Below are the prospector specific configurations.

- type: log

#============================= Filebeat modules ===============================

filebeat.config.modules:  
#Glob pattern for configuration loading  
path: ${path.config}/modules.d/\*.yml

#Set to true to enable config reloading  
reload.enabled: true

#Period on which files under path should be checked for changes  
#reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:  
index.number\_of\_shards: 3  
#index.codec: best\_compression  
#\_source.enabled: false

#================================ General =====================================

#The name of the shipper that publishes the network data. It can be used to group  
#all the transactions sent by a single shipper in the web interface.  
#name:

#The tags of the shipper are included in their own field with each  
#transaction published.  
#tags: ["service-X", "web-tier"]

#Optional fields that you can specify to add additional information to the  
#output.  
#fields:  
#env: staging

#============================== Dashboards =====================================  
#These settings control loading the sample dashboards to the Kibana index. Loading  
#the dashboards is disabled by default and can be enabled either by setting the  
#options here, or by using the `-setup` CLI flag or the `setup` command.  
setup.dashboards.enabled: false

#The URL from where to download the dashboards archive. By default this URL  
#has a value which is computed based on the Beat name and version. For released  
#versions, this URL points to the dashboard archive on the [artifacts.elastic.co](http://artifacts.elastic.co)  
#website.  
#setup.dashboards.url:

#============================== Kibana =====================================

#Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.  
#This requires a Kibana endpoint configuration.  
setup.kibana:  
host: "xyz:5601"

#----------------------------- Logstash output --------------------------------  
output.logstash:  
#The Logstash hosts  
hosts: ["xyz:5044"]  
bulk\_max\_size: 1024

#Optional SSL. By default is off.  
#List of root certificates for HTTPS server verifications  
#ssl.certificate\_authorities: ["/etc/pki/root/ca.pem"]  
ssl.certificate\_authorities: ["/etc/pki/tls/ca.crt"]  
ssl.certificate: "/etc/pki/tls/client.crt"  
ssl.key: "/etc/pki/tls/client.key"  
ssl.key\_passphrase: "ky9D=h=w2z2uUCjRqqWF"

**2. Logstash input,filter and output file:**

```
input {

```

beats {  
port =\> 5044  
ssl =\> true  
ssl\_certificate\_authorities =\> ["/etc/pki/tls/ca.crt"]  
ssl\_certificate =\> "/etc/pki/tls/server.crt"  
ssl\_key =\> "/etc/pki/tls/server.key"  
ssl\_verify\_mode =\> "peer"  
tls\_min\_version =\> "1.2"  
}  
}  
filter {  
grok {  
match =\> { "message" =\> ["%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}", "\I,\s[(?[\d-\w:.]+)\s#(?\d+)]\s+(?\w+)\s-+\s:\s[(?[\d\w-]+)]\s(?[\w\s]+)\s"(?[\w/.]+)"\s(?._)", "\I,\s[(?[\d-\w:.]+)\s#(?[\d]+)]\s\s(?[\w]+)\s--\s:\s[(?[\d-\w]+)]\s(?:[cC]urrent\s)?[dD]evice[\s:]+(?[\w\s:]+)", "\I,\s[(?[\d-\w:.]+)\s#(?\d+)]\s+(?\w+)\s-+\s:\s[(?[\d\w-]+)]\s(?._)", "\w,\s[(?[\w-:.]+)\s#(?\d+)]\s+(?\w+)\s(?.\*)" ] }  
add\_field =\> ["received\_at", "%{@timestamp}"] add\_field =\> ["received\_from", "%{host}"]  
}  
}  
output {  
elasticsearch {  
hosts =\> ["xyz:9200"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

**3. List of enabled and disabled module at filebeat:-**

```
root@xyz:~# filebeat modules list

```

Enabled:  
system

Disabled:  
apache2  
auditd  
elasticsearch  
icinga  
iis  
kafka  
kibana  
logstash  
mongodb  
mysql  
nginx  
osquery  
postgresql  
redis  
traefik

Please find the requested config files.  
Note:- I have used xyz instead of IP, DNS name for security point of view.This text will be hidden

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 3, 2018, 9:37am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/6 "2018-09-03T09:37:12Z")

</div>

Could you please format the config?

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 3, 2018, 9:48am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/7 "2018-09-03T09:48:42Z")

</div>

Hello Noemi,

I have formatted the config files. Please look earlier post.

Thanks.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 3, 2018, 10:55am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/8 "2018-09-03T10:55:35Z")

</div>

Hello All,

Can you please update me? Due to this issue my project is stuck.

Thanks.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 3, 2018, 11:14am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/9 "2018-09-03T11:14:11Z")

</div>

Your configuration is still not formatted correctly.

However, I think I see your problem. You enabled `log` prospector along with system module. You either enable system module or enable `log` input. If you enable both, `log` prospector is going to read the logs and you cannot use dashboards.

I suggest you use the following config instead of enabling prospectors:

```auto
filebeat.modules:

#------------------------------- System Module -------------------------------
#- module: system
  # Syslog
  #syslog:
    #enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
    #var.convert_timezone: false

    # Input configuration (advanced). Any input configuration option
    # can be added under this section.
    #input:

  # Authorization logs
  #auth:
    #enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
    #var.convert_timezone: false

    # Input configuration (advanced). Any input configuration option
    # can be added under this section.
    #input:

```

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 3, 2018, 11:38am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/10 "2018-09-03T11:38:51Z")

</div>

Hello Noemi,

In log prospector the auth.log and syslog both are commented. So that part is not working for auth.log and syslog.

I have few question if i will stop log prosepector which i am using for my application log then how i will handle my multi line logs?

Please find the complete filebeat.yml file:

```auto
###################### Filebeat Configuration Example #########################

# This file is an example configuration file highlighting only the most common
# options. The filebeat.reference.yml file from the same directory contains all the
# supported options with more comments. You can use it as a reference.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/filebeat/index.html

# For more available modules and options, please see the filebeat.reference.yml sample
# configuration file.

#=========================== Filebeat prospectors =============================

filebeat.prospectors:

# Each - is a prospector. Most options can be set at the prospector level, so
# you can use different prospectors for various configurations.
# Below are the prospector specific configurations.

- type: log

  # Change to true to enable this prospector configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
# - /var/log/syslog
# - /var/log/auth.log
    - /var/apps/mobilock-emm-staging/shared/log/staging.log
  exclude_lines: ['^.*nrpe.*$']
  document_type: syslog
    #- c:\programdata\elasticsearch\logs\*

  # Exclude lines. A list of regular expressions to match. It drops the lines that are
  # matching any regular expression from the list.
  #exclude_lines: ['^DBG']

  # Include lines. A list of regular expressions to match. It exports the lines that are
  # matching any regular expression from the list.
  #include_lines: ['^ERR', '^WARN']

  # Exclude files. A list of regular expressions to match. Filebeat drops the files that
  # are matching any regular expression from the list. By default, no files are dropped.
  #exclude_files: ['.gz$']

  # Optional additional fields. These fields can be freely picked
  # to add additional information to the crawled log files for filtering
  #fields:
  # level: debug
  # review: 1

  ### Multiline options

  # Mutiline can be used for log messages spanning multiple lines. This is common
  # for Java Stack Traces or C-Line Continuation

  # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
  #multiline.pattern: ^\[

  # Defines if the pattern set under pattern should be negated or not. Default is false.
  #multiline.negate: false

  # Match can be set to "after" or "before". It is used to define if lines should be append to a pattern
  # that was (not) matched before or after or as long as a pattern is not matched based on negate.
  # Note: After is the equivalent to previous and before is the equivalent to to next in Logstash
  #multiline.match: after
  multiline.pattern: '[A-Z]{1}, \[[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after

#============================= Filebeat modules ===============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: true

  # Period on which files under path should be checked for changes
  #reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 3
  #index.codec: best_compression
  #_source.enabled: false

#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:

# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the
# output.
#fields:
# env: staging

#============================== Dashboards =====================================
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards is disabled by default and can be enabled either by setting the
# options here, or by using the `-setup` CLI flag or the `setup` command.
setup.dashboards.enabled: false

# The URL from where to download the dashboards archive. By default this URL
# has a value which is computed based on the Beat name and version. For released
# versions, this URL points to the dashboard archive on the artifacts.elastic.co
# website.
#setup.dashboards.url:

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  host: "xyz:5601"
#============================= Elastic Cloud ==================================

# These settings simplify using filebeat with the Elastic Cloud (https://cloud.elastic.co/).

# The cloud.id setting overwrites the `output.elasticsearch.hosts` and
# `setup.kibana.host` options.
# You can find the `cloud.id` in the Elastic Cloud web UI.
#cloud.id:

# The cloud.auth setting overwrites the `output.elasticsearch.username` and
# `output.elasticsearch.password` settings. The format is `<user>:<pass>`.
#cloud.auth:

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------
#output.elasticsearch:
  # Array of hosts to connect to.
# hosts: ["xyz:9200"]

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["xyz:5044"]
  bulk_max_size: 1024

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
  ssl.certificate_authorities: ["/etc/pki/tls/ca.crt"]
  ssl.certificate: "/etc/pki/tls/client.crt"
  ssl.key: "/etc/pki/tls/client.key"
  ssl.key_passphrase: "ky9D=h=w2z2uUCjRqqWF"

```

If you can tell me the place where i need to comment my config file that will be helpful for me.

Thanks.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 4, 2018, 3:33am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/11 "2018-09-04T03:33:22Z")

</div>

Hello Team,

Can you please help me on above issue. I have question if i disable the log prospector then how i can send my application logs to logstash? is it possible via filebeat system module?

If yes then how i will manage my multiline logs?

Please help me to fix the issue.

Thanks.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 4, 2018, 10:42am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/12 "2018-09-04T10:42:26Z")

</div>

You can put the multiline config under the `input` section of the module configuration. (Please set the multiline config according to your case. I just put dummy values there.)

```auto
filebeat.modules:

#------------------------------- System Module -------------------------------
- module: system
  # Syslog
  syslog:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
    #var.convert_timezone: false

    # Input configuration (advanced). Any input configuration option
    # can be added under this section.
    input:
      multiline.pattern: '\['
      multiline.negate: false
      multiline.match: after

```

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 4, 2018, 10:46am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/13 "2018-09-04T10:46:34Z")

</div>

Hello Noemi,

Thank you for your response.

Its very helpful for me.

I have final question on this thread where i can add my log file path which is our application log?  
Except syslog and auth.log.

Thanks.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 4, 2018, 10:48am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/14 "2018-09-04T10:48:05Z")

</div>

Is it an arbitrary app log which is not going to be read through Filebeat modules?  
If yes, you just need to configure a `prospector` as you did previously.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 4, 2018, 10:53am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/15 "2018-09-04T10:53:36Z")

</div>

Hello Noemi,

Yes..its an arbitrary app.

It means we can't use the filebeat syslog dashboard feature of system module?

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 4, 2018, 3:36pm UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/16 "2018-09-04T15:36:36Z")

</div>

In case of an arbitrary prospector no. But if you send the logs using the system module, you can use it. But if that app can produce syslog formatted logs, you could add it to the modules config by setting `var.paths` of the module.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 5, 2018, 3:51am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/17 "2018-09-05T03:51:58Z")

</div>

Hello Noemi,

Thanks for valuable information.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 3, 2018, 3:52am UTC](https://discuss.elastic.co/t/data-is-not-available-for-syslog-dashboard/147026/18 "2018-10-03T03:52:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.