# ELK Stack Setup to Receive Beats from Magento **URL:** \ **Category:** Logstash\ **Created:** [August 7, 2018, 6:20pm UTC](https://discuss.elastic.co/t/elk-stack-setup-to-receive-beats-from-magento/143380 "2018-08-07T18:20:54Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![imranvirk](https://avatars.discourse-cdn.com/v4/letter/i/0ea827/32.png) [@imranvirk](https://discuss.elastic.co/u/imranvirk)\ **Post date:** [August 7, 2018, 6:20pm UTC](https://discuss.elastic.co/t/elk-stack-setup-to-receive-beats-from-magento/143380/1 "2018-08-07T18:20:54Z")
I have 2 droplets configured in my DO project: 1. ELK Stack 2. Magento eCommerce platform on a LAMP Stack Both instances are up and running, with all services setup. However, I am having an issue with my Magento server, which is configured as the Filebeat client, sending its system logs over to the ELK stack. I've followed the instructions from this KB article: [https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-14-04) I'm stuck at the "Test Filebeat Installation" section and no matter what I'm trying with the Filebeat configuration the logs do not seem to be getting sent to the ELK stack. I've gone back through the configurations a few time to ensure I have it setup properly, but it's possible I'm overlooking something simple. When I search for filebeat logs on the ELK stack I get 0 hits (see below extract from my terminal). Can anyone provide any guidance/suggestions on where to troubleshoot? ``` curl -XGET 'http://localhost:9200/filebeat-*/_search?pretty' { "took" : 1, "timed_out" : false, "_shards" : { "total" : 0, "successful" : 0, "failed" : 0 }, "hits" : { "total" : 0, "max_score" : 0.0, "hits" : [] } } ``` Here's the various configs I have setup. **filebeat.yml** ``` filebeat: prospectors: paths: - /var/log/*.log #- c:\programdata\elasticsearch\logs\* # - /var/www/html/magento/var/log/syslog # - /var/www/html/magento/var/log/auth.log - /var/www/html/magento/var/log/connector.log - /var/www/html/magento/var/log/debug.log - /var/www/html/magento/var/log/exception.log - /var/www/html/magento/var/log/install.log - /var/www/html/magento/var/log/support_report.log - /var/www/html/magento/var/log/system.log input_type: log document_type: syslog registry_file: /var/lib/filebeat/registry output: ### Logstash as output logstash: # The Logstash hosts hosts: ["209.97.151.135:5044"] bulk_max_size: 1024 # Optional TLS. By default is off. tls: # List of root certificates for HTTPS server verifications certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"] shipper: logging: files: rotateeverybytes: 10485760 # = 10MB ``` **30-elasticsearch-output.conf** ``` output { elasticsearch { hosts => ["localhost:9200"] sniffing => true manage_template => false index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } } ``` **10-syslog-filter.conf** ``` filter { if [type] == "syslog" { grok { match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" } add_field => ["received_at", "%{@timestamp}"] add_field => ["received_from", "%{host}"] } syslog_pri { } date { match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } } ``` **02-beats-input.conf** ``` input { beats { port => 5044 ssl => true ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt" ssl_key => "/etc/pki/tls/private/logstash-forwarder.key" } } ``` Please let me know if I can provide any additional information or details. Thanks so much, ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 7, 2018, 6:31pm UTC](https://discuss.elastic.co/t/elk-stack-setup-to-receive-beats-from-magento/143380/2 "2018-08-07T18:31:42Z")
First you need to establish whether Filebeat is reading any logs and attempting to send them to Logstash. Do so by looking in Filebeat's own log. If you don't see anything useful try increasing the loglevel. ---
**Author:** ![imranvirk](https://avatars.discourse-cdn.com/v4/letter/i/0ea827/32.png) [@imranvirk](https://discuss.elastic.co/u/imranvirk)\ **Post date:** [August 7, 2018, 6:49pm UTC](https://discuss.elastic.co/t/elk-stack-setup-to-receive-beats-from-magento/143380/3 "2018-08-07T18:49:18Z")
Thanks for the feedback, Magnusbaeck. Can you provide some guidance on how to check for that? I'm very new to this so not sure where/how to look for those logs exactly. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 7, 2018, 6:55pm UTC](https://discuss.elastic.co/t/elk-stack-setup-to-receive-beats-from-magento/143380/4 "2018-08-07T18:55:40Z")
If you run Filebeat from a terminal I'd expect the log entries to show up in your terminal. Otherwise the logging section of the Filebeat configuration file controls how logging is performed. See [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-logging.html](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-logging.html) for available configuration options and an example. ---
**Author:** ![imranvirk](https://avatars.discourse-cdn.com/v4/letter/i/0ea827/32.png) [@imranvirk](https://discuss.elastic.co/u/imranvirk)\ **Post date:** [August 7, 2018, 7:04pm UTC](https://discuss.elastic.co/t/elk-stack-setup-to-receive-beats-from-magento/143380/5 "2018-08-07T19:04:31Z")
For some reason I don't have a filebeat log directory in `/var/log/` which could certainly be part of the problem. I'll look at the examples and try to get this setup I also noticed that the instructions from the Digital Ocean article specified 2.2 for the Logstash version, which from the looks of it is a much older version. Could this be a source of the issue (at least partly)? ---
**Author:** ![imranvirk](https://avatars.discourse-cdn.com/v4/letter/i/0ea827/32.png) [@imranvirk](https://discuss.elastic.co/u/imranvirk)\ **Post date:** [August 7, 2018, 7:20pm UTC](https://discuss.elastic.co/t/elk-stack-setup-to-receive-beats-from-magento/143380/6 "2018-08-07T19:20:14Z")
Another strange issue - when I try to run filebeat manually from the terminal using `sudo filebeat run` the command fails with a `"Loading config file error"`. But if run that command from /etc/filebeat/ (which is where the filebeat.yml file is, the command seems like it's doing something but after 15 mins of that command being run nothing has happened. It almost seems like that filebeat is not properly installed on this client server. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 8, 2018, 5:52am UTC](https://discuss.elastic.co/t/elk-stack-setup-to-receive-beats-from-magento/143380/7 "2018-08-08T05:52:03Z")
> For some reason I don't have a filebeat log directory in /var/log/ which could certainly be part of the problem. I'll look at the examples and try to get this setup Well, you can store the logs anywhere you want. > I also noticed that the instructions from the Digital Ocean article specified 2.2 for the Logstash version, which from the looks of it is a much older version. Could this be a source of the issue (at least partly)? Unlikely. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 5, 2018, 5:56am UTC](https://discuss.elastic.co/t/elk-stack-setup-to-receive-beats-from-magento/143380/8 "2018-09-05T05:56:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.