# New filebeat/ELK stack user: trouble with Raspberry **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 3, 2017, 1:32pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435 "2017-01-03T13:32:15Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![jarif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarif/32/14270_2.png) [@jarif](https://discuss.elastic.co/u/jarif)\ **Post date:** [January 3, 2017, 1:32pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/1 "2017-01-03T13:32:15Z")
Hello all. I have an ELK stack running on a CentOS server. I also have logging satellites running CentOS, and those are sending logs using filebeat. Fine, and works nice. I'm also trying to set up my Raspberrys for filebeat, and while filebeat is running and looks working, nothing appears into Elastic. There is a session: ``` gauntlet:~# /opt/filebeat/filebeat-linux-arm -c /etc/filebeat/filebeat.yml -e -d output 2017/01/03 13:24:12.305137 beat.go:267: INFO Home path: [/opt/filebeat] Config path: [/opt/filebeat] Data path: [/opt/filebeat/data] Logs path: [/opt/filebeat/logs] 2017/01/03 13:24:12.305446 beat.go:177: INFO Setup Beat: filebeat; Version: 6.0.0-alpha1-git1744740 2017/01/03 13:24:12.305505 logp.go:219: INFO Metrics logging every 30s 2017/01/03 13:24:12.306409 output.go:167: INFO Loading template enabled. Reading template file: /opt/filebeat/filebeat.template.json 2017/01/03 13:24:12.334737 output.go:178: INFO Loading template enabled for Elasticsearch 2.x. Reading template file: /opt/filebeat/filebeat.template-es2x.json 2017/01/03 13:24:12.353874 client.go:120: INFO Elasticsearch url: http://elasticsearch-dev:9200 2017/01/03 13:24:12.354158 outputs.go:106: INFO Activated elasticsearch as output plugin. 2017/01/03 13:24:12.355446 publish.go:291: INFO Publisher name: gauntlet 2017/01/03 13:24:12.356439 async.go:63: INFO Flush Interval set to: 1s 2017/01/03 13:24:12.356553 async.go:64: INFO Max Bulk Size set to: 50 2017/01/03 13:24:12.357595 beat.go:207: INFO filebeat start running. 2017/01/03 13:24:12.357981 registrar.go:85: INFO Registry file set to: /opt/filebeat/data/registry 2017/01/03 13:24:12.358282 registrar.go:106: INFO Loading registrar data from /opt/filebeat/data/registry 2017/01/03 13:24:12.361308 registrar.go:131: INFO States Loaded from registrar: 6 2017/01/03 13:24:12.361561 crawler.go:34: INFO Loading Prospectors: 5 2017/01/03 13:24:12.362314 prospector_log.go:57: INFO Prospector with previous states loaded: 1 2017/01/03 13:24:12.363741 prospector_log.go:57: INFO Prospector with previous states loaded: 1 2017/01/03 13:24:12.364536 registrar.go:230: INFO Starting Registrar 2017/01/03 13:24:12.364536 sync.go:41: INFO Start sending events to output 2017/01/03 13:24:12.366173 prospector_log.go:57: INFO Prospector with previous states loaded: 2 2017/01/03 13:24:12.368378 prospector_log.go:57: INFO Prospector with previous states loaded: 1 2017/01/03 13:24:12.368885 spooler.go:63: INFO Starting spooler: spool_size: 2048; idle_timeout: 5s 2017/01/03 13:24:12.370058 prospector_log.go:57: INFO Prospector with previous states loaded: 1 2017/01/03 13:24:12.371198 crawler.go:46: INFO Loading Prospectors completed. Number of prospectors: 5 2017/01/03 13:24:12.371672 crawler.go:61: INFO All prospectors are initialised and running with 6 states to persist 2017/01/03 13:24:12.371706 prospector.go:111: INFO Starting prospector of type: log 2017/01/03 13:24:12.371740 prospector.go:111: INFO Starting prospector of type: log 2017/01/03 13:24:12.371806 prospector.go:111: INFO Starting prospector of type: log 2017/01/03 13:24:12.371804 prospector.go:111: INFO Starting prospector of type: log 2017/01/03 13:24:12.371866 prospector.go:111: INFO Starting prospector of type: log 2017/01/03 13:24:12.379202 log.go:84: INFO Harvester started for file: /var/log/auth.log 2017/01/03 13:24:12.379883 log.go:84: INFO Harvester started for file: /var/log/syslog 2017/01/03 13:24:12.383826 log.go:84: INFO Harvester started for file: /var/log/mail.log 2017/01/03 13:24:17.438123 client.go:652: INFO Connected to Elasticsearch version 5.1.1 2017/01/03 13:24:17.438358 output.go:214: INFO Trying to load template for client: http://elasticsearch-dev:9200 2017/01/03 13:24:17.440716 output.go:235: INFO Template already exists and will not be overwritten. 2017/01/03 13:24:17.612737 single.go:150: DBG send completed 2017/01/03 13:24:17.759963 single.go:150: DBG send completed 2017/01/03 13:24:27.503592 single.go:150: DBG send completed 2017/01/03 13:24:37.521485 single.go:150: DBG send completed 2017/01/03 13:24:42.307568 logp.go:230: INFO Non-zero metrics in the last 30s: libbeat.es.call_count.PublishEvents=4 libbeat.es.publish.read_bytes=2107 registar.states.current=6 filebeat.harvester.running=3 libbeat.es.publish.write_bytes=39767 libbeat.publisher.published_events=85 libbeat.es.published_and_acked_events=85 publish.events=94 filebeat.harvester.open_files=3 registrar.writes=3 registrar.states.update=94 filebeat.harvester.started=3 2017/01/03 13:24:42.498645 single.go:150: DBG send completed 2017/01/03 13:24:47.458720 single.go:150: DBG send completed ``` Here is the Yaml: ``` gauntlet:~# cat /etc/filebeat/filebeat.yml filebeat: prospectors: - paths: - /var/log/auth.log input_type: log document_type: auth scan_frequency: 1s - paths: - /var/log/apache2/access.log input_type: log document_type: apache_access scan_frequency: 1s - paths: - /var/log/apache2/error.log input_type: log document_type: apache_error scan_frequency: 1s - paths: - /var/log/mail.log input_type: log document_type: mail scan_frequency: 1s - paths: - /var/log/syslog input_type: log document_type: syslog scan_frequency: 5s output: elasticsearch: hosts: ["elasticsearch-dev:9200"] index: "filebeat" # logstash: # # The Logstash hosts # hosts: ["elasticsearch-dev:5044"] # index: "filebeat" ``` What might be wrong in my setup for Raspberry? Like I said, the CentOSes work fine. ---
**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\ **Post date:** [January 3, 2017, 2:36pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/2 "2017-01-03T14:36:47Z")
Hi @jarif, log seems good to me. Can you please share the output from the following command `curl http://elasticsearch-dev:9200/filebeat/_stats/docs?pretty=true` ---
**Author:** ![jarif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarif/32/14270_2.png) [@jarif](https://discuss.elastic.co/u/jarif)\ **Post date:** [January 3, 2017, 2:42pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/3 "2017-01-03T14:42:18Z")
``` gauntlet:~# curl http://elasticsearch-dev:9200/filebeat/_stats/docs?pretty=true { "_shards" : { "total" : 10, "successful" : 5, "failed" : 0 }, "_all" : { "primaries" : { "docs" : { "count" : 236329, "deleted" : 0 } }, "total" : { "docs" : { "count" : 236329, "deleted" : 0 } } }, "indices" : { "filebeat" : { "primaries" : { "docs" : { "count" : 236329, "deleted" : 0 } }, "total" : { "docs" : { "count" : 236329, "deleted" : 0 } } } } } ``` ---
**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\ **Post date:** [January 3, 2017, 2:46pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/4 "2017-01-03T14:46:02Z")
Looks like there are entries in the index. How do you figure out that elasticsearch is not getting events from your raspberry? ---
**Author:** ![jarif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarif/32/14270_2.png) [@jarif](https://discuss.elastic.co/u/jarif)\ **Post date:** [January 3, 2017, 2:47pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/5 "2017-01-03T14:47:02Z")
I do not see anything in Kibana screen but constant flow of messages from my CentOSes. Nothing from "gauntlet"... ---
**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\ **Post date:** [January 3, 2017, 2:50pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/6 "2017-01-03T14:50:31Z")
I assume you use different indices or do you write into the same index? Maybe you have some filter in kibana for your CentOS machines? ---
**Author:** ![jarif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarif/32/14270_2.png) [@jarif](https://discuss.elastic.co/u/jarif)\ **Post date:** [January 3, 2017, 2:53pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/7 "2017-01-03T14:53:35Z")
Not thatI am aware of. I just created ELK stack using [https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7) and then installed my raspi beat using [http://ict.renevdmark.nl/2016/07/05/elastic-beats-on-raspberry-pi/](http://ict.renevdmark.nl/2016/07/05/elastic-beats-on-raspberry-pi/) I have not done anything in Kibana besides that. Only index that I am aware of is "filebeat" ---
**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\ **Post date:** [January 3, 2017, 2:57pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/8 "2017-01-03T14:57:19Z")
Can you post some json example from kibana. ---
**Author:** ![jarif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarif/32/14270_2.png) [@jarif](https://discuss.elastic.co/u/jarif)\ **Post date:** [January 3, 2017, 2:59pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/9 "2017-01-03T14:59:16Z")
Means that I have to learn to use the dev tool now;) All right. ---
**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\ **Post date:** [January 3, 2017, 3:24pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/10 "2017-01-03T15:24:54Z")
If you open a message in kibana you have a json tab which shows the raw data. ---
**Author:** ![jarif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarif/32/14270_2.png) [@jarif](https://discuss.elastic.co/u/jarif)\ **Post date:** [January 3, 2017, 3:36pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/11 "2017-01-03T15:36:08Z")
Ahm, yes. ``` { "_index": "filebeat-2017.01.03", "_type": "syslog", "_id": "AVlk-Gn3wm5zYvOseB0f", "_score": null, "_source": { "@timestamp": "2017-01-03T15:35:15.129Z", "beat": { "hostname": "sunderland.fredriksson.dy.fi", "name": "sunderland.fredriksson.dy.fi" }, "count": 1, "fields": null, "input_type": "log", "message": "Jan 3 17:35:09 sunderland guardrail_agent: [03 Jan 2017 05:35:09pm EET][DEBUG] []: Polling secure for tasks", "offset": 1670136, "source": "/var/log/messages", "type": "syslog" }, "fields": { "@timestamp": [ 1483457715129 ] }, "sort": [ 1483457715129 ] } ``` ---
**Author:** ![jarif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarif/32/14270_2.png) [@jarif](https://discuss.elastic.co/u/jarif)\ **Post date:** [January 3, 2017, 3:43pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/12 "2017-01-03T15:43:33Z")
Hmm this works in dev tool! ``` GET /_search { "query": { "query_string" : { "default_field" : "message", "query" : "gauntlet" } } } ``` It returns: ``` { "took": 9, "timed_out": false, "_shards": { "total": 16, "successful": 16, "failed": 0 }, "hits": { "total": 121856, "max_score": 0.82289654, "hits": [ { "_index": "filebeat", "_type": "mail", "_id": "AVlkUfdNwm5zYvOsdYWp", "_score": 0.82289654, "_source": { "@timestamp": "2017-01-03T12:33:25.722Z", "beat": { "hostname": "gauntlet", "name": "gauntlet", "version": "6.0.0-alpha1-git1744740" }, "input_type": "log", "message": "Jan 3 14:33:22 gauntlet dovecot: imap(jarif): Warning: Fixed a duplicate: /home/jarif/Maildir/.Confirmed-SPAM/cur/1483321241.M110582P25566.gauntlet,S=5633,W=5752:2,Sd -> 1483446802.M72959P14737.gauntlet,S=5633,W=5752", "offset": 8573164, "source": "/var/log/mail.log", "type": "mail" } }, ``` ---
**Author:** ![jarif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarif/32/14270_2.png) [@jarif](https://discuss.elastic.co/u/jarif)\ **Post date:** [January 3, 2017, 3:47pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/13 "2017-01-03T15:47:34Z")
I can't make a working query in Kibana console or whatever that UI is. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [January 3, 2017, 4:01pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/14 "2017-01-03T16:01:21Z")
You configured the index to be `filebeat`. You probably wanted a `filebeat-%{+yyyy.MM.dd}` to get daily indexes like `filebeat-2016.01.03`. ---
**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\ **Post date:** [January 3, 2017, 4:18pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/15 "2017-01-03T16:18:49Z")
Just remove the option "index: filebeat" ---
**Author:** ![jarif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarif/32/14270_2.png) [@jarif](https://discuss.elastic.co/u/jarif)\ **Post date:** [January 3, 2017, 4:51pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/16 "2017-01-03T16:51:00Z")
Thanks! It works now. Somehow the CentOS configs do have that index: filebeat and they still work. But I'm happy now. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [January 3, 2017, 4:53pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/17 "2017-01-03T16:53:14Z")
Probably you are using different Filebeat versions. Older versions did not support a configurable date so `index: filebeat` implicitly meant `filebeat-yyyy.MM.dd`. ---
**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\ **Post date:** [January 3, 2017, 5:01pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/18 "2017-01-03T17:01:35Z")
👍 good to know 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 31, 2017, 5:01pm UTC](https://discuss.elastic.co/t/new-filebeat-elk-stack-user-trouble-with-raspberry/70435/19 "2017-01-31T17:01:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.