# Problems getting started with Filebeat and Elasticsearch?

**URL:** <https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974>\
**Category:** Elasticsearch\
**Created:** [April 27, 2020, 2:28pm UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974 "2020-04-27T14:28:12Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Forcep](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forcep/32/67198_2.png) [@Forcep](https://discuss.elastic.co/u/Forcep)\
**Post date:** [April 27, 2020, 2:28pm UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/1 "2020-04-27T14:28:12Z")

</div>

Hi,

Im running the following on a local computer for test :

- Elastcisserach 7.5.1
- Kibana 7.5.1
- Filebeat 7.5.1

Its using HTTPS connection with ignor cert errors. I have followed the guide on how ti install filebeat with the kibana dashboard. Now I can see that the following indeices have red flag : kibana\_1 and filebeat

I can also see that these shards is red :

`{"status":"red","number_of_shards":1,"number_of_replicas":0,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":1,"shards":{"0":{"status":"red","primary_active":false,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":1}}},"filebeat-7.5.1-2020.04.27-000001":{"status":"red","number_of_shards":1,"number_of_replicas":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":2,"shards":{"0":{"status":"red","primary_active":false,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":2}}}}}`

Elastciserach is throwing this exception :

```
[2020-04-27T16:11:00,081][WARN][r.suppressed] [My-node-1] path: /.kibana/_count, params: {index=.kibana}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:534) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:305) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:563) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:384) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.lambda$performPhaseOnShard$0(AbstractSearchAsyncAction.java:219) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction$2.doRun(AbstractSearchAsyncAction.java:284) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.5.1.jar:7.5.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]

```

I have already reinstalled the entire environment but that just results in the same exception? It seems like everything works great until filebeat is insatlled and running.

Regards

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 27, 2020, 11:02pm UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/2 "2020-04-27T23:02:46Z")

</div>

What's the output from the `_cat/indices?v` and `_cat/node?v` endpoints?

---

<div class="post-metadata">

**Author:** ![Forcep](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forcep/32/67198_2.png) [@Forcep](https://discuss.elastic.co/u/Forcep)\
**Post date:** [April 28, 2020, 10:01am UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/3 "2020-04-28T10:01:14Z")

</div>

I restarted Elasticsearch and Kibana, then I re-installed filebeat :

Load template, had to remove the -e and the outputelastic to get the command running else it just complained about the format.

`filebeat.exe setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'`

This command could not be run, Invoke-RestMethod is not known in PowerShell :

`Invoke-RestMethod -Method Delete "http://localhost:9200/filebeat-*"`

This command was successful :

`filebeat.exe setup --dashboards`

This command did not work :

`Start-Service filebeat`

So I started the filebeat service with TaskManager \> Services

No problem to browse Kibana(with login), the discovery grapth was shown. But when trying to click a couple of filebeat modules I got alot of errors, and when I went back to discovery it was stuck in loading loop, refresh did nothing.

I went back to the CMDs/PS and both Elasticsearch and Kibana was complaining about all shards failed again. Restarting Kibana did not help(could not connect all shared failed), Elasticsearch needs to be restarted again and maybe I need to setup the filebeat again.

Also, even if filebeat have been running as a service it does not seems like it is picking up my logfiles? Not sure where I can see what work the filebeat really do?

When trying to browse `_cat/node?v` I get :

`{"error":"Incorrect HTTP method for uri [/_cat/node?v] and method [GET], allowed: [POST]","status":405}`

The /localhost:9200/\_cat/indices?v gets this :

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size  
green open .security-7 xssVpomvQYCjyJtzp6ILMQ 1 0  
green open .kibana\_task\_manager\_1 OBJL2YGpRSGu3V6JDQXD3g 1 0  
yellow open 5.12test\_ecacherowpatient iBIKh0NXSZO0KGIfVbueTQ 1 1 0 0 283b 283b  
green open .apm-agent-configuration 4ZptB0agRAq6Nv83UsZ1OA 1 0  
red open .kibana\_1 dEiI9h1SQauLEiKGFZB4Vg 1 0  
red open filebeat-7.5.1-2020.04.28-000001 \_tbLDOUqRuysMXuzI5zecA 1 1

Note : Im using a certificate but it is not fully valid so I hade to set allow all on some of these components.

Right now there is many lose parts that do not really work, hope we can solve it.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 28, 2020, 10:18am UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/4 "2020-04-28T10:18:54Z")

</div>

It's a little hard to follow what's happening here sorry.

If shards are failing in Elasticsearch and your cluster is red, there is a problem. To resolve that we'd need to see more of your Elasticsearch logs please.

---

<div class="post-metadata">

**Author:** ![Forcep](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forcep/32/67198_2.png) [@Forcep](https://discuss.elastic.co/u/Forcep)\
**Post date:** [April 28, 2020, 12:18pm UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/5 "2020-04-28T12:18:51Z")

</div>

I have started Elastic from cmd but all I can see at this point is :

```
[2020-04-28T14:12:33,072][DEBUG][o.e.a.s.TransportSearchAction] [My-node-1] All shards failed for phase: [query]
[2020-04-28T14:12:37,019][WARN][r.suppressed] [My-node-1] path: /.kibana/_count, params: {index=.kibana}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:534) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:305) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:563) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:384) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.lambda$performPhaseOnShard$0(AbstractSearchAsyncAction.java:219) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction$2.doRun(AbstractSearchAsyncAction.java:284) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773) [elasticsearch-7.5.1.jar:7.5.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.5.1.jar:7.5.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]

```

I have checked the ElasticSearch\log folder but there is only vary old logfiles.

How do I get what you need?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 28, 2020, 12:29pm UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/6 "2020-04-28T12:29:36Z")

</div>

Have you edited any settings in you Elasticsearch.yml file?

---

<div class="post-metadata">

**Author:** ![Forcep](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forcep/32/67198_2.png) [@Forcep](https://discuss.elastic.co/u/Forcep)\
**Post date:** [April 28, 2020, 2:04pm UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/7 "2020-04-28T14:04:40Z")

</div>

The elasticsearch.yml looks like this :

```
cluster.name: My-cluster
cluster.initial_master_nodes: ["My-node-1"]
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.flood_stage: 1gb
cluster.routing.allocation.disk.watermark.low: 3gb
cluster.routing.allocation.disk.watermark.high: 2gb

node.name: My-node-1

http.port: 9200
transport.tcp.port: 9300
network.host: 0.0.0.0

xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: none 
xpack.security.transport.ssl.keystore.path: certs/cert.pfx
xpack.security.transport.ssl.truststore.path: certs/cert.pfx

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: none 
xpack.security.http.ssl.keystore.path: certs/cert.pfx
xpack.security.http.ssl.truststore.path: certs/cert.pfx

```

It is already in use by another part of my software which works fine. Its however on a diffrent index.

---

<div class="post-metadata">

**Author:** ![Forcep](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forcep/32/67198_2.png) [@Forcep](https://discuss.elastic.co/u/Forcep)\
**Post date:** [April 29, 2020, 12:34pm UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/8 "2020-04-29T12:34:26Z")

</div>

Is there anymore information you need?

---

<div class="post-metadata">

**Author:** ![Forcep](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forcep/32/67198_2.png) [@Forcep](https://discuss.elastic.co/u/Forcep)\
**Post date:** [May 4, 2020, 8:13am UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/9 "2020-05-04T08:13:36Z")

</div>

So, any thoughts on why this is not working?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 1, 2020, 8:13am UTC](https://discuss.elastic.co/t/problems-getting-started-with-filebeat-and-elasticsearch/229974/10 "2020-06-01T08:13:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.