# Logstash not sending logs to Elasticsearch **URL:** \ **Category:** Logstash\ **Created:** [March 18, 2022, 11:43am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037 "2022-03-18T11:43:58Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![wallace84](https://avatars.discourse-cdn.com/v4/letter/w/bb73d2/32.png) [@wallace84](https://discuss.elastic.co/u/wallace84)\ **Post date:** [March 18, 2022, 11:43am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/1 "2022-03-18T11:43:58Z")
Hello, I have been trying for some time to send a simple log to Elasticsearch and after trying a very simple example, the logs are not been sent to Elasticsearch from logstash. Services: In same server for this test Operative Sytem: Centos 7 The logstash version is: 7.17.1 The Elasticsearch version is: 7.17.1 /etc/logstash/conf.d input { file { path =\> "/var/log/Elasticsearch/Elasticsearch.log" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { } output { Elasticsearch { hosts =\> ["localhost:9200"] } } /var/log/logstash/logstash-plain.log [2022-03-18T11:33:29,161][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600, :ssl\_enabled=\>false} [2022-03-18T11:33:30,690][INFO][org.reflections.Reflections] Reflections took 118 ms to scan 1 urls, producing 119 keys and 417 values [2022-03-18T11:33:32,042][INFO][logstash.outputs.Elasticsearch][main] New Elasticsearch output {:class=\>"LogStash::Outputs::Elasticsearch", :hosts=\>["[//localhost:9200](https://localhost:9200)"]} [2022-03-18T11:33:32,540][INFO][logstash.outputs.Elasticsearch][main] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://localhost:9200/](http://localhost:9200/)]}} [2022-03-18T11:33:32,889][WARN][logstash.outputs.Elasticsearch][main] Restored connection to ES instance {:url=\>"[http://localhost:9200/](http://localhost:9200/)"} [2022-03-18T11:33:32,908][INFO][logstash.outputs.Elasticsearch][main] Elasticsearch version determined (7.17.1) {:es\_version=\>7} [2022-03-18T11:33:32,913][WARN][logstash.outputs.Elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7} [2022-03-18T11:33:33,037][INFO][logstash.outputs.Elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false` [2022-03-18T11:33:33,113][INFO][logstash.outputs.Elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false` [2022-03-18T11:33:33,311][INFO][logstash.outputs.Elasticsearch][main] Using a default mapping template {:es\_version=\>7, :ecs\_compatibility=\>:disabled} [2022-03-18T11:33:33,337][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>2, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>250, "pipeline.sources"=\>["/etc/logstash/conf.d/logstash.conf"], :thread=\>"#\"} [2022-03-18T11:33:34,573][INFO][logstash.javapipeline][main] Pipeline Java execution initialization time {"seconds"=\>1.23} [2022-03-18T11:33:34,664][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=\>"main"} [2022-03-18T11:33:34,761][INFO][filewatch.observingtail][main][2b6c69038f817ebf29690e5d479fe4c6e56f482b9d6cc052978d217447903269] START, creating Discoverer, Watch with file and sincedb collections [2022-03-18T11:33:34,771][INFO][logstash.agent] Pipelines running {:count=\>1, :running\_pipelines=\>[:main], :non\_running\_pipelines=\>} /var/log/Elasticsearch/Elasticsearch.log [2022-03-18T01:30:00,079][INFO][o.e.x.m.MlDailyMaintenanceService] [ip-_.eu-west-2.compute.internal] Successfully completed [ML] maintenance task: triggerDeleteExpiredDataTask [2022-03-18T10:23:44,010][INFO][o.e.c.m.MetadataIndexTemplateService] [ip-_.eu-west-2.compute.internal] adding template [logstash] for index patterns [logstash-_] [2022-03-18T10:23:44,189][INFO][o.e.c.m.MetadataCreateIndexService] [ip-_.eu-west-2.compute.internal] [logstash-2022.03.18-000001] creating index, cause [api], templates [logstash], shards [1]/[1] [2022-03-18T10:23:44,522][INFO][o.e.x.i.a.TransportPutLifecycleAction] [ip-_.eu-west-2.compute.internal] adding index lifecycle policy [logstash-policy] [2022-03-18T10:23:44,603][INFO][o.e.x.i.IndexLifecycleTransition] [ip-_.eu-west-2.compute.internal] moving index [logstash-2022.03.18-000001] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [logstash-policy] [2022-03-18T10:23:44,671][INFO][o.e.x.i.IndexLifecycleTransition] [ip-_.eu-west-2.compute.internal] moving index [logstash-2022.03.18-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] in policy [logstash-policy] [2022-03-18T10:23:44,726][INFO][o.e.x.i.IndexLifecycleTransition] [ip-_.eu-west-2.compute.internal] moving index [logstash-2022.03.18-000001] from [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] to [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}] in policy [logstash-policy] [2022-03-18T10:23:55,371][INFO][o.e.c.r.a.DiskThresholdMonitor] [ip-\*.eu-west-2.compute.internal] low disk watermark [85%] exceeded on [r51WwHrKTE-VK6UCAaR4IA][ip-\*8.eu-west-2.compute.internal][/var/lib/Elasticsearch/nodes/0] free: 1.1gb[14.1%], replicas will not be assigned to this node Any help would be really appreciated : ) ---
**Author:** ![wallace84](https://avatars.discourse-cdn.com/v4/letter/w/bb73d2/32.png) [@wallace84](https://discuss.elastic.co/u/wallace84)\ **Post date:** [March 18, 2022, 11:58am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/2 "2022-03-18T11:58:28Z")
I add an additional note: From my understanding there is nothing being writen in Elasticsearch, not sure if I am missing anything in the output section. Networking issue shouldn't be if it is the same server. curl localhost:9200/global2-\*/\_search?pretty { "took" : 0, "timed\_out" : false, "\_shards" : { "total" : 0, **"successful" : 0,** "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 0, "relation" : "eq" }, "max\_score" : 0.0, "hits" : } } ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 18, 2022, 1:34pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/3 "2022-03-18T13:34:19Z")
What is the return for: `curl localhost:9200/logstash-*/_search?pretty` You are not setting an index in your logstash output, so it will use the default `logstash-*` index. Also, do you have the file `elasticsearch.log` and the user running logstash has access to it? Normally the Elasticsearch log has the name of the cluster and only the `elasticsearch` user (and root) has access to it. ---
**Author:** ![Krunal](https://avatars.discourse-cdn.com/v4/letter/k/ce7236/32.png) [@Krunal](https://discuss.elastic.co/u/Krunal)\ **Post date:** [March 18, 2022, 1:45pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/4 "2022-03-18T13:45:09Z")
Hello, May be the issue of permission mode chmod xxx (linux) of Elasticsearch.log check it out... I think logstash not taken any input such as Elasticsearch.log... it is not able to load any file in the pipeline of logstash ---
**Author:** ![wallace84](https://avatars.discourse-cdn.com/v4/letter/w/bb73d2/32.png) [@wallace84](https://discuss.elastic.co/u/wallace84)\ **Post date:** [March 18, 2022, 2:32pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/5 "2022-03-18T14:32:33Z")
Hello The command returns: curl localhost:9200/global2-\*/\_search?pretty { "took" : 0, "timed\_out" : false, "\_shards" : { "total" : 0, "successful" : 0, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 0, "relation" : "eq" }, "max\_score" : 0.0, "hits" : } } Is any better sample I could use for testing? The first thing I want to see is that is working? Is the config file configured correctly? 🙂 It is true it can be a permissions issue. Aobut the logstash config has the following permissions: /etc/logstash/conf.d -rw-r--r-- 1 root root 447 Mar 18 13:46 logstash.conf I think the elasticsearchlog maybe was giving permission issues, how I would be able to read any file with the logstash user? Any similar simple example to understand these first steps would help a lot. Thank you for your help. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 18, 2022, 2:37pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/6 "2022-03-18T14:37:48Z")
Your index is not `global2-*`, you didn't specified an index name in your logstash output, so logstash will use the default name which is `logstash-*`. You need to run: ```auto curl localhost:9200/logstash-*/_search?pretty ``` Also, the permissions that can give you an issue is not the logstash config, is the Elasticsearch log config, You need to check that. Do you have a file named `elasticsearch.log`? What are the permissions for that file? ---
**Author:** ![wallace84](https://avatars.discourse-cdn.com/v4/letter/w/bb73d2/32.png) [@wallace84](https://discuss.elastic.co/u/wallace84)\ **Post date:** [March 18, 2022, 2:39pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/7 "2022-03-18T14:39:34Z")
Additional info otal 52 /etc/logstash/ drwxr-xr-x 3 root root 156 Mar 18 11:34 . drwxr-xr-x 95 root root 8192 Mar 17 17:28 .. drwxr-xr-x 2 root root 27 Mar 18 13:46 conf.d -rw-r--r-- 1 root root 2129 Feb 23 22:19 jvm.options -rw-r--r-- 1 root root 7437 Feb 23 22:19 log4j2.properties -rw-r--r-- 1 root root 342 Feb 23 22:19 logstash-sample.conf -rw-r--r-- 1 root root 12841 Mar 17 17:28 logstash.yml -rw-r--r-- 1 root root 285 Feb 23 22:19 pipelines.yml -rw------- 1 root root 1696 Feb 23 22:19 startup.options /var/log/logstash drwxr-xr-x 2 logstash root 187 Mar 18 14:36 . drwxr-xr-x 11 root root 4096 Mar 18 09:48 .. -rw-r--r-- 1 logstash logstash 10811 Mar 18 13:48 logstash-deprecation.log -rw-r--r-- 1 logstash logstash 0 Mar 18 10:10 logstash-json.log -rw-r--r-- 1 logstash logstash 35090 Mar 18 13:48 logstash-plain.log -rw-r--r-- 1 root root 35090 Mar 18 14:36 logstash-plain-test.log -rw-r--r-- 1 logstash logstash 0 Mar 18 10:10 logstash-slowlog-json.log -rw-r--r-- 1 logstash logstash 0 Mar 18 10:10 logstash-slowlog-plain.log ---
**Author:** ![wallace84](https://avatars.discourse-cdn.com/v4/letter/w/bb73d2/32.png) [@wallace84](https://discuss.elastic.co/u/wallace84)\ **Post date:** [March 18, 2022, 2:41pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/8 "2022-03-18T14:41:09Z")
> [@wallace84](#): > curl localhost:9200/global2-\*/\_search?pretty { "took" : 0, "timed\_out" : false, "\_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 0, "relation" : "eq" }, "max\_score" : null, "hits" : } } I have changed the config file to process /var/logstash/logstash-plain.log and maybe has done something now perhaps with the successful 1? About the previous message and the permissions is anything that is not ok as the config files are root? (They came by default like this during the installation process) ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 18, 2022, 2:54pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/9 "2022-03-18T14:54:27Z")
You are not looking into the correct index. What is `global2-*`? Your logstash configuration will send data to the index `logstash-*`, not `global2-*`. You need to run `curl localhost:9200/logstash-*/_search?pretty` to see if you have data in that index. ---
**Author:** ![wallace84](https://avatars.discourse-cdn.com/v4/letter/w/bb73d2/32.png) [@wallace84](https://discuss.elastic.co/u/wallace84)\ **Post date:** [March 18, 2022, 11:58pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/10 "2022-03-18T23:58:53Z")
Hi leandrojmp, the command was run and I got this: curl localhost:9200/logstash-\*/\_search?pretty { "took" : 0, "timed\_out" : false, "\_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 0, "relation" : "eq" }, "max\_score" : null, "hits" : } } Just the successful stays in 1, I don't know if it is doing something else or just stopped sending data to Elasticsearch : | ---
**Author:** ![Krunal](https://avatars.discourse-cdn.com/v4/letter/k/ce7236/32.png) [@Krunal](https://discuss.elastic.co/u/Krunal)\ **Post date:** [March 22, 2022, 11:38am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/11 "2022-03-22T11:38:26Z")
> **[Logstash Tutorial: How to Get Started Shipping Logs | Logz.io](https://logz.io/blog/logstash-tutorial/)** > > Want to learn how to use Logstash for log and time-series data analysis? Jurgens du Toit's introductory tutorial on Logz.io will get you started. check it for more info... ---
**Author:** ![wallace84](https://avatars.discourse-cdn.com/v4/letter/w/bb73d2/32.png) [@wallace84](https://discuss.elastic.co/u/wallace84)\ **Post date:** [March 23, 2022, 5:46pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/12 "2022-03-23T17:46:32Z")
Thank you Kunal, my problem now is that there is something taking all the space in Elasticsearch and is not the logstash log file that is being parsed exceed the high disk watermark when these relocations are complete [2022-03-23T17:40:06,161][INFO][o.e.c.r.a.DiskThresholdMonitor] [ip-10-201-1-208.eu-west-2.compute.internal] high disk watermark [90%] no longer exceeded on [r51WwHrKTE-VK6UCAaR4IA][ip-10-201-1-208.eu-west-2.compute.internal][/var/lib/Elasticsearch/nodes/0] free: 879.3mb[10.7%], but low disk watermark [85%] is still exceeded [2022-03-23T17:41:06,169][INFO][o.e.c.r.a.DiskThresholdMonitor] [ip-10-201-1-208.eu-west-2.compute.internal] low disk watermark [85%] no longer exceeded on [r51WwHrKTE-VK6UCAaR4IA][ip-10-201-1-208.eu-west-2.compute.internal][/var/lib/Elasticsearch/nodes/0] free: 1.3gb[17.2%] [2022-03-23T17:42:36,183][INFO][o.e.c.r.a.DiskThresholdMonitor] [ip-10-201-1-208.eu-west-2.compute.internal] low disk watermark [85%] exceeded on [r51WwHrKTE-VK6UCAaR4IA][ip-10-201-1-208.eu-west-2.compute.internal][/var/lib/Elasticsearch/nodes/0] free: 897.4mb[10.9%], replicas will not be assigned to this node Any reason why Elasticsearch space gets very low so quickly? or a way to find out from where is coming this issue? once Elasticsearch surpases the 85% stops getting more new indexes. Logstash seems not to be the reason. : / Thanks a lot for your help ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 20, 2022, 5:47pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elasticsearch/300037/13 "2022-04-20T17:47:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.