# Logstash not sending logs to elastic

**URL:** <https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239>\
**Category:** Logstash\
**Created:** [March 9, 2018, 9:53am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239 "2018-03-09T09:53:57Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![muthu\_kumar1](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@muthu\_kumar1](https://discuss.elastic.co/u/muthu_kumar1)\
**Post date:** [March 9, 2018, 9:53am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/1 "2018-03-09T09:53:57Z")

</div>

[2018-03-09T09:46:31,839][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>6}  
[2018-03-09T09:46:31,840][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=\>nil}  
[2018-03-09T09:46:31,843][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}}  
[2018-03-09T09:46:31,850][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//10.23.210.35:9200](https://10.23.210.35:9200)"]}  
[2018-03-09T09:46:32,050][INFO][logstash.pipeline] Pipeline started succesfully {:pipeline\_id=\>"main", :thread=\>"#\<Thread:0x3e92988d@/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/pipeline.rb:246 sleep\>"}  
[2018-03-09T09:46:32,061][INFO][logstash.agent] Pipelines running {:count=\>1, :pipelines=\>["main"]}

---

<div class="post-metadata">

**Author:** ![muthu\_kumar1](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@muthu\_kumar1](https://discuss.elastic.co/u/muthu_kumar1)\
**Post date:** [March 9, 2018, 9:55am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/2 "2018-03-09T09:55:48Z")

</div>

Can any one help me on this?

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [March 9, 2018, 10:08am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/3 "2018-03-09T10:08:13Z")

</div>

I don't see any error here. Are you sure the data is being collected by logstash and isn't being discarded. You should add a stdout output to your config and ensure that data is actually making it to the output.

---

<div class="post-metadata">

**Author:** ![muthu\_kumar1](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@muthu\_kumar1](https://discuss.elastic.co/u/muthu_kumar1)\
**Post date:** [March 9, 2018, 10:27am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/4 "2018-03-09T10:27:39Z")

</div>

Yes Robert..my config file now is as below

input {  
file {  
path =\> "/apps/appname/HUB/logs/2018-02-02/Default.log"  
start\_position =\> "beginning"  
ignore\_older =\> 0  
}  
}  
output {  
stdout {}  
file {  
path =\> "/tmp/output.txt"  
}  
}

But while executing below command  
bin/logstash -e logstash-syslog.conf

[2018-03-09T10:25:00,225][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/apps/logstash/logstash-6.2.1/modules/fb\_apache/configuration"}  
[2018-03-09T10:25:00,230][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/apps/logstash/logstash-6.2.1/modules/netflow/configuration"}  
[2018-03-09T10:25:00,454][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified  
[2018-03-09T10:25:00,545][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.2.1"}  
[2018-03-09T10:25:00,592][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}  
[2018-03-09T10:25:00,595][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, input, filter, output at line 3, column 1 (byte 76) after ", :backtrace=\>["/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/compiler.rb:50:in`compile\_graph'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in`map'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/pipeline.rb:51:in`initialize'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/pipeline_action/create.rb:40:in`execute'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/agent.rb:141:in`with\_pipelines'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in`each'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/agent.rb:166:in`block in converge\_state\_and\_update'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/agent.rb:164:in`converge\_state\_and\_update'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/runner.rb:348:in`block in execute'", "/apps/logstash/logstash-6.2.1/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

Error i am getting.please help.

The above test case to read a file using logstash and write it in one file

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [March 9, 2018, 10:53am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/5 "2018-03-09T10:53:11Z")

</div>

When specifying a pipeline config file you must use `-f` not `-e`

---

<div class="post-metadata">

**Author:** ![muthu\_kumar1](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@muthu\_kumar1](https://discuss.elastic.co/u/muthu_kumar1)\
**Post date:** [March 9, 2018, 10:58am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/6 "2018-03-09T10:58:13Z")

</div>

> [@rcowart](#):
>
> When specifying a pipeline config file you must use -f not -e

ya i used the below command

bin/logstash -f logstash-syslog.conf

But no output.txt is written in target folder

Output as below

[2018-03-09T10:59:13,687][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/apps/logstash/logstash-6.2.1/modules/fb\_apache/configuration"}  
[2018-03-09T10:59:13,692][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/apps/logstash/logstash-6.2.1/modules/netflow/configuration"}  
[2018-03-09T10:59:13,932][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified  
[2018-03-09T10:59:14,028][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.2.1"}  
[2018-03-09T10:59:14,079][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}  
[2018-03-09T10:59:14,289][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>8, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50}  
[2018-03-09T10:59:14,456][INFO][logstash.pipeline] Pipeline started succesfully {:pipeline\_id=\>"main", :thread=\>"#\<Thread:0x6cf4f4a4@/apps/logstash/logstash-6.2.1/logstash-core/lib/logstash/pipeline.rb:246 sleep\>"}  
[2018-03-09T10:59:14,467][INFO][logstash.agent] Pipelines running {:count=\>1, :pipelines=\>["main"]}

---

<div class="post-metadata">

**Author:** ![muthu\_kumar1](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@muthu\_kumar1](https://discuss.elastic.co/u/muthu_kumar1)\
**Post date:** [March 9, 2018, 11:03am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/7 "2018-03-09T11:03:05Z")

</div>

Do we need to keep on feed the data to the input file or it can read the old values also from that file?

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [March 9, 2018, 11:12am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/8 "2018-03-09T11:12:31Z")

</div>

Add `sincedb_path => "/dev/null"` to your file input so it will start over reading the file each time you start logstash. Otherwise it will read from where it last read which is probably the EOF, and the reason you get no data.

---

<div class="post-metadata">

**Author:** ![muthu\_kumar1](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@muthu\_kumar1](https://discuss.elastic.co/u/muthu_kumar1)\
**Post date:** [March 12, 2018, 9:10am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/9 "2018-03-12T09:10:02Z")

</div>

thanks robert..for your help..i am able to see logs in kibana now 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 9, 2018, 9:10am UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic/123239/10 "2018-04-09T09:10:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
