# Logstash unable to send network log to elastic search database but raw data successfully store in system

**URL:** <https://discuss.elastic.co/t/logstash-unable-to-send-network-log-to-elastic-search-database-but-raw-data-successfully-store-in-system/340243>\
**Category:** Logstash\
**Tags:** elastic-stack-monitoring, ilm-index-lifecycle-management\
**Created:** [August 7, 2023, 8:49am UTC](https://discuss.elastic.co/t/logstash-unable-to-send-network-log-to-elastic-search-database-but-raw-data-successfully-store-in-system/340243 "2023-08-07T08:49:54Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kiran\_K](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran_k/32/123472_2.png) [@Kiran\_K](https://discuss.elastic.co/u/Kiran_K)\
**Post date:** [August 7, 2023, 8:49am UTC](https://discuss.elastic.co/t/logstash-unable-to-send-network-log-to-elastic-search-database-but-raw-data-successfully-store-in-system/340243/1 "2023-08-07T08:49:54Z")

</div>

[WARN] 2023-08-07 08:45:44.506 [[main]-pipeline-manager] elasticsearch - Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>8}  
[INFO] 2023-08-07 08:45:44.519 [[main]-pipeline-manager] elasticsearch - Not eligible for data streams because config contains one or more settings that are not compatible with data streams: {"index"=\>"[@metadata][pipeline]-%{+YYYY.MM.dd}"}  
**[INFO] 2023-08-07 08:45:44.519 [[main]-pipeline-manager] elasticsearch - Data streams auto configuration (`data_stream => auto` or unset) resolved to `false`**  
[INFO] 2023-08-07 08:45:44.536 [Ruby-0-Thread-11: /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.15.9-java/lib/logstash/plugin\_mixins/elasticsearch/common.rb:160] elasticsearch - Using a default mapping template {:es\_version=\>8, :ecs\_compatibility=\>:v8}  
[INFO] 2023-08-07 08:45:44.548 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>4, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>500, "pipeline.sources"=\>["/etc/logstash/conf.d/test2.conf"], :thread=\>"#\<Thread:0x4ac9cea7@/usr/share/logstash/logstash-core/lib/logstash/java\_pipeline.rb:134 run\>"}  
[INFO] 2023-08-07 08:45:45.302 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=\>0.75}  
[INFO] 2023-08-07 08:45:45.308 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=\>"main"}  
[INFO] 2023-08-07 08:45:45.316 [Agent thread] agent - Pipelines running {:count=\>1, :running\_pipelines=\>[:main], :non\_running\_pipelines=\>}  
[INFO] 2023-08-07 08:45:45.367 [[main]\<udp] udp - Starting UDP listener {:address=\>"0.0.0.0:9006"}  
[INFO] 2023-08-07 08:45:45.377 [[main]\<udp] udp - UDP listener started {:address=\>"0.0.0.0:9006", :receive\_buffer\_bytes=\>"106496", :queue\_size=\>"2000"}  
**[INFO] 2023-08-07 08:45:45.771 [[main]\>worker2] file - Opening file {:path=\>"/var/data/logs-2023.08.07.json"**

**Logstash\_configuartion**  
input {  
udp {  
port =\> 9006  
codec =\> cef {}  
type =\> syslog  
tags =\> ["name"]  
}  
}  
output {  
if [@metadata][pipeline] {  
elasticsearch {  
hosts =\> ["[https://192.168.1.152:9200](https://192.168.1.152:9200)"]  
ssl =\> true  
pipeline =\> "%{[@metadata][pipeline]}"  
ssl\_certificate\_verification =\> false  
user =\> "elastic"  
password =\> "password"  
cacert =\> "/usr/share/elasticsearch/ca/ca.crt"  
index =\> "[@metadata][pipeline]-%{+YYYY.MM.dd}"  
}  
}  
file {  
path =\> "/var/data/logs-%{+YYYY.MM.dd}.json"  
}  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 7, 2023, 5:22pm UTC](https://discuss.elastic.co/t/logstash-unable-to-send-network-log-to-elastic-search-database-but-raw-data-successfully-store-in-system/340243/2 "2023-08-07T17:22:10Z")

</div>

What makes you think [@metadata][pipeline] is not nil?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 4, 2023, 5:22pm UTC](https://discuss.elastic.co/t/logstash-unable-to-send-network-log-to-elastic-search-database-but-raw-data-successfully-store-in-system/340243/3 "2023-09-04T17:22:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.