# Could not index event to Elasticsearch DataStream **URL:** \ **Category:** Elasticsearch\ **Tags:** datastreams\ **Created:** [April 19, 2022, 2:35pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-datastream/302722 "2022-04-19T14:35:45Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![wcpoon](https://avatars.discourse-cdn.com/v4/letter/w/da6949/32.png) [@wcpoon](https://discuss.elastic.co/u/wcpoon)\ **Post date:** [April 19, 2022, 2:35pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-datastream/302722/1 "2022-04-19T14:35:46Z")
Hi, I'm having warning to ingest data to Data Stream from winlogbeat \> logstash then output to Elasticsearch. Below is the warning message, any one can assist me? what should I do to resolve it? ```auto Apr 19 21:34:10 z3lynx-logs-02 logstash[47910]: [2022-04-19T21:34:10,723][WARN][logstash.outputs.elasticsearch][main][0a7212f931cbeda2c2d8a0b7cfa39e757359a378bb5b2ce2811861c717a414f8] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-8.1.2-2022.04.19", :routing=>nil}, {"@version"=>"1", "@timestamp"=>2022-04-19T13:34:17.950Z, "winlog"=>{"process"=>{"thread"=>{"id"=>4776}, "pid"=>4}, "keywords"=>["Audit Success"], "opcode"=>"Info", "api"=>"wineventlog", "provider_guid"=>"{54849625-5478-4994-A5BA-3E3B0328C30D}", "task"=>"Filtering Platform Connection", "provider_name"=>"Microsoft-Windows-Security-Auditing", "event_id"=>"5156", "record_id"=>323167969, "computer_name"=>"zeus01.my.internal", "version"=>1, "event_data"=>{"RemoteMachineID"=>"S-1-0-0", "DestAddress"=>"10.1.4.1", "ProcessID"=>"2484", "LayerName"=>"%%14610", "Direction"=>"%%14592", "FilterRTID"=>"0", "DestPort"=>"53", "SourcePort"=>"45803", "Application"=>"\\device\\harddiskvolume2\\windows\\system32\\dns.exe", "SourceAddress"=>"10.1.4.11", "Protocol"=>"17", "LayerRTID"=>"44", "RemoteUserID"=>"S-1-0-0"}, "channel"=>"Security"}, "agent"=>{"version"=>"8.1.2", "id"=>"2d8c24dd-1d21-4a30-ade1-725eff0873c8", "ephemeral_id"=>"d4e5ad47-6830-4130-af16-fe2113f0548c", "type"=>"winlogbeat", "name"=>"zeus01"}, "event"=>{"kind"=>"event", "original"=>"The Windows Filtering Platform has permitted a connection.\n\nApplication Information:\n\tProcess ID:\t\t2484\n\tApplication Name:\t\\device\\harddiskvolume2\\windows\\system32\\dns.exe\n\nNetwork Information:\n\tDirection:\t\tInbound\n\tSource Address:\t\t10.1.4.11\n\tSource Port:\t\t45803\n\tDestination Address:\t10.1.4.1\n\tDestination Port:\t\t53\n\tProtocol:\t\t17\n\nFilter Information:\n\tFilter Run-Time ID:\t0\n\tLayer Name:\t\tReceive/Accept\n\tLayer Run-Time ID:\t44", "created"=>"2022-04-19T13:34:19.292Z", "action"=>"Filtering Platform Connection", "provider"=>"Microsoft-Windows-Security-Auditing", "outcome"=>"success", "code"=>"5156"}, "host"=>{"id"=>"0d16da59-b137-4ce2-be39-d10604f23521", "ip"=>["10.1.4.1", "fe80::5efe:a01:401"], "name"=>"zeus01.my.internal", "mac"=>["00:50:56:89:ba:9a", "00:00:00:00:00:00:00:e0"], "hostname"=>"zeus01", "os"=>{"version"=>"10.0", "type"=>"windows", "name"=>"Windows Server 2016 Datacenter", "build"=>"14393.4825", "platform"=>"windows", "kernel"=>"10.0.14393.4825 (rs1_release.211202-1611)", "family"=>"windows"}, "architecture"=>"x86_64"}, "message"=>"The Windows Filtering Platform has permitted a connection.\n\nApplication Information:\n\tProcess ID:\t\t2484\n\tApplication Name:\t\\device\\harddiskvolume2\\windows\\system32\\dns.exe\n\nNetwork Information:\n\tDirection:\t\tInbound\n\tSource Address:\t\t10.1.4.11\n\tSource Port:\t\t45803\n\tDestination Address:\t10.1.4.1\n\tDestination Port:\t\t53\n\tProtocol:\t\t17\n\nFilter Information:\n\tFilter Run-Time ID:\t0\n\tLayer Name:\t\tReceive/Accept\n\tLayer Run-Time ID:\t44", "log"=>{"level"=>"information"}, "tags"=>["beats_input_codec_plain_applied"], "ecs"=>{"version"=>"8.0.0"}}], :response=>{"index"=>{"_index"=>"winlogbeat-8.1.2-2022.04.19", "_id"=>nil, "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"only write ops with an op_type of create are allowed in data streams"}}}} ``` ```auto "reason"=>"only write ops with an op_type of create are allowed in data streams"}}}} ``` ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 19, 2022, 2:54pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-datastream/302722/2 "2022-04-19T14:54:10Z")
Perhaps you could post your logstash configuration Also did you follow the docs > **[Elasticsearch output plugin | Logstash Reference \[8.1\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data-streams)** ---
**Author:** ![wcpoon](https://avatars.discourse-cdn.com/v4/letter/w/da6949/32.png) [@wcpoon](https://discuss.elastic.co/u/wcpoon)\ **Post date:** [April 20, 2022, 3:19am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-datastream/302722/3 "2022-04-20T03:19:38Z")
@stephenb Below is my logstash output, ```auto input { beats { port => 5044 } } output { elasticsearch { hosts => ["https://10.10.10.100:9200""] cacert => '/etc/logstash/certs/http_ca.crt' index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" ssl => "true" api_key => "xxxxxxxxxxxxxxxxxxxxx" } } ``` I read through few articles, after I added in **action =\> "create**" in the output. My index event to datastream is working now. I read the documentation, by default, it is **action =\> create** , why I need to include in my config? And my data stream look like that now. ```auto winlogbeat-8.1.2-2022.04.19 .ds-winlogbeat-8.1.2-2022.04.19-2022.04.18-000001 <<< Ingest data yesterday .ds-winlogbeat-8.1.2-2022.04.19-2022.04.20-000002 winlogbeat-8.1.2-2022.04.20 .ds-winlogbeat-8.1.2-2022.04.20-2022.04.19-000001 <<< It is ingesting data now ``` Why there are 2 datastream created? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 20, 2022, 3:29am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-datastream/302722/4 "2022-04-20T03:29:18Z")
You did not look close `index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"` Should be `index => "%{[@metadata][beat]}-%{[@metadata][version]}"` Without the date part at the end... I specifically put that so it uses what is known as the write alias. > [@wcpoon](#): > > I read the documentation, by default, it is **action =\> create** , why I need to include in my config? Long story But the way we configure it logstash does not know we are passing through a datastream (which would use that default,) The data stream is set up by metricbest or winlogbest whatever you're using so we have to explicitly set that setting in logstash so it can only create. If you were to clean up and do it exactly as I showed you, you should get the exact result. ---
**Author:** ![wcpoon](https://avatars.discourse-cdn.com/v4/letter/w/da6949/32.png) [@wcpoon](https://discuss.elastic.co/u/wcpoon)\ **Post date:** [April 20, 2022, 4:16am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-datastream/302722/5 "2022-04-20T04:16:07Z")
Hi, I'm trying it out now. Will update the outcome later. Thanks for your assistant. Really appreciate it. ---
**Author:** ![wcpoon](https://avatars.discourse-cdn.com/v4/letter/w/da6949/32.png) [@wcpoon](https://discuss.elastic.co/u/wcpoon)\ **Post date:** [April 21, 2022, 5:55am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-datastream/302722/6 "2022-04-21T05:55:06Z")
@stephenb Finally, everything looks good now. ```auto green open .ds-winlogbeat-8.1.2-2022.04.21-000002 pz0luwQ4SC2iHFCX2MmxAQ 1 1 479624 0 377.6mb 186mb green open .ds-winlogbeat-8.1.2-2022.04.20-000001 FhKN6HItQ-61im2r8mcBhg 1 1 6189173 0 4.3gb 2.1gb ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 19, 2022, 5:55am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-datastream/302722/7 "2022-05-19T05:55:59Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.