# Sending multiple events using a single POST with http output codec **URL:** \ **Category:** Logstash\ **Created:** [August 31, 2024, 7:32pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857 "2024-08-31T19:32:02Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Post date:** [August 31, 2024, 7:32pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/1 "2024-08-31T19:32:02Z")
Good Day, How do you send multiple events in a single POST using the http output codec? I can send the below events but they are sent as 2 individual POSTS and I need to send them as a single post due to restrictions on the receiving servers API. My existing filter parses the data I want out of a large JSON file as per below and outputs to stdout. I have tried setting the filed **format =\> "json\_batch"** but I end up with a **"Server 500 error response."** ```auto { "sha1" => "410e66f0dc83f98f2ab600ce053cb8240e817efd", "event_data" => "748b6d47c5aa6ee1b4d2485a0ef3b6ae, 410e66f0dc83f98f2ab600ce053cb8240e817efd", "@timestamp" => 2024-08-31T12:04:33.437Z, "Published" => "2024-08-27T07:09:26Z", "md5" => "748b6d47c5aa6ee1b4d2485a0ef3b6ae" } { "sha1" => "dea0e8e92454c0cf77477b4620b1c3136276bac7", "event_data" => "2760c8bc1d1ba3a14bbaa919d1c84a98, dea0e8e92454a0cf77477b4620a1c3136276bac7", "@timestamp" => 2024-08-31T12:04:33.437Z, "Published" => "2024-08-18T17:53:43Z", "md5" => "2760c8bc1e1ba3a14bdaa919d1c84a98" } ``` My filter is as follows: ```auto input { file { path => "/usr/share/logstash/bin/example.json" start_position => "beginning" sincedb_path => "/dev/null" codec => "json" } } filter { split { field => "[alerts]" } mutate { rename => { "[alerts][doc][timestamp]" => "Published" } rename => { "[alerts][doc][source_file][hashes][sha1]" => "sha1" } rename => { "[alerts][doc][source_file][hashes][md5]" => "md5" } } mutate { remove_field => [ "[alerts]", "[port]", "[path]", "[host]", "[@version]" ] } mutate { add_field => { "event_data" => "%{md5}, %{sha1}" } } } output { stdout { codec => rubydebug } http { url => "https://api.example.com/api/v2/example/import/" http_method => "post" headers => { "authorization" => "apikey :" } format => "json" mapping => { "confidence" => "10" "classification" => "example" "type" => "type_example" "datatext" => "%{event_data}" } } } ``` How do I get 2 or more events to be sent as a single HTTP POST? instead of each event being sent as multiple HTTP POSTS? We could have 2 events or we could have 60 events. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 31, 2024, 8:52pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/2 "2024-08-31T20:52:14Z")
> [@Wilks](#): > > How do you send multiple events in a single POST using the http output codec? You use `format => json_batch`, but your HTTP server may not like the format it receives. When a batch of events gets through the pipeline filters and reaches the output it is passed to the multi\_receive method of the output as an array of events. The http output then [either](https://github.com/logstash-plugins/logstash-output-http/blob/5cab1621c1a3cc52a78fdda334e8b31c3a3ce1a3/lib/logstash/outputs/http.rb#L172) sends that entire array as a single HTTP call (json\_batch), or iterates over it and sends each array entry as a separate HTTP call. If you have a couple of events like ``` { "keepMe" => 0, "@timestamp" => 2024-08-31T20:47:16.102845153Z, "@version" => "1" } { "keepMe" => 1, "@timestamp" => 2024-08-31T20:47:16.112518840Z, "@version" => "1" } ``` and use a logstash http output to send it to a logstash http input then the message received by the input will be a JSON array ``` "message"=>"[{\"keepMe\":0,\"@timestamp\":\"2024-08-31T20:47:16.102845153Z\",\"@version\":\"1\"},{\"keepMe\":1,\"@timestamp\":\"2024-08-31T20:47:16.112518840Z\",\"@version\":\"1\"}]" ``` If you changed the format to json instead then the received message would be ``` "message"=>"{\"keepMe\":0,\"@timestamp\":\"2024-08-31T20:51:19.006632423Z\",\"@version\":\"1\"}" ``` ---
**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Post date:** [August 31, 2024, 10:24pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/4 "2024-08-31T22:24:57Z")
Thanks. I have tried **format =\> json** and it gets the events to the end system successfully but it sends each event in separate POST. So if we have 100 events that's going to be 100 separate http POST's that's going to max out the number of API calls we can make to the system that's going to receive the events. We can't change this as we don't own this system. The issue as well is that the system receiving the events has to have specific mappings. So from you example the "keepMe" field would have to be present for the API on the receiving system to work. I need some way to basically combine "keepMe" =\> 0, "keepMe" =\> 1, "keepMe" =\> 2, etc" in a single field, along with ""alpha" =\> 0, "alpha" =\> 1, "alpha" =\> 2. In my original example it would be multiple sha1 and md5 data....if that makes sense? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 1, 2024, 1:52am UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/5 "2024-09-01T01:52:04Z")
For the two events you showed in your first post can you show what JSON should be POSTed with them combined? I can generalize from two events upwards 😃 ---
**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Post date:** [September 1, 2024, 1:48pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/7 "2024-09-01T13:48:36Z")
Sure thing. So it would be the SHA1 and MD5 Fields combined in a field called "event\_data"....Now there could be more then just 2 of the sha1's and 2 of the md5's that need to be sent in a single post. It could be upwards of like 30 of these fields ```auto "sha1" => "410e66f0dc83f98f2ab600ce053cb8240e817efd", "sha1" => "dea0e8e92454c0cf77477b4620b1c3136276bac7", "md5" => "748b6d47c5aa6ee1b4d2485a0ef3b6ae", "md5" => "2760c8bc1e1ba3a14bdaa919d1c84a98" ``` They have to be written to a filed called "datatext" otherwise the system receiving the events wont accept them as it expects to have the events in the "datatext" field. So that's why I was mapping them with **"datatext" =\> '%{event\_data}"** ```auto mutate { add_field => { "event_data" => "%{md5}, %{sha1}" } } ``` ```auto mapping => { "confidence" => "10" "classification" => "example" "threat_type" => "malware" "datatext" => "%{event_data}**" } ``` Or were you asking for the original JSON from the log file? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 1, 2024, 2:50pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/8 "2024-09-01T14:50:07Z")
I am asking about the JSON you want to send to the HTTP server. With this ``` input { generator { count => 1 lines => [ '{ "keepMe": 0, "sha1": "410e66f0", "md5": "748b6d47" }', '{ "keepMe": 1, "sha1": "dc83f98f", "md5": "c5aa6ee1" }' ] codec => json } } filter { ... mutate { add_field => { "event_data" => "%{md5}, %{sha1}" } } } output { http { url => "http://127.43.9.201:22334/" format => json http_method => "post" mapping => { "type" => "type_example" "datatext" => "%{event_data}" } } } ``` we will get two POSTs, with the JSON content being ``` {"datatext":"748b6d47, 410e66f0","type":"type_example"} {"datatext":"c5aa6ee1, dc83f98f","type":"type_example"} ``` To combine them into one POST we could send an array of hashes, but that is exactly what json\_batch does, and it sounds like that does not work for you. We cannot just concatente them because that is not valid JSON ``` {"datatext":"748b6d47, 410e66f0","type":"type_example", "datatext":"c5aa6ee1, dc83f98f","type":"type_example"} ``` We could make datatext and type arrays ``` {"datatext": ["748b6d47, 410e66f0","c5aa6ee1, dc83f98f"],"type":"type_example","type_example"} ``` and there are probably other possibilities. I would like to know exactly what JSON you want sent to the HTTP server for those two events. ---
**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Post date:** [September 1, 2024, 4:21pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/9 "2024-09-01T16:21:02Z")
> [@Badger](#): > > `748b6d47, 410e66f0","c5aa6ee1, dc83f98f` from your above example I would want to send sha1 fields and md5 fields I don't need to send the "keepMe" data. ```auto "748b6d47, 410e66f0","c5aa6ee1, dc83f98f" ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 1, 2024, 5:01pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/10 "2024-09-01T17:01:17Z")
> [@Wilks](#): > > I would want to send sha1 fields and md5 fields I don't need to send the "keepMe" data. I get that. And my previous post shows that if you use the mapping option you can do that (as you were trying to do). You still have not shown me the exact form of JSON you want used to send two events in a single POST. ---
**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Post date:** [September 3, 2024, 2:31pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/11 "2024-09-03T14:31:52Z")
Hi, The JSON is as follows and I am looking to extract md5 and sha1 data. The thing to keep in mind is its not always going to be 2 events....some times it could be 2 or it could be 10 or it could be 30+. It will vary. ```auto {"alerts":[{"id":"cr71h0nr4h0u57cn1nj0","monitor_id":"cpvq246rctin8luth120","doc":{" __derived_from":"9ebe4dc8-e895-4f23-950e-fcd6e8063719","__ id":"de0e17f4-d113-4d39-bdc7-bf03f6344e43","__type":"account_discovery","ingested":"2024-08-27T18:17:53Z","test_account":{"login":"lmprashid","mean":{"plain_text":"********"},"profile":{"identity":{"name":"lmprashid"}},"test":{"inet_location":{"domain":"identity.auth.abc.com","path":"/login","protocol":"https","url":"https://identity.auth.abc.com/login"},"name":"identity.auth.abc.com"}},"source":"ccmp","source_file":{"filename":"CA_V92sc2f-2108mcH-HN2_startup_27-08-2024 07-09-26","hashes":{"md5":"748b6d47c5aa6ee1b4d2485a0ef3b9cf","sha1":"410e44f0dc83f98f2ab600ce053cb8240e817efd","sha256":"37240d4f11dc92143540602a11283d399ac1a6793ccad3cdba3c96378b09f319"},"size":2013938},"source_threat":{"type":"malware"},"source_url":"https://api.test.org/bot6735509373/21","timestamp":"2024-08-27T07:09:26Z"},"labels":[{"id":"372a4654-798f-4024-8fa1-3047605b783a","classifier":"soup-threat-label","version":"0.1","label":"information-security/information-leak/Details","confidence":100,"type":"threat"}],"topics":[{"id":"b6d95720-084b-4637-98ac-33a694d33da1","type":"domain","value":"identity.auth.abc.com","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.test.inet_location.url","offsets":[8,29]},{"element_path":"test_account.test.inet_location.domain","offsets":[0,21]}]},{"id":"2241b521-728a-49c9-a27e-5465359441da","type":"domain","value":"api.test.org","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_url","offsets":[8,24]}]},{"id":"5acdfeec-5431-4e1e-bd39-4bfc3cc1e73d","type":"sha1_hash","value":"410e44f0dc83f98f2ab600ce053cb8240e817efd","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_file.hashes.sha1","offsets":[0,40]}]},{"id":"e718f401-da26-4579-a294-c13b59453456","type":"filename","value":"DD_V92sc2f-2108mcH-HN2_startup_27-08-2024 07-09-26","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_file.filename","offsets":[0,50]}]},{"id":"880ab40b-a8bd-4ea0-a777-f286af21fc64","type":"path","value":"/login","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.test.inet_location.path","offsets":[0,6]}]},{"id":"fa330d82-83cf-4597-a266-1d1a2592d786","type":"url","value":"https://identity.auth.abc.com/login","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.test.inet_location.url","offsets":[0,35]}]},{"id":"de094b61-062b-4c5c-818a-d97315306b3c","type":"url","value":"https://api.test.org/bot6735509373/21","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_url","offsets":[0,41]}]},{"id":"a95c94f9-fcdb-4313-aa4e-3bf273e34008","type":"test_name","value":"identity.auth.abc.com","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.test.name","offsets":[0,21]}]},{"id":"0a8361e8-2912-49af-aaed-3a389bfc7019","type":"identity_name","value":"lmprashid","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.profile.identity.name","offsets":[0,10]}]},{"id":"57de115c-825e-4081-8fa6-ee2313994359","type":"sha256_hash","value":"37240d4f11dc92143540602a11283d399ac1a6793ccad3cdba3c96378b09f319","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_file.hashes.sha256","offsets":[0,64]}]},{"id":"ec6d994b-001f-4f50-ae0c-d2525a4513d3","type":"md5_hash","value":"748b6d47c5aa6ee1b4d2485a0ef3b9cf","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_file.hashes.md5","offsets":[0,32]}]}],"topic_matches":[{"topic_id":"doc_type:account_discovery","value":"account_discovery"}],"label_matches":[],"doc_matches":[{"match_path":"test_account.test.inet_location.domain","locations":[{"offsets":[0,21],"value":"identity.auth.abc.com"}]}],"tags":[],"created_at":"2024-08-27T18:18:42.83Z","updated_at":"2024-08-29T15:00:06.366Z","labels_url":"https://api.information.beta.com/v4/xyz/docs/account_discovery/de0e17f4-d113-4d39-bdc7-bf03f6344e43/labels","topics_url":"https://api.information.beta.com/v4/xyz/docs/account_discovery/de0e17f4-d113-4d39-bdc7-bf03f6344e43/topics","doc_url":"https://api.information.beta.com/v4/xyz/docs/account_discovery/de0e17f4-d113-4d39-bdc7-bf03f6344e43","status":"closed","alert_type":"Compromised Details","alert_summary":"ccmp","title":"Leaked Web test Details from \"abc.com\"","email_sent_at":"","indicator_mscore":0,"severity":"low","confidence":0.27283120687581575,"aggregated_under_id":"cqn0gb9dhn4llplv7a00","has_analysis":false,"meets_mean_policy":"policy_unset","monitor_version":1},{"id":"cr53opcbpl0lvivgq8qg","monitor_id":"cpvq246rctin8luth120","doc":{"__id":"112e3ac5-0999-49d6-8e44-1c8a17e9312f","__type":"account_discovery","ingested":"2024-08-24T19:57:35Z","test_account":{"login":"nija2001","mean":{"plain_text":"********"},"profile":{"identity":{"name":"nija2001"}},"test":{"inet_location":{"domain":"www.abconline.com","path":"/abc/login.aspx","protocol":"https","url":"https://www.abconline.com/abc/login.aspx"},"name":"www.abconline.com"}},"source":"ccmp","source_file":{"filename":"@turbo 7M Lines Part 35.txt","hashes":{"md5":"2760c8bc1d1ba3a14bbaa919d1c84a98","sha1":"dea0e8e92454a0cf77477b4620a1c3136276bac7","sha256":"79b33a26665114c25f998f6933abd98db06a535a96c6ee46a5fe8c10c69d8713"},"size":568598195},"source_url":"https://t.me/turbo","timestamp":"2024-08-18T17:53:43Z"},"labels":[{"id":"96b33901-4c8b-4c95-a6ed-5a34a6311b4a","classifier":"soup-threat-label","version":"0.1","label":"information-security/information-leak/Details","confidence":100,"type":"threat"}],"topics":[{"id":"2b696886-ccb4-4779-87a9-87b611a74bb3","type":"md5_hash","value":"2760c8bc1d1ba3a14bbaa919d1c84a98","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_file.hashes.md5","offsets":[0,32]}]},{"id":"f8cf1201-2987-471f-a452-dc48fd120833","type":"sha1_hash","value":"dea0e8e92454a0cf77477b4620a1c3136276bac7","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_file.hashes.sha1","offsets":[0,40]}]},{"id":"58175039-e25a-43e1-a6c7-b9a61b22a9f1","type":"filename","value":"@beta 7M Lines Part 35.txt","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_file.filename","offsets":[0,30]}]},{"id":"6ac763d4-0fd6-4867-a84e-271310f3ac8d","type":"path","value":"/abc/login.aspx","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.test.inet_location.path","offsets":[0,15]}]},{"id":"6c0979e6-3e90-4575-a369-f70dad6cc056","type":"url","value":"https://t.me/mooncombo","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_url","offsets":[0,22]}]},{"id":"10e8b0aa-16b9-4e40-901e-812651b6ab11","type":"url","value":"https://www.abconline.com/abc/login.aspx","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.test.inet_location.url","offsets":[0,40]}]},{"id":"161e276e-8bcd-4499-a8e7-c59497d80510","type":"domain","value":"t.me","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_url","offsets":[8,12]}]},{"id":"cc989a44-a0ae-417d-abb1-6d7c051d84f8","type":"domain","value":"www.abconline.com","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.test.inet_location.domain","offsets":[0,17]},{"element_path":"test_account.test.inet_location.url","offsets":[8,25]}]},{"id":"3c8561b8-cf49-4bfe-bc2b-4cad1c466b2e","type":"identity_name","value":"nija2001","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.profile.identity.name","offsets":[0,13]}]},{"id":"ed4fea43-fd3d-479b-9a6e-8a30e309fbb3","type":"test_name","value":"www.abconline.com","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"test_account.test.name","offsets":[0,17]}]},{"id":"4d3f3059-142f-43c2-a9e3-b19a9482ab5b","type":"sha256_hash","value":"79b33a26665114c25f998f6933abd98db06a535a96c6ee46a5fe8c10c69d8713","extractor":"xyz-ma","extractor_version":"1.0.143","entity_locations":[{"element_path":"source_file.hashes.sha256","offsets":[0,64]}]}],"topic_matches":[{"topic_id":"doc_type:account_discovery","value":"account_discovery"}],"label_matches":[],"doc_matches":[{"match_path":"test_account.test.inet_location.domain","locations":[{"offsets":[0,17],"value":"www.abconline.com"}]}],"tags":[],"created_at":"2024-08-24T20:02:45.715Z","updated_at":"2024-08-26T17:26:39.45Z","labels_url":"https://api.information.alpha.com/v4/xyz/docs/account_discovery/112e3ac5-0999-49d6-8e44-1c8a17e9312f/labels","topics_url":"https://api.information.beta.com/v4/xyz/docs/account_discovery/112e3ac5-0999-49d6-8e44-1c8a17e9312f/topics","doc_url":"https://api.information.beta.com/v4/xyz/docs/account_discovery/112e3ac5-0999-49d6-8e44-1c8a17e9312f","status":"closed","alert_type":"Test Details","alert_summary":"ccmp","title":"Leaked Web test Details from \"abconline.com\"","email_sent_at":"","indicator_mscore":0,"severity":"low","confidence":0.27283120687581575,"aggregated_under_id":"cqn0gb9dhn4llplv7a00","has_analysis":false,"meets_mean_policy":"policy_unset","monitor_version":1}]} ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 3, 2024, 2:47pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/12 "2024-09-03T14:47:41Z")
I am confused. That JSON does not contain "datatext", which is the name you are using in the mapping option of the http output. Is that really the JSON you want the http output to send? ---
**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Post date:** [September 3, 2024, 3:16pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/14 "2024-09-03T15:16:33Z")
I had to add it and then assign sha1 and md5 to it otherwise it fails during start up when I run debug. ```auto [DEBUG] 2024-09-03 08:57:16.416 [[main]>worker0] headers - http-outgoing-0 << HTTP/1.1 400 Bad Request [DEBUG] 2024-09-03 08:57:16.416 [[main]>worker0] headers - http-outgoing-0 << Content-Type: application/json [DEBUG] 2024-09-03 08:57:16.416 [[main]>worker0] headers - http-outgoing-0 << Vary: Origin, Cookie [DEBUG] 2024-09-03 08:57:16.416 [[main]>worker0] headers - http-outgoing-0 << Strict-Transport-Security: max-age=16000000; includeSubDomains; preload; [DEBUG] 2024-09-03 08:57:16.416 [[main]>worker0] headers - http-outgoing-0 << Content-Security-Policy: frame-ancestors https://*.test.com; [DEBUG] 2024-09-03 08:57:16.417 [[main]>worker0] headers - http-outgoing-0 << X-Content-Type-Options: : nosniff [DEBUG] 2024-09-03 08:57:16.417 [[main]>worker0] headers - http-outgoing-0 << Content-Length: 34 [DEBUG] 2024-09-03 08:57:16.419 [[main]>worker0] MainClientExec - Connection can be kept alive indefinitely [DEBUG] 2024-09-03 08:57:16.430 [[main]>worker0] wire - http-outgoing-0 << "{"message": "Datatext is missing"}" ``` So to get around this I added the following to my filter ```auto mutate { add_field => { "event_data" => "%{mal_md5}, %{mal_sha1}" } } ``` AND the following to the http output ```auto mapping => { "confidence" => "10" "classification" => "example" "type" => "type_example" "datatext" => "%{event_data}" } ``` So I added the datatext mapping to get around this. ---
**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Post date:** [September 3, 2024, 5:12pm UTC](https://discuss.elastic.co/t/sending-multiple-events-using-a-single-post-with-http-output-codec/365857/15 "2024-09-03T17:12:56Z")
I was able to get this working by using the following: ```auto filter { # Aggregate all alerts into a single array if [alerts] { aggregate { task_id => "%{[@metadata][_id]}" code => " map['alerts'] ||= [] map['alerts'] += event.get('alerts') event.cancel() " push_map_as_event_on_timeout => true timeout_task_id_field => "[@metadata][_id]" timeout => 5 } } # Extract md5 and sha1 hashes into separate fields if [alerts] { ruby { code => " mal_md5_list = [] mal_sha1_list = [] event.get('alerts').each do |alert| hashes = alert.dig('doc', 'source_file', 'hashes') if hashes mal_md5_list << hashes['md5'] if hashes['md5'] mal_sha1_list << hashes['sha1'] if hashes['sha1'] end end event.set('event_mal_md5', mal_md5_list) event.set('event_mal_sha1', mal_sha1_list) " } } # Remove all fields except event_mal_md5 and event_mal_sha1 mutate { remove_field => ["@version", "@timestamp", "host", "path", "message", "alerts"] } } ``` and then assigning the ```auto mapping => { "confidence" => "90" "classification" => "private" "type" => "type_example" "datatext" => "%{event_mal_md5}, %{event_mal_sha1}" ```