# Threshold rule can't group by with source.ip but only with source.ip.keyword **URL:** \ **Category:** SIEM\ **Created:** [October 30, 2022, 3:56pm UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761 "2022-10-30T15:56:15Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Simone\_Calo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simone_calo/32/112688_2.png) [@Simone\_Calo](https://discuss.elastic.co/u/Simone_Calo)\ **Post date:** [October 30, 2022, 3:56pm UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/1 "2022-10-30T15:56:15Z")
My problem consists in defining a threshold rule in the group by field. I can only enter source.ip.keyword and not source.ip. ![Schermata del 2022-10-30 16-54-45](https://us1.discourse-cdn.com/elastic/original/3X/9/0/905abfd18ef454e8f13fd68dd10ed32134cc8fed.png) The consequence of this is that the rule fails every time with the following error: ![Schermata del 2022-10-30 16-54-02](https://us1.discourse-cdn.com/elastic/original/3X/3/0/3093ff7d5f686acbd350f8586cc90252cb1a323c.png) Bulk Indexing of signals failed: Could not dynamically add mapping for field [source.ip.keyword]. Existing mapping for [source.ip] must be of type object but found [ip]. name: "Prova Nmap detection" id: "8c5e9c50-522d-11ed-895f-390c31d94eb0" rule id: "1fd6fcc8-e7aa-44c9-be7f-5bd699014236" signals index: ".siem-signals-default" Please help me, I am a begginer student ---
**Author:** ![georgii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/georgii/32/78076_2.png) [@georgii](https://discuss.elastic.co/u/georgii)\ **Post date:** [November 2, 2022, 12:32pm UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/2 "2022-11-02T12:32:38Z")
Hi @Simone_Calo 👋 and welcome to the forum! It looks like the issue is caused by incorrect mappings of your source events. Indices that contain source events for a given rule are determined by the `Index patterns` field. In your case, mappings of these indices seem to contain a `source.ip.keyword` field (which I'd assume has a `keyword` type) while Elastic Security expects to see there a standard [ECS](https://www.elastic.co/guide/en/ecs/current/index.html) field [`source.ip`](https://www.elastic.co/guide/en/ecs/current/ecs-source.html#field-source-ip) of type `ip`. What happens when this rule executes, is the rule copies many fields like `source.*` from a source document to the alert that is generated based on this source document, and then tries to index the alert into a separate `.alerts-security.alerts-` index. The alerts index has its own strict mappings where it is expected that `source.ip` is a field of type `ip` according to ECS. Elastic Security requires source data to be ECS-compliant to work correctly: [Elastic Security system requirements | Elastic Security Solution [8.5] | Elastic](https://www.elastic.co/guide/en/security/current/sec-requirements.html#_third_party_collectors_mapped_to_ecs) Let me know if this helps! ---
**Author:** ![Simone\_Calo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simone_calo/32/112688_2.png) [@Simone\_Calo](https://discuss.elastic.co/u/Simone_Calo)\ **Post date:** [November 2, 2022, 3:29pm UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/3 "2022-11-02T15:29:56Z")
okay, thanks a lot. So what should I do to solve the problem? ---
**Author:** ![Simone\_Calo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simone_calo/32/112688_2.png) [@Simone\_Calo](https://discuss.elastic.co/u/Simone_Calo)\ **Post date:** [November 2, 2022, 3:41pm UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/4 "2022-11-02T15:41:26Z")
Maybe here instead of filebeat I have to insert packetbeat (the only component I'm using)? ![Schermata del 2022-11-02 16-34-20](https://us1.discourse-cdn.com/elastic/original/3X/4/8/4829900a6deae360678333aa96d5e86621bbb3b1.png) ---
**Author:** ![georgii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/georgii/32/78076_2.png) [@georgii](https://discuss.elastic.co/u/georgii)\ **Post date:** [November 3, 2022, 10:57am UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/5 "2022-11-03T10:57:58Z")
> So what should I do to solve the problem? It depends. Are you running a production cluster or just learning/testing the app locally or in Cloud? ---
**Author:** ![Simone\_Calo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simone_calo/32/112688_2.png) [@Simone\_Calo](https://discuss.elastic.co/u/Simone_Calo)\ **Post date:** [November 3, 2022, 11:43am UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/6 "2022-11-03T11:43:11Z")
I'm running the app locally for educational purposes, I'm a student. As you may have guessed, I would like to generate an alarm when many packets from the same source ip address are detected. So I necessarily need the group by clause with source.ip. Thank you very much for the time you are dedicating to me ---
**Author:** ![Simone\_Calo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simone_calo/32/112688_2.png) [@Simone\_Calo](https://discuss.elastic.co/u/Simone_Calo)\ **Post date:** [November 7, 2022, 12:24pm UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/8 "2022-11-07T12:24:08Z")
I tried to display the field with the command and I got this output: ![Schermata del 2022-11-07 13-19-00](https://us1.discourse-cdn.com/elastic/original/3X/c/f/cf6f9a4619a63ec18694851eb050ccfaad1a012e.png) It can be useful? Also in discover i can run this query and safely achieve this ![pacchetti-tcp-scan](https://us1.discourse-cdn.com/elastic/original/3X/a/9/a996cd4e1d0fd28af9e98328ad14a88e5cd9ef83.png) I would be very grateful if you could help me because I don't have much time (I have an exam on Wednesday). I await your kind reply, thank you very much. ---
**Author:** ![georgii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/georgii/32/78076_2.png) [@georgii](https://discuss.elastic.co/u/georgii)\ **Post date:** [November 7, 2022, 12:56pm UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/9 "2022-11-07T12:56:30Z")
Hey @Simone_Calo, ultimately, you need to fix the incorrect mappings in your `packetbeat-*` indices. Since you're running the app locally for educational purposes, the easiest way would probably be to erase all your Elasticsearch data and start from scratch. 1. Export the rules you need (there's a bulk action for it in the Rules table). 2. Stop packetbeat and any other beats you have. 3. Stop your local Kibana and Elasticsearch instances. 4. Delete the folder where Elasticsearch stores its data on the file system. The path depends on your OS and the way you installed ES, please refer to the docs to determine it [Configuring Elasticsearch | Elasticsearch Guide [8.5] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.5/settings.html) 5. Start Elasticsearch and Kibana. 6. Execute `packetbeat setup -e` ([docs](https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-installation-configuration.html)) - this will make sure to correctly set mappings for the `packetbeat-*` indices, as well as do other preparatory things. 7. Start packetbeat. 8. Import the previously exported rules. It's important to run `...beat setup -e` for any beat before running it. Running it without this command leads to ES creating bad index mappings which was probably the reason for your issue. Hope this helps. Best of luck with the exam! ---
**Author:** ![Simone\_Calo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simone_calo/32/112688_2.png) [@Simone\_Calo](https://discuss.elastic.co/u/Simone_Calo)\ **Post date:** [November 7, 2022, 1:11pm UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/10 "2022-11-07T13:11:16Z")
thank you very much, I try as soon as possible as soon as it is done I update you, thank you very much ---
**Author:** ![Simone\_Calo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simone_calo/32/112688_2.png) [@Simone\_Calo](https://discuss.elastic.co/u/Simone_Calo)\ **Post date:** [November 7, 2022, 2:14pm UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/11 "2022-11-07T14:14:31Z")
I tried but unfortunately nothing has changed and I have the same error (among other things I already ran the suggested command every time when starting the packetbeat container) ![Schermata del 2022-11-07 15-13-58](https://us1.discourse-cdn.com/elastic/original/3X/2/a/2ae02b4905dc3a0f53b8bc63f02ef4fc6519a461.png) ---
**Author:** ![georgii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/georgii/32/78076_2.png) [@georgii](https://discuss.elastic.co/u/georgii)\ **Post date:** [November 8, 2022, 11:35am UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/12 "2022-11-08T11:35:36Z")
Have you tried the same steps with a fresh installation of Elastic Stack on the host OS? Containerized setup adds another dimension of complexity - e.g. you should know where data is persisted, in what order containers start, how error handling is done when not all the containers are available, etc. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 6, 2022, 11:36am UTC](https://discuss.elastic.co/t/threshold-rule-cant-group-by-with-source-ip-but-only-with-source-ip-keyword/317761/13 "2022-12-06T11:36:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.