# Convert IP from integears **URL:** \ **Category:** Elastic Search\ **Created:** [February 18, 2025, 1:54pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687 "2025-02-18T13:54:50Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\ **Post date:** [February 18, 2025, 1:54pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/1 "2025-02-18T13:54:50Z")
Hi I have two fields with integers. How can I convert it to IP ? netconn\_ipv4: 178,258,031 netconn\_local\_ipv4: 178,258,037 I guess I can do this with mappings but not sure. I have tested with following without luck. PUT /logs-carbon\_black.observations-default-2025.02.09-000002 { "mappings": { "properties": { "netconn\_ipv4": { "type": "ip" }, "netconn\_local\_ipv4": { "type": "ip" } } } } { "error": { "root\_cause": [ { "type": "illegal\_argument\_exception", "reason": "cannot create index with name [logs-carbon\_black.observations-default-2025.02.09-000002], because it matches with template [logs] that creates data streams only, use create data stream api instead" } ], "type": "illegal\_argument\_exception", "reason": "cannot create index with name [logs-carbon\_black.observations-default-2025.02.09-000002], because it matches with template [logs] that creates data streams only, use create data stream api instead" }, "status": 400 } ---
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [February 18, 2025, 4:59pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/2 "2025-02-18T16:59:20Z")
The error message tells you the issue. There’s an existing template / data streaming (fancy name for an alias to multiple indices, don’t shoot me for being imprecise), maybe that’s as it came with the system. Your index name matches, so you cannot use it really. Read the data stream section of the docs. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 18, 2025, 5:33pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/3 "2025-02-18T17:33:03Z")
Hi @arcsons What you are trying to do is going to lead to all sorts of issues! 🙂 Do not do that! 🙂 The right way to do this is add a custom ingest pipeline to convert those string to IPs Looks at this.. > **[Tutorial: Transform data with custom ingest pipelines | Fleet and Elastic...](https://www.elastic.co/guide/en/fleet/current/data-streams-pipeline-tutorial.html)** In your ```auto PUT _ingest/pipeline/logs@custom { "processors": [ { "pipeline": { "name": "mitre_attack_pipeline", "if": "ctx?.data_stream.dataset != null && (ctx?.data_stream.dataset == 'carbon_black.observations' || ctx?.data_stream.dataset == 'carbon_black.vulnerabilities')" } } ] } ``` Just add a gsub processor to convert the `,` to `.` The better way would be to add another pipeline called from this pipeline that does any cleanup you need to do... You could also add it to the mitre\_attack\_pipeline ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 18, 2025, 7:22pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/4 "2025-02-18T19:22:05Z")
something like this ```auto POST _ingest/pipeline/_simulate { "pipeline": { "processors": [ { "gsub": { "field": "netconn_ipv4", "pattern": ",", "replacement": "." } }, { "gsub": { "field": "netconn_local_ipv4", "pattern": ",", "replacement": "." } } ] }, "docs": [ { "_source": { "netconn_ipv4": "178,258,031", "netconn_local_ipv4": "178,258,037" } } ] } # result { "docs": [ { "doc": { "_index": "_index", "_version": "-3", "_id": "_id", "_source": { "netconn_ipv4": "178.258.031", "netconn_local_ipv4": "178.258.037" }, "_ingest": { "timestamp": "2025-02-18T19:21:20.182598305Z" } } } ] } ``` ---
**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\ **Post date:** [February 18, 2025, 9:28pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/5 "2025-02-18T21:28:11Z")
Hi Stephen, Ok so something like this will work fine? ```auto PUT _ingest/pipeline/convert_to_ip { "description": "Convert comma-separated values to IP addresses", "processors": [ { "gsub": { "field": "netconn_ipv4", "pattern": ",", "replacement": "." } }, { "gsub": { "field": "netconn_local_ipv4", "pattern": ",", "replacement": "." } } ] } PUT /logs-carbon_black.observations-default-2025.02.09-000002/_settings { "index.default_pipeline": "convert_to_ip" } ´´´ ``` ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [February 18, 2025, 9:41pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/6 "2025-02-18T21:41:24Z")
> [@arcsons](#): > > Ok so something like this will work fine? No, this can break the integraiton. You cannot change the `index.default_pipeline` nor the `index.final_pipeline` for any Elastic Agent integration. All customizations needs to be done in the `@custom` ingest pipeline for the specific dataset of the integration or globally in the `logs@custom` ingest pipeline. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 18, 2025, 10:11pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/7 "2025-02-18T22:11:51Z")
@arcsons You already created a `logs@custom` when We work together on the enrichment. You can just call the `convert_to_ip` from the `logs@custom` Just like you did the miter enrichment > [@Problems with enrichments](https://discuss.elastic.co/t/problems-with-enrichments/374215/29): > > Assuming you have the above working this is how we will call your enrich pipeline We will follow the new framework [here](https://www.elastic.co/guide/en/fleet/current/data-streams.html#data-streams-pipelines) It refers to elastic agent but works with the data streams we created. Your Existing enrich pipeline PUT \_ingest/pipeline/mitre\_attack\_pipeline { "description": "Pipeline to enrich MITRE ATT&CK fields", "processors": [ { "set": { "field": "pipeline\_name", "value": "mitre\_attack\_pipeline" } }, { "enrich": { "policy\_nam… BTW @leandrojmp There is Not in integration for carbon black observations but I had him follow the framework. ---
**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\ **Post date:** [February 18, 2025, 10:22pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/8 "2025-02-18T22:22:02Z")
Ok thanks. I will test that tomorrow morning. ---
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [February 19, 2025, 12:47am UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/9 "2025-02-19T00:47:24Z")
Out of curiosity, every day is a school day, why does the tool generate IPs like network\_ipv4: 178,258,031 netconn\_local\_ipv4: 178,258,037 I mean that is a pretty unusual “notation”, right. Is there some significance? Is 78,258,031 a shorthand for what is more usually written as 78.258.31.0/24? It’s a convention in a particular country/language/field ? ---
**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\ **Post date:** [February 19, 2025, 3:15pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/10 "2025-02-19T15:15:13Z")
Hi Rain, Yeah you are right. I maybe should fix this directly in my code where I do my API call with python. The output was actually in hexa, so Elastic though it was int. So this code fixed it. ``` if 'netconn_ipv4' in json_data: json_data['netconn_ipv4'] = int_to_ip(json_data['netconn_ipv4']) if 'netconn_local_ipv4' in json_data: json_data['netconn_local_ipv4'] = int_to_ip(json_data['netconn_local_ipv4']) ``` Thanks ---
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [February 19, 2025, 4:21pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/11 "2025-02-19T16:21:10Z")
Good that you fixed it, one way or another. Personally I think the code fix was better way, but I also learned a bit about logs@custom too. If that closes the topic, please accept one of the answers and good lick with your project. ---
**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\ **Post date:** [February 19, 2025, 7:36pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/12 "2025-02-19T19:36:05Z")
Good evening, New problems. ![fields](https://us1.discourse-cdn.com/elastic/original/3X/f/d/fd0e7354c7f8ad963c28074d33421afd4d1ffe4a.jpeg) I will post my idea which I think will solve it, give me few min. And Stephen maybe can verify it 🙂 Here is what I think will solve it: ```auto 1. GET log-carbon_black.observations-default-2025.02.09-000001/_mapping 2. Change this "netconn_ipv4": { "type": "long" }, "netconn_local_ipv4": { "type": "long" }, PUT /log-carbon_black.observations-default-2025.02.09-000002 { "mappings": { "properties": { "netconn_ipv4": { "type": "ip" }, "netconn_local_ipv4": { "type": "ip" }, ### AND ALL OTHER ### } } } 3. POST /_reindex { "source": { "index": "log-carbon_black.observations-default-2025.02.09-000001" }, "dest": { "index": "log-carbon_black.observations-default-2025.02.09-000002" } } 4. DELETE /log-carbon_black.observations-default-2025.02.09-000001 ´´´ ``` ---
**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\ **Post date:** [February 21, 2025, 2:23pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/13 "2025-02-21T14:23:24Z")
Im close now. I got it in JSON working in Elastic but the fields are strang. See the image below. ![issue](https://us1.discourse-cdn.com/elastic/original/3X/9/7/9789adc925c6ef723e8f452ab41bbdd7ae0f8458.jpeg) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 21, 2025, 2:32pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/14 "2025-02-21T14:32:05Z")
As you can see type is number not IP So you have still not set the mapping correct And or there is a conflict in the data view? Because you have two indices with two different types of mappings this can cause these kind of issues ![1000005010](https://us1.discourse-cdn.com/elastic/original/3X/a/c/ac503b7a4af99bff3d6ae193190d522b9a7463b2.jpeg) So please back up why are you trying to reindex the data? I don't understand what current state you're in, so perhaps backup and tell us where you're at and what you're trying to accomplish. I understand you're trying to get the IP but I don't know what you're doing and why If you have two indices backing indices and they have a single data view that will be a conflict and cause issues like this. You should probably really clean up the old data. Create a new custom mapping for that field. Much like the ingest pipeline > **[Data streams | Fleet and Elastic Agent Guide \[8.17\] | Elastic](https://www.elastic.co/guide/en/fleet/current/data-streams.html#data-streams-index-templates-edit)** Add a logs@custon component template with the correct mapping > **[Data streams | Fleet and Elastic Agent Guide \[8.17\] | Elastic](https://www.elastic.co/guide/en/fleet/current/data-streams.html#data-streams-index-templates-edit)** Simply ingest new data with your fixed mapping ---
**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\ **Post date:** [February 21, 2025, 2:47pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/15 "2025-02-21T14:47:30Z")
When we were working together, the script `cp.py` produced output from two fields (`netconn_ipv4` and `netconn_local_ipv4`) in hexadecimal format. In Elasticsearch, these fields were interpreted as integers. I modified `cp.py` to convert the hexadecimal values to IP addresses. I haven't changed the pipelines, and I don't need to re-index since it's not in production yet. Now, I simply want these fields to be recognized as IP addresses and nothing else. Custom logs and parsing in Elasticsearch can be challenging, and it seems I'm not alone in this struggle if you search online. I apologize if my explanation isn't clear enough. Ok I will try your suggestions, thanks. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 21, 2025, 3:20pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/16 "2025-02-21T15:20:54Z")
> [@arcsons](#): > > Now, I simply want these fields to be recognized as IP addresses and nothing else. Ok that helps. Remember, we are answering many questions... following each topic "Train of thought" can be challenging > [@arcsons](#): > > Custom logs and parsing in Elasticsearch can be challenging, and it seems I'm not alone in this struggle if you search online. Yes, it can... and there is a framework that can take a little getting used to ... and making assumption about how it works without reading the docs can sometimes make it worse, or using some other article etc... > [@arcsons](#): > > I haven't changed the pipelines, and I don't need to re-index since it's not in production yet. Ok good delete the existing data stream... That will clean everything up. Then start your data flow again... and report back. The result of this will give us 1 of 2 path to follow. 1. It will work because you no longer have conflicting mappings in the data view (pretty sure this is the case) 2. We will add a custom mapping for those 2 fields (which is simple I will show you) and then we will clean up and start again. For # 2 adding the custom mapping is simply this but test first and come back. This will force the 2 fields to be of the correct type ```auto PUT _component_template/logs@custom { "template": { "mappings": { "properties": { "netconn_ipv4": { "type": "ip" }, "netconn_local_ipv4": { "type": "ip" } } } } } ``` ---
**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\ **Post date:** [February 26, 2025, 3:06pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/18 "2025-02-26T15:06:57Z")
Hey Stephen, Everything works fine, thanks man 🙂 Case closed. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 26, 2025, 6:00pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/19 "2025-02-26T18:00:40Z")
@arcsons Excellent! Could help others if you could provide a couple of bullets on your end solutions, then we will marked as solved ... that will help others in the community. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 26, 2025, 6:01pm UTC](https://discuss.elastic.co/t/convert-ip-from-integears/374687/20 "2025-03-26T18:01:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.