# Shodan.io query return json mapping, nested dynamic mapping

**URL:** <https://discuss.elastic.co/t/shodan-io-query-return-json-mapping-nested-dynamic-mapping/325761>\
**Category:** Elasticsearch\
**Created:** [February 16, 2023, 4:23pm UTC](https://discuss.elastic.co/t/shodan-io-query-return-json-mapping-nested-dynamic-mapping/325761 "2023-02-16T16:23:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![stcdarrell](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@stcdarrell](https://discuss.elastic.co/u/stcdarrell)\
**Post date:** [February 16, 2023, 4:23pm UTC](https://discuss.elastic.co/t/shodan-io-query-return-json-mapping-nested-dynamic-mapping/325761/1 "2023-02-16T16:23:46Z")

</div>

## hi, i'm using some python to query [shodan.io](http://shodan.io), it returns a reasonably complex json that i'd like to push into Elasticsearch. i've got most mapped out and its work, but there is one field i just cant to map correctly.

the rough format:

```auto
{
... #bunch of fields i have mapped

'vulns': {
    'CVE-2022-01-02' : {
          'verified' : false,
         'references': [<<bunch of web links>>],
        'summary': <<text>>
   },
    'CVE-2021-02-12' : {
          'verified' : false,
         'references': [<<bunch of web links>>],
        'summary': <<text>>
   }
    'CVE-2019-04-11' : {
          'verified' : false,
         'references': [<<bunch of web links>>],
        'summary': <<text>>
   }
}
...
}

```

* * *

the CVE field name changes.. so that should be a dynamic field? is this the right approach?  
how do i fit the verified, references, and summary fields in this?

```auto
[
  {
    "cve-objects": {
      "mapping": {
        "include_in_parent": true,
        "type": "nested"
      },
      "match_mapping_type": "object",
      "match": "CVE-*"
    }
  }
]

```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 16, 2023, 8:12pm UTC](https://discuss.elastic.co/t/shodan-io-query-return-json-mapping-nested-dynamic-mapping/325761/2 "2023-02-16T20:12:10Z")

</div>

Having keys that are created dynamically is not recommended in Elasticsearch as it can lead to [mapping explosion](https://www.elastic.co/guide/en/elasticsearch/reference/8.6/mapping.html#mapping-limit-settings).

I would recommend restructuring the document as follows instead to avoid this:

```auto
'vulns': [
  {
    'cve_id': 'CVE-2022-01-02',
    'verified' : false,
    'references': [<<bunch of web links>>],
    'summary': <<text>>
  },
  {
    'cve_id': 'CVE-2021-02-12',
    'verified' : false,
    'references': [<<bunch of web links>>],
    'summary': <<text>>
  },
  {
    'cve_id': 'CVE-2019-04-11',
    'verified' : false,
    'references': [<<bunch of web links>>],
    'summary': <<text>>
  }
]

```

---

<div class="post-metadata">

**Author:** ![stcdarrell](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@stcdarrell](https://discuss.elastic.co/u/stcdarrell)\
**Post date:** [February 17, 2023, 6:20pm UTC](https://discuss.elastic.co/t/shodan-io-query-return-json-mapping-nested-dynamic-mapping/325761/3 "2023-02-17T18:20:03Z")

</div>

thats perfect, but how would i map that? thats what i havent figured out?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 17, 2023, 8:52pm UTC](https://discuss.elastic.co/t/shodan-io-query-return-json-mapping-nested-dynamic-mapping/325761/4 "2023-02-17T20:52:13Z")

</div>

`vulns` would be mapped as a nested field and the fields in the subdocuments mapped based on the content and how you will query them.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 17, 2023, 8:52pm UTC](https://discuss.elastic.co/t/shodan-io-query-return-json-mapping-nested-dynamic-mapping/325761/5 "2023-03-17T20:52:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.