# Sticking same top-level fields **URL:** \ **Category:** Logstash\ **Created:** [October 6, 2017, 6:27am UTC](https://discuss.elastic.co/t/sticking-same-top-level-fields/102945 "2017-10-06T06:27:15Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Valsek](https://avatars.discourse-cdn.com/v4/letter/v/ebca7d/32.png) [@Valsek](https://discuss.elastic.co/u/Valsek)\ **Post date:** [October 6, 2017, 6:27am UTC](https://discuss.elastic.co/t/sticking-same-top-level-fields/102945/1 "2017-10-06T06:27:16Z")
Hello. I've next json generated by tshark and exported with flag -T ek: { "timestamp": "1507110658491", "layers": { "frame": { "frame\_frame\_interface\_id": "0", "frame\_frame\_encap\_type": "1", "frame\_frame\_time": "Oct 4, 2017 09:50:58.491647000 UTC", "frame\_frame\_offset\_shift": "0.000000000", "frame\_frame\_time\_epoch": "1507110658.491647000", "frame\_frame\_time\_delta": "0.000034000", "frame\_frame\_time\_delta\_displayed": "0.000034000", "frame\_frame\_time\_relative": "16.587719000", "frame\_frame\_number": "994", "frame\_frame\_len": "1414", "frame\_frame\_cap\_len": "1414", "frame\_frame\_marked": "0", "frame\_frame\_ignored": "0", "frame\_frame\_protocols": "eth:ethertype:ip:tcp:http:ssl:http:ssl" } , "eth": { "eth\_eth\_dst": "90:b1:1c:a1:7f:18", "eth\_dst\_eth\_dst\_resolved": "Dell\_a1:7f:18", "eth\_dst\_eth\_addr": "90:b1:1c:a1:7f:18", "eth\_dst\_eth\_addr\_resolved": "Dell\_a1:7f:18", "eth\_dst\_eth\_lg": "0", "eth\_dst\_eth\_ig": "0", "eth\_eth\_src": "30:f7:0d:b8:6c:7f", "eth\_src\_eth\_src\_resolved": "Cisco\_b8:6c:7f", "eth\_src\_eth\_addr": "30:f7:0d:b8:6c:7f", "eth\_src\_eth\_addr\_resolved": "Cisco\_b8:6c:7f", "eth\_src\_eth\_lg": "0", "eth\_src\_eth\_ig": "0", "eth\_eth\_type": "0x00000800" } , "ip": { "ip\_ip\_version": "4", "ip\_ip\_hdr\_len": "20", "ip\_ip\_dsfield": "0x00000000", "ip\_dsfield\_ip\_dsfield\_dscp": "0", "ip\_dsfield\_ip\_dsfield\_ecn": "0", "ip\_ip\_len": "1400", "ip\_ip\_id": "0x00006b18", "ip\_ip\_flags": "0x00000000", "ip\_flags\_ip\_flags\_rb": "0", "ip\_flags\_ip\_flags\_df": "0", "ip\_flags\_ip\_flags\_mf": "0", "ip\_ip\_frag\_offset": "0", "ip\_ip\_ttl": "62", "ip\_ip\_proto": "6", "ip\_ip\_checksum": "0x000065ed", "ip\_ip\_checksum\_status": "2", "ip\_ip\_src": "10.64.48.8", "ip\_ip\_addr": "10.64.48.8", "ip\_ip\_src\_host": "10.64.48.8", "ip\_ip\_host": "10.64.48.8", "ip\_ip\_dst": "10.16.98.35", "ip\_ip\_addr": "10.16.98.35", "ip\_ip\_dst\_host": "10.16.98.35", "ip\_ip\_host": "10.16.98.35", "ip\_text": "Source GeoIP: Unknown", "ip\_text": "Destination GeoIP: Unknown" } , "tcp": { "tcp\_tcp\_srcport": "8080", "tcp\_tcp\_dstport": "49658", "tcp\_tcp\_port": "8080", "tcp\_tcp\_port": "49658", "tcp\_tcp\_stream": "42", "tcp\_tcp\_len": "1360", "tcp\_tcp\_seq": "30867", "tcp\_tcp\_nxtseq": "32227", "tcp\_tcp\_ack": "2603", "tcp\_tcp\_hdr\_len": "20", "tcp\_tcp\_flags": "0x00000010", "tcp\_flags\_tcp\_flags\_res": "0", "tcp\_flags\_tcp\_flags\_ns": "0", "tcp\_flags\_tcp\_flags\_cwr": "0", "tcp\_flags\_tcp\_flags\_ecn": "0", "tcp\_flags\_tcp\_flags\_urg": "0", "tcp\_flags\_tcp\_flags\_ack": "1", "tcp\_flags\_tcp\_flags\_push": "0", "tcp\_flags\_tcp\_flags\_reset": "0", "tcp\_flags\_tcp\_flags\_syn": "0", "tcp\_flags\_tcp\_flags\_fin": "0", "tcp\_flags\_tcp\_flags\_str": "\u00c2\u00b7\u00c2\u00b7\u00c2\u00b7\u00c2\u00b7\u00c2\u00b7\u00c2\u00b7\u00c2\u00b7A\u00c2\u00b7\u00c2\u00b7\u00c2\u00b7\u00c2\u00b7", "tcp\_tcp\_window\_size\_value": "65535", "tcp\_tcp\_window\_size": "65535", "tcp\_tcp\_window\_size\_scalefactor": "1", "tcp\_tcp\_checksum": "0x00000a44", "tcp\_tcp\_checksum\_status": "2", "tcp\_tcp\_urgent\_pointer": "0", "tcp\_tcp\_analysis": "", "tcp\_analysis\_tcp\_analysis\_initial\_rtt": "0.000300000", "tcp\_analysis\_tcp\_analysis\_bytes\_in\_flight": "1360", "tcp\_analysis\_tcp\_analysis\_push\_bytes\_sent": "5440" } , "tcp\_segments": "", "tcp\_segments\_tcp\_segment": "989", "tcp\_segments\_tcp\_segment": "991", "tcp\_segments\_tcp\_segment": "992", "tcp\_segments\_tcp\_segment": "994", "tcp\_segments\_tcp\_segment\_count": "4", "tcp\_segments\_tcp\_reassembled\_length": "4077", "http": { "http\_http\_proxy\_connect\_host": "62.102.148.67", "http\_http\_proxy\_connect\_port": "443" } , "ssl": { "ssl\_ssl\_record": "", "ssl\_record\_ssl\_record\_content\_type": "23", "ssl\_record\_ssl\_record\_version": "0x00000303", "ssl\_record\_ssl\_record\_length": "4072" } , "http": { "http\_http\_proxy\_connect\_host": "62.102.148.67", "http\_http\_proxy\_connect\_port": "443" } , "ssl": "Secure Sockets Layer" } } I'm making subfields as topfields with the mutate plugin, for example: mutate { rename =\> { "[layers][ssl][ssl\_record\_ssl\_record\_content\_type]" =\> "ssl\_record\_ssl\_record\_content\_type" } } but the problem is that i have 2 top lvl fields named 'ssl' (and that's not the only field like this). So when i'm sending this data to elastic it writes only last field - "ssl": "Secure Sockets Layer" and other data is lost. How can i stick this top level fields (ssl for example)? I know that i can use ruby filter for that, but can't find any examples of sticking same named topfields. Thank you. ---
**Author:** ![jakelandis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakelandis/32/36163_2.png) [@jakelandis](https://discuss.elastic.co/u/jakelandis)\ **Post date:** [October 6, 2017, 2:38pm UTC](https://discuss.elastic.co/t/sticking-same-top-level-fields/102945/2 "2017-10-06T14:38:09Z")
I think this maybe what you are looking for ? ```auto grok { match => { "[layers][ssl][ssl_record_ssl_record_content_type]" => "%{NUMBER:choose_your_name}$" } } ``` ---
**Author:** ![jakelandis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakelandis/32/36163_2.png) [@jakelandis](https://discuss.elastic.co/u/jakelandis)\ **Post date:** [October 6, 2017, 2:54pm UTC](https://discuss.elastic.co/t/sticking-same-top-level-fields/102945/3 "2017-10-06T14:54:32Z")
I think I was too quick to answer... is the real problem here that the duplicated JSON keys not showing up in the event at all ? ---
**Author:** ![jakelandis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakelandis/32/36163_2.png) [@jakelandis](https://discuss.elastic.co/u/jakelandis)\ **Post date:** [October 6, 2017, 3:15pm UTC](https://discuss.elastic.co/t/sticking-same-top-level-fields/102945/4 "2017-10-06T15:15:09Z")
It seems that duplicate key names in JSON is arguably within the specification, but that most libraries (the one Logstash uses included) doesn't support duplicated keys. You may need a way to prevent the duplicate keys from the source or do some String based manipulation of the JSON (yuk) prior to treating as JSON. I should also mention that [https://www.elastic.co/products/beats/packetbeat](https://www.elastic.co/products/beats/packetbeat) -\> Logstash (beats input) may be a better solution here. ---
**Author:** ![Valsek](https://avatars.discourse-cdn.com/v4/letter/v/ebca7d/32.png) [@Valsek](https://discuss.elastic.co/u/Valsek)\ **Post date:** [October 6, 2017, 7:53pm UTC](https://discuss.elastic.co/t/sticking-same-top-level-fields/102945/5 "2017-10-06T19:53:52Z")
Thank you for reply. I use filebeat, cause writing logs to .json with tshark. Yes the problem was with duplicate keys and i found a solution. The reason of duplicated keys was a bug in wireshark log jsoned by tshark -T ek. Also packetbeat doesn't support all the protocols i need. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 3, 2017, 7:54pm UTC](https://discuss.elastic.co/t/sticking-same-top-level-fields/102945/6 "2017-11-03T19:54:07Z")
Closed on November 3, 2017, 7:54pm UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.