Wireshark json file cannot display in separate fields in Elasticsearch

Elastic Stack Elasticsearch
24

I just do a research in the Internet and found out that I can modify the Tshark command to output the result:

.\tshark.exe -i rpcap://x.x.x.x:6666/cali18dd3a4d0a8 -T json -E header=y -E separator=/t -a duration:10 -e frame.time -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport -e tcp.payload > C:\wireshark\20211217_packets_137.json

However, now the json file become like that, that is exactly what I need:

[
  {
    "_index": "packets-2021-12-17",
    "_type": "doc",
    "_score": null,
    "_source": {
      "layers": {
        "frame.time": [
          "Dec 17, 2021 17:39:18.491701000 China Standard Time"
        ],
        "ip.src": [
          "10.200.25.98"
        ],
        "ip.dst": [
          "10.2.0.1"
        ],
        "tcp.srcport": [
          "51344"
        ],
        "tcp.dstport": [
          "443"
        ],
        "tcp.payload": [
          "170303002900000000006f277f3404657b2b8437c7d65021dd854cbfc32a30b315a5dfea728cc12962f81740719b"
        ]
      }
    }
  },
  {
    "_index": "packets-2021-12-17",
    "_type": "doc",
    "_score": null,
    "_source": {
      "layers": {
        "frame.time": [
          "Dec 17, 2021 17:39:18.495052000 China Standard Time"
        ],
        "ip.src": [
          "10.2.0.1"
        ],
        "ip.dst": [
          "10.200.25.98"
        ],
        "tcp.srcport": [
          "443"
        ],
        "tcp.dstport": [
          "51344"
        ],
        "tcp.payload": [
          "170303003d0000000000a6eb66340bc56ebfd2cfbf21a7ed5ea076e0c409dd23472712160630fe8313561318be39afe353b5dcd2b15b5b5bad3f4f3d4e3f90b35768"
        ]
      }
    }
  }
]

I was thinking how can I modify the configuration to deal with this. There may still the problem with UTF-8 here.

25

Did you just try setting the encoding setting I showed you instead of trying a whole different method?

- type: log
  enabled: true
  paths:
    - 'C:\wireshark\*.json'
  json.keys_under_root: true
  json.add_error_key: true
  encoding : utf-8
26

Per the Wireshark docs I would just try the utf-8 setting I showed above...

OUTPUT

TShark uses UTF-8 to represent strings internally. In some cases the output might not be valid. For example, a dissector might generate invalid UTF-8 character sequences. Programs reading TShark output should expect UTF-8 and be prepared for invalid output.

If TShark detects that it is writing to a TTY on UNIX or Linux and the locale does not support UTF-8, output will be re-encoded to match the current locale.

If TShark detects that it is writing to the console on Windows, dissection output will be encoded as UTF-16LE. Other output will be UTF-8. If extended characters don’t display properly in your terminal you might try setting your console code page to UTF-8 ( chcp 65001 ) and using a modern terminal application if possible.

27

I installed Wireshark
Ran

tshark -i en7 -T ek -a duration:10 > wireshark-test-utf.json

And it ingested fine... I have Mac so perhaps the encoding is a little different, if you just get the right encoding I think you will be fine.

If you open your json in something like VSCode it will tell you the encoding in the bottom right.

28

Hello stephen,

Would you mind show me your result in screen capture? I would actually confirm if my expected result is right or not. Thank you.

29

Apologies, I'm not quite sure what you're asking.

Do you want to see what the raw data from Wireshark looks like?

Do you want to see what it looks like after it's ingested in Kibana

I will state. I don't believe I have the encoding issue that you have because I have a simple Mac with US English code set

30

Sorry for misunderstanding, I would like to have a look after the data ingested in Kibana.

31

Hey!! I got it!! It is done!!

image
image1603×1005 167 KB

Thank you, man!!

You save my time

1 Like
32

@lauyatkin Awesome! Good to hear

But Okay, here's the help we need. When you solve something like this, please put what solved it in this thread so others with the same issue know what fixed it

33

image
image2524×1281 404 KB

34

Yes, I will put a summary on this question.

Thanks a lot for the support. I can have my Christmas holiday.

1 Like
35

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.