Tshark output support

Elastic Stack Elasticsearch
1

Which versions of Tshark json for ES are supported? Looks like old version (v2) is working fine when in new (v3) output didn't get parsed in latest ES (accually why, because it have no sense because this output is dedicated).

2

Hey,

can you clarify your setup? Are you using packetbeat together with tshark? If so, which version?

As Elasticsearch only understands HTTP, you need to explain your tooling around that to prevent confusion and possibly giving wrong hints.

Thanks!

--Alex

3

I am using tshark with logstash, as in article https://www.elastic.co/blog/analyzing-network-packets-with-wireshark-elasticsearch-and-kibana it works fine with old tshark version, but not with new one.

4

can you share an exception? And also maybe a few lines of the json dump that tshark has produced?

5

When I try upload mapping from tshark3 have error:

{"error":{"root_cause":[{"type":"parse_exception","reason":"Failed to parse content to map"}],"type":"parse_exception","reason":"Failed to parse content to map","caused_by":{"type":"json_parse_exception","reason":"Duplicate field 'dhcp_dhcp_option_value_uint'\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@4c5d35f8; line: 1, column: 14708]"}},"status":400}

and in mapping json I have 3 fields dhcp_dhcp_option_value_uint

6

You may want to read about coercing

7

Can I somehow just update selected field to be IP? Looks like coercing have option like this but I can't find any example to exacly do this. Also when I have my field name, how use that in such query? What I mean I don't know how it is recessed in json etc - I just know my field name from Kibana.

Also I need setup that on index pattern, because my indexes are generated every day (have date in name). All is generated automatic from reading jsons from tshark. I just want fix IPv4 field (I know name from Kibana) from string to IP, that is all :).

8

Additionaly tshark mapping isn't good, because when I want filter by some protocols, it just dosent always works.

For example there is mdns in wireshark as filter, there is protocol with such name, there are fields in kibana from tshark etc... but when I do:

tshark -G elastic-mapping --elastic-mapping-filter mdns

{
  "index_patterns": "packets-*",
  "settings": {
    "index.mapping.total_fields.limit": 1000000
  },
  "mappings": {
    "doc": {
      "dynamic": false,
      "properties": {
        "timestamp": {
          "type": "date"
        },
        "layers": {
          "properties": {}
        }
      }
    }
  }
}

Just tshark don't have mapping for that protocol, btw it accept anything in --elastic-mapping-filter so I really don't know if name is wrong or just there is no info etc.

--elastic-mapping-filter cyberzlo

$ tshark -G elastic-mapping --elastic-mapping-filter cyberzlo
{
  "index_patterns": "packets-*",
  "settings": {
    "index.mapping.total_fields.limit": 1000000
  },
  "mappings": {
    "doc": {
      "dynamic": false,
      "properties": {
        "timestamp": {
          "type": "date"
        },
        "layers": {
          "properties": {}
        }
      }
    }
  }
}

But when logstash put pcap in elastic format in elasticsearch, there are many fields - so just mapping export dont work, because packets have many fields…

So most easy would be update selected fields by names and just parse tchem as IP, like some filter to logstash, which import it and change index name etc. Is this possible?

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.