Pushing pcap json into Kibana using BULK API

Elastic Stack Elasticsearch
1

I am trying to create index, for pcap files.

here is the request in Dev Tools console:
POST /pcaptest/_doc
{"mappings":
{"pcap_file":{
"properties":{
"dynamic": "false",
"index" : {"type":"nested"},
"timestamp":{"type":"date"},
"layers":{
"properties":{
"tcp":{
"properties":{
"tcp_tcp_srcport":{"type":"integer"},
"tcp_tcp_dstport":{"type":"integer"}
}}}}}}}}

it works fine without bulk, my curl command is as follows:
curl -s -H "Content-Type: application/json" -XPOST http://x.x.x.x:9200/pcaptest/_doc --data-binary "@page1.json

Can anybody help me with bulk?

  1. what will be the curl request? and
  2. what will the POST request?
    I want to have doc as I eventually want to send 1 packet per doc.
    I tried
    POST /pcaptest/_doc/_bulk
    it's giving me "error": {
    "root_cause": [
    {
    "type": "json_e_o_f_exception",
    "reason": "Unexpected end-of-input within/between Object entries\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@38a3acfa; line: 1, column: 25]"
    }
    I searched existing topics , but could not figure out exact format for POST and curl request

For above request my data json was single valid json block :
{"index":{"_index":"packets-20191119","_type":"pcap_file"},"timestamp":"1574163762676","layers":{"frame":{"frame_frame_interface_id":"0"}}}

but now I want to send packet json data as follows, which is not a valid json:

{"index":{"_index":"packets-2019-11-21","type":"pcap_file"}}
{"timestamp":"1574330692870","layers":{"frame":{"frame_frame_interface_id":"0","frame_interface_id_frame_interface_name":"\Device\NPF{C3664358-1A0C-4DB0-A6DB-66A9168C645E}"}}}

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.