# General Question : Best method for handling JSON files **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 23, 2020, 2:53pm UTC](https://discuss.elastic.co/t/general-question-best-method-for-handling-json-files/224687 "2020-03-23T14:53:00Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![zebulyon](https://avatars.discourse-cdn.com/v4/letter/z/58f4c7/32.png) [@zebulyon](https://discuss.elastic.co/u/zebulyon)\ **Post date:** [March 23, 2020, 2:53pm UTC](https://discuss.elastic.co/t/general-question-best-method-for-handling-json-files/224687/1 "2020-03-23T14:53:00Z")
Hello, So I have a collection of PCAP files and I want to upload their content to Kibana. Now I am fairly certain that I will need to use tshark to convert the files to JSON, which I have done. Though I have a question regarding the conversion process, when converting the pcap files it becomes a multiline format this is the command I use: ` tshark -r FILEPATH_TO_PCACP -Tjson > output.json` This is a problem as it seem that Filebeat is only able to handle single line JSON. But now the big question, how should I go about sending this information to Kibana. Should I go Filebeat -\> Elasticsearch -\> Kibana or Filebeat -\> Logstash -\> Elasticsearch -\> Kibana I have been testing both methods and I do not get it work properly. The problem seem to be when the information is sent to Elasticsearch. Because I can see the JSON file being sent properly to Logstash from Filebeat. But when it arraives at Kibana I only get a single Entry saying that ``` Error decoding JSON: unexpected EOF ``` Even though as said I can see that Logstash received multiple JSON events. And when I go Filebeat -\> Elasticsearch -\> Kibana I get another weird error among them being that the index is becomes `ilm-history-1-000001` Even when I set it manually in filebeat. And secondly the bigger problem is that none of the JSON events arrives at Kibana instead I just get these weird event that that are almost empty. ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [March 23, 2020, 6:53pm UTC](https://discuss.elastic.co/t/general-question-best-method-for-handling-json-files/224687/2 "2020-03-23T18:53:27Z")
> [@zebulyon](#): > > `tshark -r FILEPATH_TO_PCACP -Tjson > output.json` > This is a problem as it seem that Filebeat is only able to handle single line JSON. I'm not familar with `tshark` so you might have to help me out with that portion. When you pipe it's output to `output.json`, does `output.json` contain pretty-printed JSON? Perhaps posting a sample of that file here might help (or if you want to post it to [pastebin.com](http://pastebin.com) or [gist.github.com](http://gist.github.com) first and post a link here, that's fine too). Thanks, Shaunak ---
**Author:** ![zebulyon](https://avatars.discourse-cdn.com/v4/letter/z/58f4c7/32.png) [@zebulyon](https://discuss.elastic.co/u/zebulyon)\ **Post date:** [March 24, 2020, 8:26am UTC](https://discuss.elastic.co/t/general-question-best-method-for-handling-json-files/224687/3 "2020-03-24T08:26:57Z")
Here is one json entry all of the other ones follow a very similar formatting. ``` [ { "_index": "packets-2020-02-06", "_type": "pcap_file", "_score": null, "_source": { "layers": { "frame": { "frame.encap_type": "25", "frame.time": "Dec 2, 2019 07:37:53.050476000 CET", "frame.offset_shift": "0.000000000", "frame.time_epoch": "1575268673.050476000", "frame.time_delta": "0.000000000", "frame.time_delta_displayed": "0.000000000", "frame.time_relative": "0.000000000", "frame.number": "1", "frame.len": "360", "frame.cap_len": "360", "frame.marked": "0", "frame.ignored": "0", "frame.protocols": "sll:ethertype:ip:sctp:data" }, "sll": { "sll.pkttype": "0", "sll.hatype": "1", "sll.halen": "6", "sll.src.eth": "fa:16:3e:e6:39:ec", "sll.unused": "00:00", "sll.etype": "0x00000800" }, "ip": { "ip.version": "4", "ip.hdr_len": "20", "ip.dsfield": "0x00000000", "ip.dsfield_tree": { "ip.dsfield.dscp": "0", "ip.dsfield.ecn": "0" }, "ip.len": "344", "ip.id": "0x00000000", "ip.flags": "0x00004000", "ip.flags_tree": { "ip.flags.rb": "0", "ip.flags.df": "1", "ip.flags.mf": "0", "ip.frag_offset": "0" }, "ip.ttl": "64", "ip.proto": "132", "ip.checksum": "0x0000b5f8", "ip.checksum.status": "2", "ip.src": "172.2.21.149", "ip.addr": "172.2.21.149", "ip.src_host": "172.2.21.149", "ip.host": "172.2.21.149", "ip.dst": "172.2.21.144", "ip.addr": "172.2.21.144", "ip.dst_host": "172.2.21.144", "ip.host": "172.2.21.144" }, "sctp": { "sctp.srcport": "3906", "sctp.dstport": "3906", "sctp.verification_tag": "0x04a0a9d5", "sctp.assoc_index": "0", "sctp.port": "3906", "sctp.port": "3906", "sctp.checksum": "0xfffeb617", "sctp.checksum.status": "2", "DATA chunk(ordered, complete segment, TSN: 45809598, SID: 13, SSN: 33049, PPID: 0, payload length: 296 bytes)": { "sctp.chunk_type": "0", "sctp.chunk_type_tree": { "sctp.chunk_bit_1": "0", "sctp.chunk_bit_2": "0" }, "sctp.chunk_flags": "0x00000003", "sctp.chunk_flags_tree": { "sctp.data_e_bit": "1", "sctp.data_b_bit": "1", "sctp.data_u_bit": "0", "sctp.data_i_bit": "0" }, "sctp.chunk_length": "312", "sctp.data_tsn": "45809598", "sctp.data_sid": "0x0000000d", "sctp.data_ssn": "33049", "sctp.data_payload_proto_id": "0" } }, "data": { "data.data": "01:00:01:28:80:00:01:10:01:00:00:33:01:1b:05:fa:00:06:70:ed:00:00:01:07:40:00:00:2c:4f:67:50:72:48:65:49:4d:43:4c:34:43:69:6a:75:3b:33:36:36:37:37:30:3b:33:35:31:37:39:30:32:36:36:30:3b:34:33:00:00:01:02:40:00:00:0c:01:00:00:33:00:00:01:a0:40:00:00:0c:00:00:00:01:00:00:01:08:40:00:00:13:72:6e:39:2e:6d:6d:65:2e:63:6f:6d:00:00:00:01:28:40:00:00:0f:6d:6d:65:2e:63:6f:6d:00:00:00:01:1b:40:00:00:0f:68:73:73:2e:63:6f:6d:00:00:00:01:9f:40:00:00:0c:00:00:00:01:00:00:01:bb:40:00:00:2c:00:00:01:c2:40:00:00:0c:00:00:00:00:00:00:01:bc:40:00:00:17:34:37:39:35:33:36:32:37:33:31:36:31:32:38:39:00:00:00:00:08:40:00:00:17:6b:73:2e:72:64:2e:74:69:65:74:6f:2e:63:6f:6d:00:00:00:01:cd:40:00:00:17:65:75:2e:76:69:73:69:74:69:6e:67:2e:63:6f:6d:00:00:00:01:c8:40:00:00:34:00:00:01:b0:40:00:00:0c:00:00:00:01:00:00:01:b7:40:00:00:0c:00:00:00:01:00:00:01:b5:40:00:00:14:00:00:01:a4:40:00:00:0c:00:00:00:01", "data.len": "296" } } } } ] ``` So this is the output from tshark ---
**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\ **Post date:** [March 24, 2020, 3:36pm UTC](https://discuss.elastic.co/t/general-question-best-method-for-handling-json-files/224687/4 "2020-03-24T15:36:38Z")
You will have to do something about that 🙂 First, it looks like the output in an _array_ (based on the `[]`, start and end) You should be able to pass everything through `jq` Something like this should do the trick ``` cat tshark_out.json | jq -c .[] > compact_json_per_line.json ``` That should give you something you can feed into Filebeat ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 21, 2020, 3:50pm UTC](https://discuss.elastic.co/t/general-question-best-method-for-handling-json-files/224687/5 "2020-04-21T15:50:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.