I'm new to the Elastic Stack and have been attempting to input 4 simple single-line JSON files into elasticsearch without success. The format of the logs are:
{"0":{"path":"\Users\Administrator\AppData\Local\Microsoft\Windows\Temporary Internet Files\Content.IE5\6YL4T24G\additional-methods.min[1].js","artifact_sha256":"bc351c64cd81a9b0f1d9e00bb14fc9b1c4cd6fff55a173e856b2e1fd055e1c9e","sample_id":"acb30d9e58f0d801a453d66010e305b5","artifact_md5":"7e91dafdbca663ffa2e130179155a3c5","severity":80,"aid":18,"timestamp":"2019-02-05T17:58:22Z","confidence":80,"ioc":"js-contains-massive-strings"}}
{"0":{"domain":"mimiabner.com","timestamp":"2019-02-05T17:24:04Z","ioc":"document-network-traffic","confidence":90,"severity":90,"sample_id":"8ec17e9079314e82b00130b80e086221","sample_sha256":"c6449171b29a631014a8001ce785c45b707fe962b8ccea2c89f99d005447b4a0"}}
{"0":{"ip":"174.84.250.37","port":443,"timestamp":"2019-02-05T17:24:04Z","ioc":"document-direct-ip-traffic","confidence":90,"severity":100,"sample_id":"8ec17e9079314e82b00130b80e086221","sample_sha256":"c6449171b29a631014a8001ce785c45b707fe962b8ccea2c89f99d005447b4a0"}}
Since the fields and data are similar, I would like to load these into the same index. I have attempted to do this with curl to elasticsearch and with logstash without success. The errors are related to mapping, and I have not been able to find good documentation / online resources on how to do this.
My first question is; Given that this is native JSON, should the data be loaded directly into elasticsearch or is logstash the better method? (Data updates once daily, but should be overwritten)
The second question is how to map the fields? The first field, "0" is the id, but the rest of the fields are defined. E.g. "path" is the path, "domain" is domain, "url" is URL, etc.
Any assistance is greatly appreciated!