# Struggling here, grateful for any help **URL:** \ **Category:** Elasticsearch\ **Created:** [September 11, 2017, 7:39pm UTC](https://discuss.elastic.co/t/struggling-here-grateful-for-any-help/100123 "2017-09-11T19:39:42Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![lutonmad](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@lutonmad](https://discuss.elastic.co/u/lutonmad)\ **Post date:** [September 11, 2017, 7:39pm UTC](https://discuss.elastic.co/t/struggling-here-grateful-for-any-help/100123/1 "2017-09-11T19:39:42Z")
Hi there, So I'm probably confusing myself here but I am trying to import logs from our Cloudflare instance into Elasticsearch, via a file that has many json entries and looks like the following (this is just a single log) > { "brandId":100,"flags":2,"hosterId":0,"ownerId":XXXXXX,"rayId":"XXXXXXXXX","securityLevel":"med","timestamp":1504731172284000000,"unstablePublic":null,"zoneId":XXXXXXX,"zoneName":"[test.com](http://test.com)","zonePlan":"enterprise","client":{"asNum":4648,"country":"XX","deviceType":"desktop","ip":"XXXXXXXX","ipClass":"noRecord","srcPort":63970,"sslCipher":"NONE","sslFlags":0,"sslProtocol":"none"},"clientRequest":{"accept":"text/html, application/xhtml+xml, _/_","body":null,"bodyBytes":0,"bytes":246,"cookies":null,"flags":1,"headers":,"httpHost":"[www.test.com](http://www.test.com)","httpMethod":"GET","httpProtocol":"HTTP/1.1","referer":"","uri":"/","userAgent":"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"},"edge":{"bbResult":"0","cacheResponseTime":0,"colo":26,"enabledFlags":12,"endTimestamp":1504731172288000000,"flServerIp":"XXXXXXXX","flServerName":"26f27","flServerPort":80,"pathingOp":"wl","pathingSrc":"macro","pathingStatus":"nr","startTimestamp":1504731172284000000,"usedFlags":0,"rateLimit":{"ruleId":0,"mitigationId":null,"sourceId":"","processedRules":null},"dnsResponse":{"rcode":0,"error":"ok","cached":false,"duration":0,"errorMsg":"","overrideError":false}},"edgeResponse":{"bodyBytes":5,"bytes":285,"compressionRatio":0,"contentType":"","headers":null,"setCookies":null,"status":301}} Originally I set up mappings as follows > curl -XPUT [http://localhost:9200/cloudflare](http://localhost:9200/cloudflare) -d ' > { > "mappings" : { > "_default_" : { > "properties" : { > "securityLevel" : {"type": "string" }, > "zoneName" : {"type": "string" }, > "cacheExternalIP" : {"type": "string" }, > "cacheInternalIP" : {"type": "string" }, > "cacheExternalPort" : { "type" : "integer" }, > "ruleType" : { "type" : "integer" } > } > } > } > } > '; When I go to import the logs using curl -XPUT localhost:9200/cloudflare/\_bulk --data-binary @cloudflare.json I get the following error message. > {"error":{"root\_cause":[{"type":"illegal\_argument\_exception","reason":"Malformed action/metadata line [1], expected START\_OBJECT or END\_OBJECT but found [VALUE\_NUMBER]"}],"type":"illegal\_argument\_exception","reason":"Malformed action/metadata line [1], expected START\_OBJECT or END\_OBJECT but found [VALUE\_NUMBER]"},"status":400} I did find this threat (\_[https://discuss.elastic.co/t/bulk-insert-file-having-many-json-entries-into-elasticsearch/46470\_](https://discuss.elastic.co/t/bulk-insert-file-having-many-json-entries-into-elasticsearch/46470_)) and I understand the format is wrong when I am trying to insert using the Bulk format but I am not having much luck changing it. Would anyone please be able to help me? Thank you ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 12, 2017, 1:38am UTC](https://discuss.elastic.co/t/struggling-here-grateful-for-any-help/100123/2 "2017-09-12T01:38:16Z")
This is what the format for a bulk post needs to look like - [https://www.elastic.co/guide/en/elasticsearch/reference/5.6/docs-bulk.html](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/docs-bulk.html) ---
**Author:** ![mujtabahussain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mujtabahussain/32/17514_2.png) [@mujtabahussain](https://discuss.elastic.co/u/mujtabahussain)\ **Post date:** [September 12, 2017, 1:41am UTC](https://discuss.elastic.co/t/struggling-here-grateful-for-any-help/100123/3 "2017-09-12T01:41:11Z")
Hey! So basically take one entry from the cloudflare.json file, and try and insert it into bulk format. That way, you can ensure that your data is correct and bulk format is working. Then try working a small subset of the data into bulk format and inserting! If that works as well, then do the full set. If that doesn't work, you know you have hit an issue with bulk inserting from the ES end and not your data or formatting 🙂 Try that and let us know 🙂 Best of luck ---
**Author:** ![lutonmad](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@lutonmad](https://discuss.elastic.co/u/lutonmad)\ **Post date:** [September 12, 2017, 2:23pm UTC](https://discuss.elastic.co/t/struggling-here-grateful-for-any-help/100123/4 "2017-09-12T14:23:05Z")
Thanks, so this what I've done so far. > action\_and\_meta\_data > {"brandId":100,"flags":2,"hosterId":0,"ownerId":4855861,"rayId":"XXXXXXXXXX","securityLevel":"med","timestamp":XXXXXXXXX,"unstablePublic":null,"zoneId":XXXXXX,"zoneName":"[test.com](http://test.com)","zonePlan":"enterprise","cache":{"bckType":"c3","cacheExternalIp":"","cacheExternalPort":0,"cacheFileKey":"XXXXXXXXX","cacheInternalIp":"XXXXXXX","cacheServerName":"12c161","cacheStatus":"hit","cacheTokens":0,"endTimestamp":1504669892158999808,"startTimestamp":1504669892158999808},"cacheResponse":{"bodyBytes":0,"bytes":2505,"contentType":"text/javascript","retriedStatus":0,"status":200},"client":{"asNum":15003,"country":"XX","deviceType":"desktop","ip":"104.238.45.55","ipClass":"noRecord","srcPort":50406,"sslCipher":"XXXXXXXXXXXX","sslFlags":1,"sslProtocol":"TLSv1.2"},"clientRequest":{"accept":"_/_","body":null,"bodyBytes":0,"bytes":1063,"cookies":null,"flags":1,"headers":,"httpHost":"[www.test.com](http://www.test.com)","httpMethod":"GET","httpProtocol":"HTTP/1.1","referer":"[https://www.ehs.com/","uri":"/wp-content/themes/velocity/lib/js/resizer.js?ver=1.0.0","userAgent":"Mozilla/5.0](https://www.ehs.com/%22,%22uri%22:%22/wp-content/themes/velocity/lib/js/resizer.js?ver=1.0.0%22,%22userAgent%22:%22Mozilla/5.0) (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"},"edge":{"bbResult":"0","cacheResponseTime":5000000,"colo":12,"enabledFlags":12,"endTimestamp":1504669892160000000,"flServerIp":"104.20.7.52","flServerName":"12f190","flServerPort":443,"pathingOp":"wl","pathingSrc":"macro","pathingStatus":"nr","startTimestamp":1504669892155000064,"usedFlags":0,"rateLimit":{"ruleId":0,"mitigationId":null,"sourceId":"","processedRules":null},"dnsResponse":{"rcode":0,"error":"ok","cached":true,"duration":0,"errorMsg":"","overrideError":false}},"edgeRequest":{"bodyBytes":0,"bytes":1833,"headers":null,"httpHost":"[www.test.com](http://www.test.com)","httpMethod":"GET","keepaliveStatus":"reuseAccepted","uri":"/wp-content/themes/velocity/lib/js/resizer.js?ver=1.0.0"},"edgeResponse":{"bodyBytes":599,"bytes":1017,"compressionRatio":2.47,"contentType":"text/javascript","headers":null,"setCookies":null,"status":200}} Which then gives me this error. > {"error":{"root\_cause":[{"type":"parse\_exception","reason":"Failed to derive xcontent"}],"type":"parse\_exception","reason":"Failed to derive xcontent"},"status":400} Any thoughts? Thanks in advance! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 12, 2017, 10:41pm UTC](https://discuss.elastic.co/t/struggling-here-grateful-for-any-help/100123/5 "2017-09-12T22:41:46Z")
That's still not a valid bulk request. You need to look lower in that doc page for the actual structure, what you have there is simple logical representation of the structure and not valid json. ---
**Author:** ![lutonmad](https://avatars.discourse-cdn.com/v4/letter/l/df788c/32.png) [@lutonmad](https://discuss.elastic.co/u/lutonmad)\ **Post date:** [September 14, 2017, 9:03pm UTC](https://discuss.elastic.co/t/struggling-here-grateful-for-any-help/100123/6 "2017-09-14T21:03:26Z")
Thanks! I've reached out to Cloudflare for their assistance. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 12, 2017, 9:03pm UTC](https://discuss.elastic.co/t/struggling-here-grateful-for-any-help/100123/7 "2017-10-12T21:03:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.