# Why is it so hard to just grab a file and upload to Elasticsearch? **URL:** \ **Category:** Elasticsearch\ **Created:** [August 15, 2019, 8:34pm UTC](https://discuss.elastic.co/t/why-is-it-so-hard-to-just-grab-a-file-and-upload-to-elasticsearch/195392 "2019-08-15T20:34:02Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![joaociocca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joaociocca/32/15827_2.png) [@joaociocca](https://discuss.elastic.co/u/joaociocca)\ **Post date:** [August 15, 2019, 8:34pm UTC](https://discuss.elastic.co/t/why-is-it-so-hard-to-just-grab-a-file-and-upload-to-elasticsearch/195392/1 "2019-08-15T20:34:02Z")
so, I was wondering this today: why is it so hard to just grab a log file and punch it into Elasticsearch? I understand when you need to setup Filebeat listeners, other beats, some Logstash pipelines and such... but what about if you just have a really big Apache log, or a huge CSV file, that you just want to easily take a look using Elastic? I found one "easy" way to do such a thing with .evtx files, but everything else... nop. Tips? Suggestions? Pointers? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [August 15, 2019, 9:36pm UTC](https://discuss.elastic.co/t/why-is-it-so-hard-to-just-grab-a-file-and-upload-to-elasticsearch/195392/2 "2019-08-15T21:36:43Z")
Have you tried the File Import Wizard in the Data Visualizer? Its a "Basic Feature" under Machine Learning... Not made for huge files but quick way to get some data in and take a look... [http://localhost:5601/app/ml#/datavisualizer?\_g=()](http://localhost:5601/app/ml#/datavisualizer?_g=()) ---
**Author:** ![joaociocca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joaociocca/32/15827_2.png) [@joaociocca](https://discuss.elastic.co/u/joaociocca)\ **Post date:** [August 15, 2019, 9:40pm UTC](https://discuss.elastic.co/t/why-is-it-so-hard-to-just-grab-a-file-and-upload-to-elasticsearch/195392/3 "2019-08-15T21:40:52Z")
I have not! But sadly that's not a way for me at work: the DLP solution at work here prevents any file from being uploaded via browser, even if the destination is localhost:5601... ((edit)) I've finally got people to configure the DLP solution the right way, and was able to use the File Import Wizard! Awesome, @stephenb! Thanks a lot! Is there a way around the 100MB limit? ---
**Author:** ![forloop](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forloop/32/9021_2.png) [@forloop](https://discuss.elastic.co/u/forloop)\ **Post date:** [August 16, 2019, 10:19pm UTC](https://discuss.elastic.co/t/why-is-it-so-hard-to-just-grab-a-file-and-upload-to-elasticsearch/195392/4 "2019-08-16T22:19:32Z")
Another way to ingest data is to use one of the [language clients for Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/client/index.html). Granted, you'll need to write some code, but it is a flexible approach. ---
**Author:** ![joaociocca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joaociocca/32/15827_2.png) [@joaociocca](https://discuss.elastic.co/u/joaociocca)\ **Post date:** [August 17, 2019, 1:23am UTC](https://discuss.elastic.co/t/why-is-it-so-hard-to-just-grab-a-file-and-upload-to-elasticsearch/195392/5 "2019-08-17T01:23:50Z")
yeah... I'm not good at programming in general. I can do some bash/batch though =p I was trying to make a bash using tshark + bulky API to load some pcaps, but for some reason, after I deleted the packt-\* index the first time, no matter how many times I tried again, nothing would show up, no matter how much time curl took trying to XPUT my json files in elasticsearch... ((edit)) kudos to me: I'm working on Windows and forgot tshark would output packets.json with CRLF instead of just LF. ((/edit)) Oh, I was following this article, btw! > **[Analyzing network packets with Wireshark, Elasticsearch, and Kibana](https://www.elastic.co/pt/blog/analyzing-network-packets-with-wireshark-elasticsearch-and-kibana)** > > Learn how to architect a real-time data pipeline for network packet analysis using Wireshark, Filebeat, Logstash, Ingest Pipelines, Elasticsearch, and Kibana. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [August 17, 2019, 6:34am UTC](https://discuss.elastic.co/t/why-is-it-so-hard-to-just-grab-a-file-and-upload-to-elasticsearch/195392/6 "2019-08-17T06:34:10Z")
I described here a recipe using logstash: [https://www.elastic.co/blog/enriching-your-postal-addresses-with-the-elastic-stack-part-1](https://www.elastic.co/blog/enriching-your-postal-addresses-with-the-elastic-stack-part-1) You can also use filebeat and ingest node pipeline with this plugin > **[johtani/elasticsearch-ingest-csv](https://github.com/johtani/elasticsearch-ingest-csv)** > > Ingest CSV processor parses CSV data and stores it as individual fields - johtani/elasticsearch-ingest-csv ---
**Author:** ![joaociocca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joaociocca/32/15827_2.png) [@joaociocca](https://discuss.elastic.co/u/joaociocca)\ **Post date:** [August 18, 2019, 3:12am UTC](https://discuss.elastic.co/t/why-is-it-so-hard-to-just-grab-a-file-and-upload-to-elasticsearch/195392/7 "2019-08-18T03:12:13Z")
David, I have to say it: I loved that post. LOVED it. The way you start slowly and build up things, tearing them apart and explaining what's going on and why you built each thing their way is just AMAZING. I wish more people would follow this idea! I tried using logstash a couple times, but even trying to follow the documentation was kinda hard for me, when the configuration files just came over and _BAM_ do this! I always get like... why? What's each of those parts doing? I dunno if there's something wrong in my head, but it's really hard for me. Your post, on the other hand... BEAUTIFUL. Specially the "Writing the Logstash Pipeline" part. ((edit)) I had no idea so far that I could cat something pipe into logstash to load them into Elasticsearch!!! 😱 ((/edit)) Thank you so very much! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 15, 2019, 3:12am UTC](https://discuss.elastic.co/t/why-is-it-so-hard-to-just-grab-a-file-and-upload-to-elasticsearch/195392/8 "2019-09-15T03:12:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.