# Ingesting .doc .txt .csv .sql and some others **URL:** \ **Category:** Logstash\ **Created:** [August 30, 2020, 12:07am UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861 "2020-08-30T00:07:12Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [August 30, 2020, 12:07am UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861/1 "2020-08-30T00:07:13Z")
Hi everyone, I have a question. What would be the fastest way to ingest data? I have a folder that contains a variety of file formats including .doc, .txt, .csv and .sql. I have roughly 30 gigs of files like this that I would like to be able to search on Kibana. Any help would be great thank you. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 31, 2020, 2:02am UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861/2 "2020-08-31T02:02:41Z")
At the moment there is no single way to get all that data into Elasticsearch; - Workplace Search can handle the .doc files - CSV = Filebeat or Logstash - .sql is text, but the custom format means you'd need to figure out exactly how you want to search these - .txt is similar, it depends on what is in them and how you want to search them ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [August 31, 2020, 3:52am UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861/3 "2020-08-31T03:52:31Z")
Awesome so I am on Ubuntu 20 LTS Server and I wrote this as my config file. ``` input { file { path => "/media/alex/XXXX/converted/*/*/*.csv" start_position => "beginning" sincedb_path => "/dev/null" } } filter { csv { separator => "," columns => [""] } output { elasticsearch { hosts => "localhost" index => "docs" } stdout {} } ``` Then I ran `$ sudo bin/logstash -f /usr/share/logstash/csv.conf` and I get the error of ``` WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults ERROR: Unknown command 'sudo' See: 'bin/logstash --help' [ERROR] 2020-08-31 03:49:03.102 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit ``` So then I try to run ` sudo -u logstash /usr/share/logstash/bin/logstash --path.settings=/etc/logstash -f /usr/share/logstash/csv.conf` and I get error Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties [2020-08-31T03:51:15,351][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"7.9.0", "jruby.version"=\>"jruby 9.2.12.0 (2.5.7) 2020-07-01 db01a49ba6 OpenJDK 64-Bit Server VM 25.265-b01 on 1.8.0\_265-8u265-b01-0ubuntu2~20.04-b01 +indy +jit [linux-x86\_64]"} [2020-08-31T03:51:15,898][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified [2020-08-31T03:51:17,164][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of [\t\r\n], "#", "{" at line 9, column 3 (byte 120) after filter\n\t\t", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:183:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:44:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline\_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:357:in `block in converge\_state'"]} [2020-08-31T03:51:17,466][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} [2020-08-31T03:51:22,558][INFO][logstash.runner] Logstash shut down. [2020-08-31T03:51:22,575][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit any ideas what I am doing wrong? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 31, 2020, 4:22am UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861/4 "2020-08-31T04:22:39Z")
You seem to be missing a final closing bracket on your filter section. ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [August 31, 2020, 12:46pm UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861/5 "2020-08-31T12:46:51Z")
Yea it didnt copy over for some reason. Any ideas what why it wont execute? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 31, 2020, 8:42pm UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861/6 "2020-08-31T20:42:09Z")
Did you add the bracket? Is there a different error? ---
**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Post date:** [September 1, 2020, 7:35pm UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861/7 "2020-09-01T19:35:59Z")
I added the bracket. Seems to be an issue with Java ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 1, 2020, 8:48pm UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861/8 "2020-09-01T20:48:07Z")
If there is a different outcome, posting the log would be helpful. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 29, 2020, 8:48pm UTC](https://discuss.elastic.co/t/ingesting-doc-txt-csv-sql-and-some-others/246861/9 "2020-09-29T20:48:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.