# Error code when importing csv file with logstash

**URL:** https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762
**Category:** Logstash
**Created:** [October 2, 2018, 8:14pm UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762 "2018-10-02T20:14:38Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![drr](https://avatars.discourse-cdn.com/v4/letter/d/47e85d/32.png) [@drr](https://discuss.elastic.co/u/drr)
#### Post date: [October 2, 2018, 8:14pm UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/1 "2018-10-02T20:14:38Z")

</div>

hi logstash team

I am trying to import csv file andI cant figure out why I am getting below error, your help would be greatly appreciated

```
./logstash -f ../conf/logstash.yml
Sending Logstash's logs to /home/d/Documents/logstash-6.2.4/logs which is now configured via log4j2.properties
[2018-10-02T21:32:03,882][INFO][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/home/d/Documents/logstash-6.2.4/modules/fb_apache/configuration"}
[2018-10-02T21:32:03,945][INFO][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/home/d/Documents/logstash-6.2.4/modules/netflow/configuration"}
[2018-10-02T21:32:04,055][FATAL][logstash.runner] An unexpected error occurred! {:error=>#<ArgumentError: Setting "" hasn't been registered>, :backtrace=>["/home/d/Documents/logstash-6.2.4/logstash-core/lib/logstash/settings.rb:37:in `get_setting'", "/home/d/Documents/logstash-6.2.4/logstash-core/lib/logstash/settings.rb:70:in `set_value'", "/home/d/Documents/logstash-6.2.4/logstash-core/lib/logstash/settings.rb:89:in `block in merge'", "org/jruby/RubyHash.java:1343:in `each'", "/home/d/Documents/logstash-6.2.4/logstash-core/lib/logstash/settings.rb:89:in `merge'", "/home/d/Documents/logstash-6.2.4/logstash-core/lib/logstash/settings.rb:138:in `validate_all'", "/home/d/Documents/logstash-6.2.4/logstash-core/lib/logstash/runner.rb:264:in `execute'", "/home/d/Documents/logstash-6.2.4/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/home/d/Documents/logstash-6.2.4/logstash-core/lib/logstash/runner.rb:219:in `run'", "/home/d/Documents/logstash-6.2.4/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/home/d/Documents/logstash-6.2.4/lib/bootstrap/environment.rb:67:in `<main>'"]}
[2018-10-02T21:32:04,073][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit

```

my yml file looks like this

```
input {
  file {
    path => "/home/d/pcap/test.csv"
  }
}
filter {
  csv {
     separator => ","
    columns => ["No","Time","Source","Destination","Protocol","Length","Info"]
  }
}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "pcap"
  }
stdout {}
}
```

---

<div class="post-metadata">

### Author: ![bardie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bardie/32/36173_2.png) [@bardie](https://discuss.elastic.co/u/bardie)
#### Post date: [October 3, 2018, 3:42pm UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/2 "2018-10-03T15:42:55Z")

</div>

Try the following:

```
input {
  file {
    path => "/home/d/pcap/test.csv"
    start_position => "beginning"
    sincedb_path => "/var/log/tracking_no"
 }
  }

filter {
csv {
  columns => ["No","Time","Source","Destination","Protocol","Length","Info"]
  separator => ","
}
}

output {
stdout { codec => json }
elasticsearch {
hosts => ["localhost:9200"]
index => "pcap-%{+YYYY.MM.dd}"
}
}
```

---

<div class="post-metadata">

### Author: ![drr](https://avatars.discourse-cdn.com/v4/letter/d/47e85d/32.png) [@drr](https://discuss.elastic.co/u/drr)
#### Post date: [October 4, 2018, 7:44am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/3 "2018-10-04T07:44:01Z")

</div>

Thanks @bardie , I changed yml to the one you sugested but it doesn't work.

this is what I get

```
d@ubuntu:~/Documents/logstash-6.4.2/bin$ ./logstash -f ../config/logstash1.yml 
Sending Logstash logs to /home/d/Documents/logstash-6.4.2/logs which is now configured via log4j2.properties
[2018-10-04T09:37:18,170][FATAL][logstash.runner] An unexpected error occurred! {:error=>#<ArgumentError: Setting "" hasn't been registered>, :backtrace=>["/home/d/Documents/logstash-6.4.2/logstash-core/lib/logstash/settings.rb:36:in `get_setting'", "/home/d/Documents/logstash-6.4.2/logstash-core/lib/logstash/settings.rb:69:in `set_value'", "/home/d/Documents/logstash-6.4.2/logstash-core/lib/logstash/settings.rb:88:in `block in merge'", "org/jruby/RubyHash.java:1343:in `each'", "/home/d/Documents/logstash-6.4.2/logstash-core/lib/logstash/settings.rb:88:in `merge'", "/home/d/Documents/logstash-6.4.2/logstash-core/lib/logstash/settings.rb:137:in `validate_all'", "/home/d/Documents/logstash-6.4.2/logstash-core/lib/logstash/runner.rb:278:in `execute'", "/home/d/Documents/logstash-6.4.2/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/home/d/Documents/logstash-6.4.2/logstash-core/lib/logstash/runner.rb:237:in `run'", "/home/d/Documents/logstash-6.4.2/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/home/d/Documents/logstash-6.4.2/lib/bootstrap/environment.rb:73:in `<main>'"]}
[2018-10-04T09:37:18,187][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
d@ubuntu:~/Documents/logstash-6.4.2/bin$ 

```

has anyone had this problem before? yml file looks legit

---

<div class="post-metadata">

### Author: ![bardie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bardie/32/36173_2.png) [@bardie](https://discuss.elastic.co/u/bardie)
#### Post date: [October 4, 2018, 8:44am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/4 "2018-10-04T08:44:14Z")

</div>

> [@drr](#):
>
> Setting "" hasn't been registered

Please could you post your logstash.yml file

---

<div class="post-metadata">

### Author: ![drr](https://avatars.discourse-cdn.com/v4/letter/d/47e85d/32.png) [@drr](https://discuss.elastic.co/u/drr)
#### Post date: [October 4, 2018, 8:54am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/5 "2018-10-04T08:54:28Z")

</div>

sure, its logstash1.yml but i am calling it with -f when trying to run logstash (dont think this is the problem)

```
d@ubuntu:~/Documents/logstash-6.4.2/bin$ cat ../config/logstash1.yml 
input {
  file {
    path => "/home/d/pcap/test.csv"
    start_position => "beginning"
    sincedb_path => "/var/log/tracking_no"
 }
  }

filter {
csv {
  columns => ["No","Time","Source","Destination","Protocol","Length","Info"]
  separator => ","
}
}

output {
stdout { codec => json }
elasticsearch {
hosts => ["localhost:9200"]
index => "pcap-%{+YYYY.MM.dd}"
}
}
d@ubuntu:~/Documents/logstash-6.4.2/bin$
```

---

<div class="post-metadata">

### Author: ![bardie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bardie/32/36173_2.png) [@bardie](https://discuss.elastic.co/u/bardie)
#### Post date: [October 4, 2018, 9:33am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/6 "2018-10-04T09:33:05Z")

</div>

Logstash has one configuration file called logstash.yml

Sample:

> <https://github.com/elastic/logstash/blob/master/config/logstash.yml>

This file determines how logstash runs. If you do not have this file then that might be whats causing the error.

The file that you have posted is the logstash parser.

This is my logstash directory. The conf.d folder contains all my logstash parsers and the logstash.yml contains the logstash configurations.

![image](https://us1.discourse-cdn.com/elastic/original/3X/4/7/47e829091eb3c8f81980b0a987cea29123c69e3c.png)

Do you have a logstash.yml file other than the one you submitted?

---

<div class="post-metadata">

### Author: ![drr](https://avatars.discourse-cdn.com/v4/letter/d/47e85d/32.png) [@drr](https://discuss.elastic.co/u/drr)
#### Post date: [October 4, 2018, 9:46am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/7 "2018-10-04T09:46:01Z")

</div>

hi there,

I have this file and it contains exactly what I shared before - just to be sure it is not the case I renamed it once again to logstash.yml but still error message remains the same.

I seen in other posts that the problem might be with logstash.yml content but I dont see there anything what could cause it.

it looks like I got all files where they should be in config folder as well

```
d@ubuntu:~/Documents/logstash-6.4.2/config$ ls
jvm.options log4j2.properties logstash-sample.conf logstash.yml pipelines.yml startup.options
d@ubuntu:~/Documents/logstash-6.4.2/config$ 

```

I wasnt changing there anything except that logstash.yml

---

<div class="post-metadata">

### Author: ![bardie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bardie/32/36173_2.png) [@bardie](https://discuss.elastic.co/u/bardie)
#### Post date: [October 4, 2018, 9:52am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/8 "2018-10-04T09:52:30Z")

</div>

STEP 1: Create a folder called pipeline  
STEP 2: Rename the logstash.yml to sample.conf and move it to the pipelines folder  
STEP 3: Create a new file called logstash.yml and paste the following

* * *

```
http.host: "0.0.0.0"
path.config: /usr/share/logstash/pipeline

```

Then try running it

---

<div class="post-metadata">

### Author: ![drr](https://avatars.discourse-cdn.com/v4/letter/d/47e85d/32.png) [@drr](https://discuss.elastic.co/u/drr)
#### Post date: [October 4, 2018, 10:37am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/9 "2018-10-04T10:37:46Z")

</div>

ok this changed output and I dont see critical error at the moment but that last message is being constantly thrown on the console. no index was created either.

```
d@ubuntu:~/Documents/logstash-6.4.2/bin$ ./logstash
Sending Logstash logs to /home/d/Documents/logstash-6.4.2/logs which is now configured via log4j2.properties
[2018-10-04T12:32:10,040][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-10-04T12:32:10,099][INFO][logstash.agent] No persistent UUID file found. Generating new UUID {:uuid=>"29dce12b-a71f-415f-9e71-4e68702eacf4", :path=>"/home/d/Documents/logstash-6.4.2/data/uuid"}
[2018-10-04T12:32:10,809][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.4.2"}
[2018-10-04T12:32:13,625][INFO][logstash.pipeline] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-10-04T12:32:14,152][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2018-10-04T12:32:14,166][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-10-04T12:32:14,445][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2018-10-04T12:32:14,565][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2018-10-04T12:32:14,568][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-10-04T12:32:14,701][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2018-10-04T12:32:14,762][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-10-04T12:32:14,794][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-10-04T12:32:15,104][INFO][logstash.pipeline] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x4a461065 run>"}
[2018-10-04T12:32:15,216][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-10-04T12:32:15,240][INFO][filewatch.observingtail] START, creating Discoverer, Watch with file and sincedb collections
[2018-10-04T12:32:15,341][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::File start_position=>"beginning", path=>["/home/d/pcap/test.csv"], id=>"5f0a1b5379a03722fad3e4ebbac4feeac072610f7d703b140b966224506698f1", sincedb_path=>"/var/log/tracking_no", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_26cd4a5b-035e-4695-a928-1ff7fef89209", enable_metric=>true, charset=>"UTF-8">, stat_interval=>1.0, discover_interval=>15, sincedb_write_interval=>15.0, delimiter=>"\n", close_older=>3600.0, mode=>"tail", file_completed_action=>"delete", sincedb_clean_after=>1209600.0, file_chunk_size=>32768, file_chunk_count=>140737488355327, file_sort_by=>"last_modified", file_sort_direction=>"asc">
  Error: Permission denied - /var/log/tracking_no
  Exception: Errno::EACCES
  Stack: org/jruby/RubyFile.java:366:in `initialize'
org/jruby/RubyIO.java:1154:in `open'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:1167:in `block in touch'
org/jruby/RubyArray.java:1734:in `each'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:1161:in `touch'
/home/d/Documents/logstash-6.4.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-file-4.1.6/lib/filewatch/sincedb_collection.rb:22:in `initialize'
/home/d/Documents/logstash-6.4.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-file-4.1.6/lib/filewatch/observing_base.rb:62:in `build_watch_and_dependencies'
/home/d/Documents/logstash-6.4.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-file-4.1.6/lib/filewatch/observing_base.rb:56:in `initialize'
/home/d/Documents/logstash-6.4.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-file-4.1.6/lib/logstash/inputs/file.rb:332:in `start_processing'
/home/d/Documents/logstash-6.4.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-file-4.1.6/lib/logstash/inputs/file.rb:337:in `run'
/home/d/Documents/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:409:in `inputworker'
/home/d/Documents/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:403:in `block in start_input'
[2018-10-04T12:32:15,594][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-10-04T12:32:16,349][INFO][filewatch.observingtail] START, creating Discoverer, Watch with file and sincedb collections
[2018-10-04T12:32:16,360][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::File start_position=>"beginning", path=>["/home/d/pcap/test.csv"], id=>"5f0a1b5379a03722fad3e4ebbac4feeac072610f7d703b140b966224506698f1", sincedb_path=>"/var/log/tracking_no", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_26cd4a5b-035e-4695-a928-1ff7fef89209", enable_metric=>true, charset=>"UTF-8">, stat_interval=>1.0, discover_interval=>15, sincedb_write_interval=>15.0, delimiter=>"\n", close_older=>3600.0, mode=>"tail", file_completed_action=>"delete", sincedb_clean_after=>1209600.0, file_chunk_size=>32768, file_chunk_count=>140737488355327, file_sort_by=>"last_modified", file_sort_direction=>"asc">
  Error: Permission denied - /var/log/tracking_no
  Exception: Errno::EACCES
  Stack: org/jruby/RubyFile.java:366:in `initialize'
org/jruby/RubyIO.java:1154:in `open'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:1167:in `block in touch'
org/jruby/RubyArray.java:1734:in `each'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:1161:in `touch'
```

---

<div class="post-metadata">

### Author: ![drr](https://avatars.discourse-cdn.com/v4/letter/d/47e85d/32.png) [@drr](https://discuss.elastic.co/u/drr)
#### Post date: [October 4, 2018, 10:50am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/10 "2018-10-04T10:50:17Z")

</div>

hey @bardie

changes you proposed helped push things forward.

what I additionally did is that I removed

```
sincedb_path => "/var/log/tracking_no"

```

line from the sample.config file and it worked

here this issue is described,

> <https://github.com/elastic/logstash/issues/3311>

many thanks for your help!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 1, 2018, 10:50am UTC](https://discuss.elastic.co/t/error-code-when-importing-csv-file-with-logstash/150762/11 "2018-11-01T10:50:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
