# Loading a CSV file, basic question

**URL:** <https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528>\
**Category:** Logstash\
**Created:** [January 6, 2016, 5:43pm UTC](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528 "2016-01-06T17:43:35Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Christopher\_Curzon](https://avatars.discourse-cdn.com/v4/letter/c/2acd7d/32.png) [@Christopher\_Curzon](https://discuss.elastic.co/u/Christopher_Curzon)\
**Post date:** [January 6, 2016, 5:43pm UTC](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528/1 "2016-01-06T17:43:35Z")

</div>

This is my first effort getting logstash to put data into Elasticsearch. So I think I must be missing something fairly obvious.

I have a addr.csv file with records in this format:

369|90045|123 ABC ST|LOS ANGELES|CA  
368|90045|PVKA0010|LA|CA  
etc...

and I have addr02.conf file like this:

input {  
file {  
path =\> "/home/zed/logstash-2.1.0/load\_ex/addr.csv"  
type =\> "addrtype"  
start\_position =\> "beginning"  
}  
}  
filter {  
csv {  
columns =\> ["REC\_ID", "ZIP\_CODE", "STR\_ADDR", "CITY", "ST"]  
separator =\> "|"  
}  
}  
output {  
elasticsearch {  
action =\> "index"  
hosts =\> ["10.30.90.5"]  
index =\> "logstash"  
workers =\> 1  
}

but when I start logstash like this

bin/logstash -f load\_ex/addr02.conf -l load\_ex/addrX.log --verbose

nothing gets into ES. the first few lines of the log file are this (blank lines added for clarity):

{:timestamp=\>"2016-01-05T17:34:01.559000-0800", :message=\>"Registering file input", :path=\>["/home/zed/logstash-2.1.0/load\_ex/addr.csv"], :level=\>:info}

{:timestamp=\>"2016-01-05T17:34:01.563000-0800", :message=\>"No sincedb\_path set, generating one based on the file path", :sincedb\_path=\>"/home/zed/.sincedb\_339edbe413a1e111968d1d43df97837f", :path=\>["/home/zed/logstash-2.1.0/load\_ex/addr.csv"], :level=\>:info}

{:timestamp=\>"2016-01-05T17:34:01.572000-0800", :message=\>"Worker threads expected: 4, worker threads started: 4", :level=\>:info}

{:timestamp=\>"2016-01-05T17:34:01.581000-0800", :message=\>"Using mapping template from", :path=\>nil, :level=\>:info}

{:timestamp=\>"2016-01-05T17:34:01.841000-0800", :message=\>"Attempting to install template", :manage\_template=\>{"template"=\>"logstash-", "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"all"=\>{"enabled"=\>true, "omitnorms"=\>true}, "dynamic\_templates"=\>[{"message\_field"=\>{"match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"string", "index"=\>"analyzed", "omit\_norms"=\>true, "fielddata"=\>{"format"=\>"disabled"}}}}, {"string\_fields"=\>{"match"=\>"", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"string", "index"=\>"analyzed", "omit\_norms"=\>true, "fielddata"=\>{"format"=\>"disabled"}, "fields"=\>{"raw"=\>{"type"=\>"string", "index"=\>"not\_analyzed", "doc\_values"=\>true, "ignore\_above"=\>256}}}}}, {"float\_fields"=\>{"match"=\>"", "match\_mapping\_type"=\>"float", "mapping"=\>{"type"=\>"float", "doc\_values"=\>true}}}, {"double\_fields"=\>{"match"=\>"", "match\_mapping\_type"=\>"double", "mapping"=\>{"type"=\>"double", "doc\_values"=\>true}}}, {"byte\_fields"=\>{"match"=\>"", "match\_mapping\_type"=\>"byte", "mapping"=\>{"type"=\>"byte", "doc\_values"=\>true}}}, {"short\_fields"=\>{"match"=\>"", "match\_mapping\_type"=\>"short", "mapping"=\>{"type"=\>"short", "doc\_values"=\>true}}}, {"integer\_fields"=\>{"match"=\>"", "match\_mapping\_type"=\>"integer", "mapping"=\>{"type"=\>"integer", "doc\_values"=\>true}}}, {"long\_fields"=\>{"match"=\>"", "match\_mapping\_type"=\>"long", "mapping"=\>{"type"=\>"long", "doc\_values"=\>true}}}, {"date\_fields"=\>{"match"=\>"", "match\_mapping\_type"=\>"date", "mapping"=\>{"type"=\>"date", "doc\_values"=\>true}}}, {"geo\_point\_fields"=\>{"match"=\>"", "match\_mapping\_type"=\>"geo\_point", "mapping"=\>{"type"=\>"geo\_point", "doc\_values"=\>true}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date", "doc\_values"=\>true}, "@version"=\>{"type"=\>"string", "index"=\>"not\_analyzed", "doc\_values"=\>true}, "geoip"=\>{"type"=\>"object", "dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip", "doc\_values"=\>true}, "location"=\>{"type"=\>"geo\_point", "doc\_values"=\>true}, "latitude"=\>{"type"=\>"float", "doc\_values"=\>true}, "longitude"=\>{"type"=\>"float", "doc\_values"=\>true}}}}}}}, :level=\>:info}

since it says

{:timestamp=\>"2016-01-05T17:34:01.581000-0800", :message=\>"Using mapping template from", :path=\>nil, :level=\>:info}

I'm guessing that I'm missing the correct template, or that I haven't correctly specified the template. This is a right-out-of-the-box kind of attempt at using LogStash, but I don't remember seeing anything in the docs about specifying templates.

Thank you for any help you can give.

-- Chris Curzon

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 6, 2016, 9:38pm UTC](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528/2 "2016-01-06T21:38:17Z")

</div>

Is the problem that there is no data in ES?

If so then delete this file and then restart LS;

> [@Christopher\_Curzon](#):
>
> /home/zed/.sincedb\_339edbe413a1e111968d1d43df97837f

---

<div class="post-metadata">

**Author:** ![Christopher\_Curzon](https://avatars.discourse-cdn.com/v4/letter/c/2acd7d/32.png) [@Christopher\_Curzon](https://discuss.elastic.co/u/Christopher_Curzon)\
**Post date:** [January 8, 2016, 5:30pm UTC](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528/3 "2016-01-08T17:30:03Z")

</div>

Thanks, I'm looking at that now.

How did you know what was the exact name of the file to delete?

Is there a good doc that describes Logstash from the operating system point of view? I'd like to know what files get created/deleted/etc in response to various operations.

Thanks.

-- Christopher Curzon

---

<div class="post-metadata">

**Author:** ![Christopher\_Curzon](https://avatars.discourse-cdn.com/v4/letter/c/2acd7d/32.png) [@Christopher\_Curzon](https://discuss.elastic.co/u/Christopher_Curzon)\
**Post date:** [January 8, 2016, 5:33pm UTC](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528/4 "2016-01-08T17:33:30Z")

</div>

No need to answer about the file path ... I see it in the log. 🙂

Is this the mechanism which prevents logstash from loading something multiple times? Maybe it checks for the existence of the file, and if found, then it doesn't reload the data(?).

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 8, 2016, 7:46pm UTC](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528/5 "2016-01-08T19:46:16Z")

</div>

Yep - [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-sincedb\_path](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-sincedb_path)

---

<div class="post-metadata">

**Author:** ![Christopher\_Curzon](https://avatars.discourse-cdn.com/v4/letter/c/2acd7d/32.png) [@Christopher\_Curzon](https://discuss.elastic.co/u/Christopher_Curzon)\
**Post date:** [January 11, 2016, 6:56pm UTC](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528/6 "2016-01-11T18:56:11Z")

</div>

Thank you very much.

It feels like the pieces are starting to come together.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:16am UTC](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528/7 "2017-07-06T05:16:07Z")

</div>


