# Loading a CSV file

**URL:** <https://discuss.elastic.co/t/loading-a-csv-file/113760>\
**Category:** Logstash\
**Created:** [January 2, 2018, 11:03am UTC](https://discuss.elastic.co/t/loading-a-csv-file/113760 "2018-01-02T11:03:58Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rahul\_Pm](https://avatars.discourse-cdn.com/v4/letter/r/7ea924/32.png) [@Rahul\_Pm](https://discuss.elastic.co/u/Rahul_Pm)\
**Post date:** [January 2, 2018, 11:03am UTC](https://discuss.elastic.co/t/loading-a-csv-file/113760/1 "2018-01-02T11:03:58Z")

</div>

Hi,  
I'm trying to load a csv file (stored on my windows machine) into elasticsearch via logstash. I need to visualize it on Kibana. I have everything installed on my windows machine.

P.S. It's my first time using Logstash.

Can anybody help me on how I can go about it (configuration) ? Or maybe point me to a relevant document ? 🙂

Thank you,  
Rahul

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 2, 2018, 1:22pm UTC](https://discuss.elastic.co/t/loading-a-csv-file/113760/2 "2018-01-02T13:22:49Z")

</div>

you will need a file input, a csv filter and a elasticsearch output.

See:  
[https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html](https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html) (ignore filebeat for now)  
[https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html)  
[https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html)  
[https://www.elastic.co/guide/en/logstash/current/data-deserialization.html](https://www.elastic.co/guide/en/logstash/current/data-deserialization.html) (see csv section)

You should experiment with this:

```auto
input {
  generator {
    message => "stop,wait for it,go go go - man"
    count => 1
  }
}

filter {
  csv {
    separator => ","
    columns => ["red","amber","green"]
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

```

Paste in a line from your file into the `message` quotes. Add your real columns in quotes to the `columns` array.  
Once you are happy that the filter is doing the correct thing then replace the `generator` input with a file input section - see the doc links above.  
Once you are happy with the output then replace the `stdout` section with a elasticsearch output.

Remember to delete the sincedb file before each run or the CSV file will not be re-read.

---

<div class="post-metadata">

**Author:** ![Rahul\_Pm](https://avatars.discourse-cdn.com/v4/letter/r/7ea924/32.png) [@Rahul\_Pm](https://discuss.elastic.co/u/Rahul_Pm)\
**Post date:** [January 4, 2018, 5:58am UTC](https://discuss.elastic.co/t/loading-a-csv-file/113760/3 "2018-01-04T05:58:04Z")

</div>

Hi,  
Thank you for the reply.

So I followed the steps. And Logstash was not being detected when the parent folder name had a space in it. Now logstash is being detected. And in the config folder\>pipeline.yml file\> I wrote :

[pipeline.id](http://pipeline.id): events  
path.config: "C:\logstash-6.1.1\config\Logstash.conf"

"Logstash.conf" being the conf file I created. And it contains :

input {  
file {  
path =\> "C:\Users\Desktop\Events-Timestamp.csv"  
start\_position =\> beginning

}  
}

filter {  
csv {  
columns =\> ["siteid", "hostaddress", "hostname", "service", "state", "Timestamp"]  
}  
}

output {  
stdout { codec =\> rubydebug }  
elasticsearch {  
host =\> "localhost"  
index =\> "Events-Timestamp"  
}  
}

And when I try to run logstash, I get an error that says Could not read Pipeline.yml.

I'm not sure where I'm going wrong. Please help.

Thank you,  
Rahul

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 4, 2018, 8:44am UTC](https://discuss.elastic.co/t/loading-a-csv-file/113760/4 "2018-01-04T08:44:51Z")

</div>

Did you follow the YAML syntax?

See: [https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html](https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html)

---

<div class="post-metadata">

**Author:** ![Rahul\_Pm](https://avatars.discourse-cdn.com/v4/letter/r/7ea924/32.png) [@Rahul\_Pm](https://discuss.elastic.co/u/Rahul_Pm)\
**Post date:** [January 4, 2018, 8:52am UTC](https://discuss.elastic.co/t/loading-a-csv-file/113760/5 "2018-01-04T08:52:02Z")

</div>

Yes, I got logstash running. This is the output I received in cmd.

C:\logstash-6.1.1\bin\>logstash -f logstash.conf

Sending Logstash's logs to C:/logstash-6.1.1/logs which is now configured via log4j2.properties  
[2018-01-04T14:18:45,793][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"C:/logstash-6.1.1/modules/fb\_apache/configuration"}  
[2018-01-04T14:18:45,824][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"C:/logstash-6.1.1/modules/netflow/configuration"}  
[2018-01-04T14:18:46,247][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified  
[2018-01-04T14:18:47,411][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.1.1"}  
[2018-01-04T14:18:47,658][INFO][logstash.config.source.local.configpathloader] No config files found in path {:path=\>"C:/logstash-6.1.1/bin/logstash.conf"}  
[ERROR] 2018-01-04 14:18:47.689 [Ruby-0-Thread-1: C:\logstash-6.1.1\lib\bootstrap\environment.rb:6] sourceloader - No configuration found in the configured sources.  
[2018-01-04T14:18:48,172][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}

Can you please tell me how I can visulaize my data in Kibana?

Thank you,  
Rahul

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 4, 2018, 9:04am UTC](https://discuss.elastic.co/t/loading-a-csv-file/113760/6 "2018-01-04T09:04:34Z")

</div>

It is a complex topic. You should read up on that in the Kibana docs.  
Use the [Kibana forum](https://discuss.elastic.co/c/kibana)  
Do a search for getting started posts then post your questions there.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 1, 2018, 9:04am UTC](https://discuss.elastic.co/t/loading-a-csv-file/113760/7 "2018-02-01T09:04:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
