Please help Importing csv with logstash and create an index in elasticsearch

Elastic Stack Logstash
1

Hello,
I'm new with elasticsearch and I'm trying to import csv data with logstash in Windows 10.

The configuration file i'm using for logstash is "csv-config.conf":

input {
file {
path => "/Users/asanzgarrigues/Desktop/test2.csv"
start_position => "beginning"
}
}
filter {
csv {
separator => ","
columns => ["PacketId","MACAddress","Date","PacketLength","SourceIP","SourcePort","DestIP","DestPort"]
}
}
output {
elasticsearch {
hosts => ["localhost:9203"]
action => "index"
index => "test"
}
stdout {codec => rubydebug}
}

When I execute the command:
logstash -f csv-config.conf

the response is the following:

1904×252 20.6 KB

It looks like the index have been created(or at least I think so) but when I search for my index with kibana or elasticsearch nothing appears.
1949×513 45.4 KB

1952×179 44.3 KB

I changed the elasticsearch port to 9203 because 9200 was not working.
Additonaly, the file log4j2 is located in the correct path but logstash doesn't recognize it.
Edit: I solved the problem of log4j2, the problem was in the file runner.rb (logstash-5.3.0\logstash-core\lib\logstash\runner.rb) at the line:
LogStash::Logging::Logger::initialize("file://" + log4j_config_location)
that line should be replaced by:
LogStash::Logging::Logger::initialize("file:///" + log4j_config_location)

I also try to use the debug mode to get more information about the problem but I don't get nothing clear.

Thnak you for help!

2

Researching I think that the problem is the connection between ES and logstash. Because I have the following WARN in the console response.

[2017-04-26T18:37:13,307][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#URI::HTTP:0x4ec28691 URL:http://localhost:9203/>}

Any ideas of how to solve it? I used the port 9203 for ES because the 92000 was unavailable .
Thank you

3

Please don't post pictures of text, they are difficult to read and some people may not be even able to see them :slight_smile:

4

OK I will not do it again.

I solve this problem you can close the topic.
Thank you

5

How did you solve it? Sharing may help others in future.

6

I finally make it works. I was doing it in the enterprise laptop which is restricted. I create a Ubuntu VM where I have the whole control, and there with the same configuration it finally works.
So I guess that in my enterprise computer there were permissions issues or something like that which denies logstash to work as expected.

Thank you

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.