We read most of what have been written about the topic, we tried many permutations and we can't successfully make logstash to work:
Here is the console output
start -----------------
C:\Deleteme\es_logstash>C:\Software\ES\logstash-7.3.1\bin\logstash -f "/Deleteme/data.conf"
Thread.exclusive is deprecated, use Thread::Mutex
Sending Logstash logs to C:/Software/ES/logstash-7.3.1/logs which is now configured via log4j2.properties
[2019-08-30T09:36:25,271][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-08-30T09:36:25,302][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.3.1"}
[2019-08-30T09:36:27,548][INFO ][org.reflections.Reflections] Reflections took 63 ms to scan 1 urls, producing 19 keys and 39 values
[2019-08-30T09:36:31,235][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[https://elastic:xxxxxx@fb48cb04fcd4b33e24075e0b3773c89b.us-east-1.aws.found.io:9243/]}}
[2019-08-30T09:36:32,024][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"https://:9243/"}
[2019-08-30T09:36:32,412][ERROR][logstash.javapipeline ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::Json::ParserError: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: (byte)"Kibana/* INTER UI FONT /
/ INTER UI FONT /
/ INTER UI FONT /
/ INTER UI FONT */
@font-face {
font-family: 'Inter UI';
font-style: normal;
font-weight: 100;
src: url("/ui/fonts/inter_ui/Inter-UI-Thin-BETA.woff2") format("woff2"),
url("/ui/fonts/inter_ui/Inter-UI-Thin-BETA.woff") format"[truncated 79642 bytes]; line: 1, column: 2]>, :backtrace
end ------------------
We also tried port 9200
We also tried without https://
Here is the sample data
start -----------------
badge,firstname,lastname
1,john,smith
2,mary,cooper
end ------------------
Here is the configuration file
start -----------------
input {
file {
path => "/Deleteme/data.csv"
sincedb_path => "/Deleteme/null"
start_position => "beginning"
}
}
filter {
csv {
separator => ","
skip_header => "true"
columns => ["badge","firstname","lastname"]
}
mutate {convert => ["badge", "integer"]}
}
output {
elasticsearch{
hosts => ["https://:9243"]
user => "elastic"
password => ""
index => "mytest"
}
stdout {}
}
end ------------------
Here is how do we run it (in windows)
start -----------------
C:\Software\ES\logstash-7.3.1\bin\logstash -f "/Deleteme/data.conf"
end ------------------
We also tried running from some Mac laptops and we got the same behavior/error, which make us believe we are doing something wrong.
Thanks in advance!