Logstash index is not creating

Elastic Stack Logstash
1

Hi,

I am unable to get the Logstash index created with my Logstash conf file.
Below is the configuration file,

input {
file {
path => "/logs/exampledata_01.log"
type => "json"
start_position => beginning
ignore_older => 0
}
}

filter{
json{
source => "message"
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => "localhost:9200"
index => "logstash-json"
}
}

I get the below output in cmd promt,

C:\logstash-2.3.4\logstash-2.3.4\bin>logstash -f C:\logstash-2.3.4\logstash-2.3.4\bin\logstash.conf
io/console not supported; tty will not be manipulated
Settings: Default pipeline workers: 4
Pipeline main started

After that it is supposed to show me the index created rite. But nothing happens after that.

2

it may be https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_tracking_of_current_position_in_watched_files

3

I tried

  1. sincedb_path => null
  2. Manually deleted the sincedb file

Still the same situation.

The index is not getting created.

4

What about https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-ignore_older

5

Can you try adding action => "index" to your output section. By specifying this line under hosts should work.

6

Why? That is the default action anyway.

7

just curious, can you put the hosts in array something like ["localhost:9200"] and remove manually specifying of index. Let logstash create its own index and then the meantime have a watch at both elasticsearch and logstash logs simultaneously .

8

Added Path value as array,
path => ["/logs/exampledata_01.json"]

sincedb value as string
sincedb_path => "null"

Configuration is fine, but index is not creating,

C:\logstash-2.3.4\bin>logstash agent -f C:\logstash-2.3.4\bin\exampleConf.conf
io/console not supported; tty will not be manipulated
Settings: Default pipeline workers: 4
Pipeline main started

9

Ya tried as you suggested,

output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => ["localhost:9200"]
}
}

Still no index creation,

C:\logstash-2.3.4\bin>logstash agent -f C:\logstash-2.3.4\bin\exampleConf.conf
io/console not supported; tty will not be manipulated
Settings: Default pipeline workers: 4
Pipeline main started

Stops after this.

10

Tried adding as you suggested,

output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => ["localhost:9200"]
action => "index"
index => "exampleIndex"
}
}

Nothing happens after this,

C:\logstash-2.3.4\bin>logstash agent -f C:\logstash-2.3.4\bin\exampleConf.conf
io/console not supported; tty will not be manipulated
Settings: Default pipeline workers: 4
Pipeline main started

And how can I overcome the "io/console not supported; tty will not be manipulated" message. ?

11

When you executed your conf file, were you able to see the output written to console (stdout). If you are able to see the output then it should create a index as per your config file. If not, try updating the log file by adding new lines and see.

12

What does ur ES log show? And can you start logstash with debug and see if you find something unusual.

13

I was getting "high disk watermark [90%] exceeded on" warning.

I made space in the disk and the warning is rectified.

I am unable to run the logstash in debug mode, I am getting some UsageError.

14

No I do not see the output on my console.

What new lines I should add and into which file ?

I even tried the example log and Conf files (first-pipeline.conf) provided in the Logstash setting page.

I don't see anything after "Pipeline main started".

15

I am able to create the index with the input as stdin.

But when I feed the input through some log files, the index is not created.

I think it is something related to sincedb, but don't know what is the exact issue.!!!

16

sincedb_path => "null"

No, that does not disable sincedb. Use "nul" on Windows and "/dev/null" on all other platforms.

This thread is impossible to follow. If you still haven't resolved this please post your current configuration.

Until you've gotten the file input to read files properly you should disable the elasticsearch output and only keep the stdout output. I also suggest you enable verbose Logstash logging by starting it with --debug. Look for log lines containing "discover" and post them here.

17

Hi All,

Thanks for your support. I got the issue resolved.
Had some issue with the input file Path. Gave full file path and it worked fine.