Index is not detting generated

Elastic Stack Logstash
1

Index is not getting generated . Please find the following configurations related to the issue. I am not getting any error in logstash logs.
Elasticsearch - 6.2.2
Logstash - 6.2.2
logstash.conf

input {
file
{
path =>["/log/X/Y/", "/log/P/Q/"]
tags => ["Test"]
}
}

filter {

if "Test" in [tags]
{
grok {
match => { "message" => "(%{TIMESTAMP_ISO8601:timestamp}),%{NUMBER:milisecond} - iims.services.PolicyServiceHandler - INFO - UserCode %{DATA:UserId} | PolicyNumber %{NUMBER:PolicyNumber} | Premium %{NUMBER:Premium} | productcode %{WORD:ProductCode}" }
remove_field => [ "message" , "milisecond" ]
}

date {
    match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss" ]
    timezone => "Asia/Kolkata"
     }
	 
mutate {
     convert => { "Premium" => "integer" }
     }
	 
translate
    {
    field => "UserId"
    destination => "User"
    dictionary_path => "/etc/logstash/userdictionary.yml"
    fallback => "Others"
    exact => "true"
    regex => "true"
    }
	
}

}

output {

if "Test" in [tags]
{
elasticsearch {
hosts => [ "elkmonpt1.newindia.co.in:9200" ]
index => "niacorepremium-%{+YYYY.MM.dd}"
user => logxxxx
password => TYYYY
}
}

}

Logs I am trying to parse is :

2019-02-21 17:04:19,567 - iifs.services.PolicyServiceHandler - INFO - UserCode AG_MNJSPQ | PolicyNumber 45210331180100010129 | Premium 1361 | productcode XY

Thanks for the help.

2

Your grok pattern does not match your data. Your pattern has iims.services. and your data has iifs.services.

Are you saying the index does not get created or that the data in the index does not have the fields that you want?

3

iifs.services in the data is modified by me only when I created the topic here. so, it is not the issue. My data is matching with my pattern. It is already checked on the grok debugger. My Index is just not getting created that is the issue. If grok pattern is different then logstash gives some error in logs. But in my case my pipeline is running successfully.
The path from which I am fetching the data is like,

path => ["/log/folder1/X/ ", "/log/folder1/Y/ "]

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.