Conf configuration

[Krishna_Pagar]

Hi,

I have created conf file with below details, but whenever I am searching in Kibana with type - auth_inbound_web I can see all logs with only 7 days. I need that for 30 days.

There are 2 Index namely -
logstash-test-prd0-30-auth-%{+YYYY.MM.dd} ----- which shows logs for 30 days
logstash-test-prd0-7-auth-%{+YYYY.MM.dd} ----- which shows logs for 7 days

[root@123]# cat output.conf
output {
if "test_inbound_web" in [type]
{
elasticsearch
{
hosts => ["testchesclientnode.anr53p.co.uk:9200"]
manage_template => false
index => "logstash-test-prd0-30-auth-%{+YYYY.MM.dd}"
}
}
else if "offduty" in [type] {
elasticsearch {
hosts => ["testchesclientnode.anr53p.co.uk:9200"]
manage_template => false
index => "logstash-test-prd0-7-test-%{+YYYY.MM.dd}"
}
}
}

=============================================
[root@123]# cat input.conf
input {
file {
path => "/logs/apache/test_secure_inbound.log"
type => "test_inbound_web"
tags => [ "test", "web", "apache", "access", "test", "test_secure_inbound", "apache_combined_timings", "prd0" ]
}
file {
path => "/logs/apache/offduty_test_secure_inbound.log"
type => "test_inbound_web"
tags => [ "test", "web", "apache", "access", "offduty","test", "test_secure_inbound", "apache_combined_timings", "prd0" ]
}
}
}
filter {
if "apache_combined_timings" in [tags] {
grok {
match => [ "message", "%{COMBINEDAPACHELOG:log} %{NUMBER:response_time} "%{DATA:api_transaction_id}" "%{DATA:apigw_authenticated_client}"" ]
}
}
}

Thanks,

[Krishna_Pagar]

can someone help me in this ?

Thanks

[dadoonet (David Pilato)]

Read this and specifically the "Also be patient" part.

It's fine to answer on your own thread after 2 or 3 days (not including weekends) if you don't have an answer.

I moved your question to #logstash.

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

[Badger]

I would agree with everything David said, but the logstash configuration does not matter. It does not affect how long data is retained in elasticsearch. Are you using ILM? Are you using curator?

[Krishna_Pagar]

It's logstash agent which push logs to Kibana, but when I check in Kibana it showing only 7 days logs which I need for 30 days... can you please help me and confirm whether output / input file is correct one ?

Thanks,

[Badger]

What are you using to delete old logs from elasticsearch?