Logstash runs forever but no index got created

Elastic Stack Logstash
1

Hi All,

I'm using the below configuration in logstash to populate the data on Elasticsearch 7.17.11 from logstash 7.17.12.

It shows pipeline started but no index got created .

input {
    jdbc {
        jdbc_driver_library => "/usr/share/logstash/logstash-core/lib/jars/postgresql-jdbc.jar"
        jdbc_connection_string => "jdbc:postgresql://<DBHOST>:5432/itsm_db"
        jdbc_user => "si_dev"
        jdbc_password => "Yh"
        jdbc_driver_class => "org.postgresql.Driver"
        tracking_column => "incident_number"
        schedule => "0 * * * *" # cronjob schedule format (see "Helpful Links")
        statement => "SELECT incident_number, site_id, sub_site_id, account_id, sub_account_id, closed_at, state, short_description, description, incident_parent_id from customerdata_incident"
        jdbc_paging_enabled => "true"
        jdbc_page_size => "300"
    }
}

filter {
	mutate {
		copy => {"id" => "[@metadata][_id]"}
                remove_field => ["@timestamp", "@version"]
	}
}
output {
	stdout { codec => "json" }
	elasticsearch {
		hosts => ["https://<elasticsearch fqdn>:443"]
                ssl => true
                ssl_certificate_verification => true
                cacert => "/etc/ssl/ca_dev_elasticsearch.cer"
                ecs_compatibility => disabled
		user => "elasticsearch-admin"
		password => "cr"
		index => "logstash_itsm_incident"
                ilm_enabled => true
	}
}

i ran it with debug mode. it shows the logs as follows.

[DEBUG] 2023-08-23 06:11:59.041 [pool-7-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:11:59.041 [pool-7-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:03.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:03.813 [pool-7-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:04.054 [pool-7-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:04.054 [pool-7-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:08.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:08.821 [pool-4-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:09.080 [pool-4-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:09.080 [pool-4-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:13.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:13.822 [pool-7-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:14.087 [pool-4-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:14.087 [pool-4-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:18.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:18.824 [pool-7-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:19.094 [pool-4-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:19.095 [pool-4-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:23.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:23.828 [pool-4-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:24.102 [pool-7-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:24.106 [pool-7-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:28.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:28.833 [pool-7-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:29.113 [pool-7-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:29.114 [pool-7-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:33.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:33.838 [pool-4-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:34.120 [pool-7-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:34.121 [pool-7-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:38.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:38.841 [pool-7-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:39.130 [pool-7-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:39.131 [pool-7-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:43.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:43.843 [pool-7-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:44.138 [pool-7-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:44.138 [pool-7-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2023-08-23 06:12:48.531 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2023-08-23 06:12:48.846 [pool-7-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2023-08-23 06:12:49.145 [pool-4-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2023-08-23 06:12:49.145 [pool-4-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.