Logstash not creating index in kibana

This is my logstash plain log file


[2021-06-18T08:21:45,675][INFO ][logstash.runner          ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2021-06-18T08:21:45,686][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.13.2", "jruby.version"=>"jruby 9.2.16.0 (2.5.7) 2021-03-03 f82228dc32 OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [linux-x86_64]"}
[2021-06-18T08:21:46,858][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/*.conf"}
[2021-06-18T08:21:46,883][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2021-06-18T08:21:47,116][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2021-06-18T08:21:52,198][INFO ][logstash.runner          ] Logstash shut down.
[2021-06-18T08:21:52,216][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.16.0.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.16.0.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]

This is my conf file

input {
udp {
port => 5044
type => "syslog"


}

}
filter{
      if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{USERNAME:hostname}%{DATA:syslog_program}:%{GREEDYDATA:syslog_message}" }
}
date {
        match => [ "timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
     }

ruby {
        code => "
            fieldArray = event.get('syslog_message').split(' ');
            for field in fieldArray
                name = field.split('=')[0];
                value = field.split('=')[1];
                 if value =~ /[0-9]/
                   if value =~ /^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/
                      event.set(name, value)
                   else
                      if value =~ /[A-z]/
                        event.set(name, value)
                      else
                        event.set(name, value.to_i)
                      end
                   end
                else
                    event.set(name, value)

                end
            end
        "
    }
}

}
output {
  elasticsearch {
ilm_enabled => false
    hosts => ['https://elastic.sys:9200']
    cacert => '/etc/logstash/certs/ca.crt'
    user => 'logstash_writer'
    password => 'LiL14stC0M0BZw9iBAY9'
      index => "syslog"


  }
  stdout { codec => rubydebug }
}

Where is located your configuration file ?

Cad.

[root@elastic conf.d]# ls
example.bak  syslog.bak  syslog.conf
[root@elastic conf.d]#

and path define in pipelines.yml

- pipeline.id: main
 path.config: "/etc/logstash/conf.d/*.conf"

What is the permissions of the sysolg.conf file ?

-rw-r--r--. 1 root root 1323 Jul 22 14:36 syslog.conf

How did you run logstash ?
Did you try to restart the service ?

sudo service logstash start
Yes I have already restart the service.
Logstash create index in kibana when I use HTTP but when I try ELK in HTTPS then not creating index in kibana.

plz Any one can help ??

Well . If it works with http but not https , then something should be logged in journal log.

Check log in elasticsearch. You may find some error related to certificate. Or check log for logstash.

I have check logs of Elasticsearch ,there is no any error and in logstash log file last log is 18 june 2021 after that log is not created in logstash.
The log of logstash are

[2021-06-18T08:21:45,686][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.13.2", "jruby.version"=>"jruby 9.2.16.0 (2.5.7) 2021-03-03 f82228dc32 OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [linux-x86_64]"}
[2021-06-18T08:21:46,858][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/*.conf"}
[2021-06-18T08:21:46,883][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2021-06-18T08:21:47,116][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2021-06-18T08:21:52,198][INFO ][logstash.runner          ] Logstash shut down.
[2021-06-18T08:21:52,216][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
	at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.16.0.jar:?]
	at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.16.0.jar:?]
	at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]

Plz help me.

try

ls -l /etc/logstash/conf.d/syslog.conf

-rw-r--r--. 1 root root 1323 Jul 22 14:36 /etc/logstash/conf.d/syslog.conf

Set this and restart logstash . See what happens.

- pipeline.id: main
 path.config: "/etc/logstash/conf.d/syslog.conf"

I have changed as you said and restart the logstash but as I said earlier that after 18 june 2021 logs is not created in logstash log file so unable to check the logs.

I'd like to know if this message is cleared or not.

[2021-06-18T08:21:46,883][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.

If you are saying these log is not updated after the restart on Aug 12 , then something is wrong with your logstash installation. Next , you should check journal logs , e.g journalctl -b 0

[2021-06-18T08:21:45,675][INFO ][logstash.runner          ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2021-06-18T08:21:45,686][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.13.2", "jruby.version"=>"jruby 9.2.16.0 (2.5.7) 2021-03-03 f82228dc32 OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [linux-x86_64]"}
[2021-06-18T08:21:46,858][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/*.conf"}
[2021-06-18T08:21:46,883][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2021-06-18T08:21:47,116][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2021-06-18T08:21:52,198][INFO ][logstash.runner          ] Logstash shut down.
[2021-06-18T08:21:52,216][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.16.0.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.16.0.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]

Also , you should restart logstash by sudo service logstash restart not sudo service logstash start. Stop then Start will be more solid instead of reloading the config.

sudo service logstash stop
sudo service logstash start

Output of journalctl -b 0 -r

Aug 13 12:11:55 elastic.sys logstash[1508]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will 
Aug 13 12:11:54 elastic.sys logstash[1508]: Using bundled JDK: /usr/share/logstash/jdk
Aug 13 12:11:54 elastic.sys systemd[1]: Started logstash.
Aug 13 12:11:54 elastic.sys systemd[1]: Stopped logstash.
Aug 13 12:11:54 elastic.sys systemd[1]: logstash.service holdoff time over, scheduling restart.
Aug 13 12:11:54 elastic.sys systemd[1]: logstash.service failed.
Aug 13 12:11:54 elastic.sys systemd[1]: Unit logstash.service entered failed state.
Aug 13 12:11:54 elastic.sys systemd[1]: logstash.service: main process exited, code=exited, status=1/FAILURE
Aug 13 12:11:54 elastic.sys logstash[1445]: at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environmen
Aug 13 12:11:54 elastic.sys logstash[1445]: at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.16.0.jar:?]
Aug 13 12:11:54 elastic.sys logstash[1445]: at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.16.0.jar:?]
Aug 13 12:11:54 elastic.sys logstash[1445]: org.jruby.exceptions.SystemExit: (SystemExit) exit
Aug 13 12:11:54 elastic.sys logstash[1445]: [2021-08-13T12:11:54,637][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of 
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash --help
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash -V [--log.level fatal|error|warn|info|debug|trace]
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace]
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash -e CONFIG_STR [-t] [--log.level fatal|error|warn|info|debug|trace] [-w COUNT] [-l LOG]
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash --modules MODULE_NAME [-M "MODULE_NAME.var.PLUGIN_TYPE.PLUGIN_NAME.VARIABLE_NAME=VALUE
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash -f CONFIG_PATH [-t] [-r] [] [-w COUNT] [-l LOG]
Aug 13 12:11:54 elastic.sys logstash[1445]: usage:
Aug 13 12:11:54 elastic.sys logstash[1445]: ERROR: Failed to read pipelines yaml file. Location: /etc/logstash/pipelines.yml

Are you sure you have your logstash installed properly ? You do not see these errors if it's installed correctly.

Aug 13 12:11:54 elastic.sys logstash[1445]: ERROR: Failed to read pipelines yaml file. Location: /etc/logstash/pipelines.yml

yes I have installed properly but if error comes then I want to uninstall logstash completely.Please suggest how to uninstall logstash in centos 7?

You have to put two space space before path.config. A single space also give me an error.

- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"