# Logstash not creating index in kibana

**URL:** <https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780>\
**Category:** Logstash\
**Created:** [August 9, 2021, 9:00am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780 "2021-08-09T09:00:18Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 9, 2021, 9:00am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/1 "2021-08-09T09:00:18Z")

</div>

**This is my logstash plain log file**

```auto

[2021-06-18T08:21:45,675][INFO][logstash.runner] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2021-06-18T08:21:45,686][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.13.2", "jruby.version"=>"jruby 9.2.16.0 (2.5.7) 2021-03-03 f82228dc32 OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [linux-x86_64]"}
[2021-06-18T08:21:46,858][INFO][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/*.conf"}
[2021-06-18T08:21:46,883][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2021-06-18T08:21:47,116][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2021-06-18T08:21:52,198][INFO][logstash.runner] Logstash shut down.
[2021-06-18T08:21:52,216][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.16.0.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.16.0.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]

```

**This is my conf file**

```auto
input {
udp {
port => 5044
type => "syslog"

}

}
filter{
      if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{USERNAME:hostname}%{DATA:syslog_program}:%{GREEDYDATA:syslog_message}" }
}
date {
        match => ["timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
     }

ruby {
        code => "
            fieldArray = event.get('syslog_message').split(' ');
            for field in fieldArray
                name = field.split('=')[0];
                value = field.split('=')[1];
                 if value =~ /[0-9]/
                   if value =~ /^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/
                      event.set(name, value)
                   else
                      if value =~ /[A-z]/
                        event.set(name, value)
                      else
                        event.set(name, value.to_i)
                      end
                   end
                else
                    event.set(name, value)

                end
            end
        "
    }
}

}
output {
  elasticsearch {
ilm_enabled => false
    hosts => ['https://elastic.sys:9200']
    cacert => '/etc/logstash/certs/ca.crt'
    user => 'logstash_writer'
    password => 'LiL14stC0M0BZw9iBAY9'
      index => "syslog"

  }
  stdout { codec => rubydebug }
}

```

---

<div class="post-metadata">

**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\
**Post date:** [August 9, 2021, 9:15am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/2 "2021-08-09T09:15:31Z")

</div>

> [@kiran80511](#):
>
> ```auto
> No config files found in path {:path=>"/etc/logstash/conf.d/*.conf"}
> [2021-06-18T08:21:46,883][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
> 
> ```

Where is located your configuration file ?

Cad.

---

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 9, 2021, 9:22am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/3 "2021-08-09T09:22:05Z")

</div>

```auto
[root@elastic conf.d]# ls
example.bak syslog.bak syslog.conf
[root@elastic conf.d]#

```

and path define in pipelines.yml

```auto
- pipeline.id: main
 path.config: "/etc/logstash/conf.d/*.conf"

```

---

<div class="post-metadata">

**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\
**Post date:** [August 9, 2021, 9:25am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/4 "2021-08-09T09:25:59Z")

</div>

What is the permissions of the sysolg.conf file ?

---

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 9, 2021, 9:27am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/5 "2021-08-09T09:27:14Z")

</div>

```auto
-rw-r--r--. 1 root root 1323 Jul 22 14:36 syslog.conf

```

---

<div class="post-metadata">

**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\
**Post date:** [August 9, 2021, 9:50am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/6 "2021-08-09T09:50:41Z")

</div>

How did you run logstash ?  
Did you try to restart the service ?

---

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 9, 2021, 9:55am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/7 "2021-08-09T09:55:36Z")

</div>

`sudo service logstash start`  
Yes I have already restart the service.  
Logstash create index in kibana when I use HTTP but when I try ELK in HTTPS then not creating index in kibana.

---

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 10, 2021, 10:38am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/8 "2021-08-10T10:38:19Z")

</div>

**plz Any one can help ??**

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [August 10, 2021, 12:59pm UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/9 "2021-08-10T12:59:08Z")

</div>

Well . If it works with http but not https , then something should be logged in journal log.

Check log in elasticsearch. You may find some error related to certificate. Or check log for logstash.

---

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 11, 2021, 6:29am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/10 "2021-08-11T06:29:18Z")

</div>

I have check logs of Elasticsearch ,there is no any error and in logstash log file last log is 18 june 2021 after that log is not created in logstash.  
The log of logstash are

```auto
[2021-06-18T08:21:45,686][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.13.2", "jruby.version"=>"jruby 9.2.16.0 (2.5.7) 2021-03-03 f82228dc32 OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [linux-x86_64]"}
[2021-06-18T08:21:46,858][INFO][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/*.conf"}
[2021-06-18T08:21:46,883][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2021-06-18T08:21:47,116][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2021-06-18T08:21:52,198][INFO][logstash.runner] Logstash shut down.
[2021-06-18T08:21:52,216][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
	at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.16.0.jar:?]
	at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.16.0.jar:?]
	at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]

```

Plz help me.

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [August 11, 2021, 7:51am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/11 "2021-08-11T07:51:26Z")

</div>

> [@kiran80511](#):
>
> `/etc/logstash/conf.d/*.conf`

try

`ls -l /etc/logstash/conf.d/syslog.conf`

---

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 11, 2021, 8:18am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/12 "2021-08-11T08:18:02Z")

</div>

```auto
-rw-r--r--. 1 root root 1323 Jul 22 14:36 /etc/logstash/conf.d/syslog.conf

```

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [August 11, 2021, 8:37am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/13 "2021-08-11T08:37:14Z")

</div>

> [@kiran80511](#):
>
> ```auto
> - pipeline.id: main
> path.config: "/etc/logstash/conf.d/*.conf"
> 
> ```

Set this and restart `logstash` . See what happens.

```auto
- pipeline.id: main
 path.config: "/etc/logstash/conf.d/syslog.conf"

```

---

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 12, 2021, 5:39am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/14 "2021-08-12T05:39:02Z")

</div>

I have changed as you said and restart the logstash but as I said earlier that after 18 june 2021 logs is not created in logstash log file so unable to check the logs.

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [August 12, 2021, 7:01am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/15 "2021-08-12T07:01:48Z")

</div>

I'd like to know if this message is cleared or not.

```auto
[2021-06-18T08:21:46,883][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.

```

If you are saying these log is not updated **after the restart on Aug 12** , then something is wrong with your logstash installation. Next , you should check journal logs , e.g `journalctl -b 0`

```auto
[2021-06-18T08:21:45,675][INFO][logstash.runner] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2021-06-18T08:21:45,686][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.13.2", "jruby.version"=>"jruby 9.2.16.0 (2.5.7) 2021-03-03 f82228dc32 OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [linux-x86_64]"}
[2021-06-18T08:21:46,858][INFO][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/*.conf"}
[2021-06-18T08:21:46,883][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2021-06-18T08:21:47,116][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2021-06-18T08:21:52,198][INFO][logstash.runner] Logstash shut down.
[2021-06-18T08:21:52,216][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.16.0.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.16.0.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]

```

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [August 12, 2021, 7:04am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/16 "2021-08-12T07:04:15Z")

</div>

Also , you should restart logstash by `sudo service logstash restart` not `sudo service logstash start`. _Stop_ then _Start_ will be more solid instead of reloading the config.

```auto
sudo service logstash stop
sudo service logstash start

```

---

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 13, 2021, 6:49am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/17 "2021-08-13T06:49:13Z")

</div>

Output of `journalctl -b 0 -r`

```auto
Aug 13 12:11:55 elastic.sys logstash[1508]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will 
Aug 13 12:11:54 elastic.sys logstash[1508]: Using bundled JDK: /usr/share/logstash/jdk
Aug 13 12:11:54 elastic.sys systemd[1]: Started logstash.
Aug 13 12:11:54 elastic.sys systemd[1]: Stopped logstash.
Aug 13 12:11:54 elastic.sys systemd[1]: logstash.service holdoff time over, scheduling restart.
Aug 13 12:11:54 elastic.sys systemd[1]: logstash.service failed.
Aug 13 12:11:54 elastic.sys systemd[1]: Unit logstash.service entered failed state.
Aug 13 12:11:54 elastic.sys systemd[1]: logstash.service: main process exited, code=exited, status=1/FAILURE
Aug 13 12:11:54 elastic.sys logstash[1445]: at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environmen
Aug 13 12:11:54 elastic.sys logstash[1445]: at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.16.0.jar:?]
Aug 13 12:11:54 elastic.sys logstash[1445]: at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.16.0.jar:?]
Aug 13 12:11:54 elastic.sys logstash[1445]: org.jruby.exceptions.SystemExit: (SystemExit) exit
Aug 13 12:11:54 elastic.sys logstash[1445]: [2021-08-13T12:11:54,637][FATAL][org.logstash.Logstash] Logstash stopped processing because of 
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash --help
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash -V [--log.level fatal|error|warn|info|debug|trace]
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace]
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash -e CONFIG_STR [-t] [--log.level fatal|error|warn|info|debug|trace] [-w COUNT] [-l LOG]
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash --modules MODULE_NAME [-M "MODULE_NAME.var.PLUGIN_TYPE.PLUGIN_NAME.VARIABLE_NAME=VALUE
Aug 13 12:11:54 elastic.sys logstash[1445]: bin/logstash -f CONFIG_PATH [-t] [-r] [] [-w COUNT] [-l LOG]
Aug 13 12:11:54 elastic.sys logstash[1445]: usage:
Aug 13 12:11:54 elastic.sys logstash[1445]: ERROR: Failed to read pipelines yaml file. Location: /etc/logstash/pipelines.yml

```

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [August 13, 2021, 7:52am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/18 "2021-08-13T07:52:29Z")

</div>

> [@kiran80511](#):
>
> ```auto
> Aug 13 12:11:54 elastic.sys logstash[1445]: ERROR: Failed to read pipelines yaml file. Location: /etc/logstash/pipelines.yml
> 
> ```

Are you sure you have your logstash installed properly ? You do not see these errors if it's installed correctly.

```auto
Aug 13 12:11:54 elastic.sys logstash[1445]: ERROR: Failed to read pipelines yaml file. Location: /etc/logstash/pipelines.yml

```

---

<div class="post-metadata">

**Author:** ![kiran80511](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kiran80511/32/80487_2.png) [@kiran80511](https://discuss.elastic.co/u/kiran80511)\
**Post date:** [August 18, 2021, 5:53am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/19 "2021-08-18T05:53:24Z")

</div>

yes I have installed properly but if error comes then I want to uninstall logstash completely.Please suggest how to uninstall logstash in centos 7?

---

<div class="post-metadata">

**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\
**Post date:** [August 18, 2021, 12:58pm UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780/20 "2021-08-18T12:58:05Z")

</div>

> [@kiran80511](#):
>
> ```auto
> - pipeline.id: main
> path.config: "/etc/logstash/conf.d/*.conf"
> 
> ```

You have to put two space space before `path.config`. A single space also give me an error.

```auto
- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"

```

[Next page](https://discuss.elastic.co/t/logstash-not-creating-index-in-kibana/280780.md?page=2)