Kibana Index Pattern Configuration Problem

Elastic Stack Kibana
1

I configured Logstash as follows:

input {
file { path => "/var/log/apache2/access.log" }
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}


date {
    match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
}

mutate {
    convert => {
        "response" => "integer"
        "bytes" => "integer"
    }
}


}
output {
elasticsearch {
hosts => "localhost:9200"
index => "apache-%{+YYYY.MM.dd}"
}
}

I installed and configured Apache.
When I try to configure the index in Kibana, I get the following message: The index pattern I set does not match any data stream.

I don't know where the problem lies.
Can you please help me?

2

Hi @Pacous,

Welcome back! It looks like it's expecting data stream settings. Can you change your configuration to use the data stream settings, similar to the below example:

output {
  elasticsearch {
    hosts => ["https://your-elasticsearch-host:9200"]
    data_stream => true
    data_stream_type => "logs"         # or metrics, etc.
    data_stream_dataset => "app"
    data_stream_namespace => "default"
  }
}

Let us know if that works.

3

Hi Carly,

I applied the configuration you provided.
I still can't see the index on Kibana.

input {
file {
path => "/var/log/apache2/access.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter {
grok {
match => {
"message" => "%{COMBINEDAPACHELOG}"
}
}

date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}

}

output {
elasticsearch {
hosts => ["https://localhost:9200"]
index => "apache-access-%{+YYYY.MM.dd}"
data_stream => true
data_stream_type => "logs" # or metrics, etc.
data_stream_dataset => "app"
data_stream_namespace => "default"
}
}

4

Ok, can can share the output of the latest logs? Are there Ny changes/ new errors in there?

5

Hi Carly,

When i do the following command m,

sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/apache.conf -t

I have the following result,

[2026-06-19T13:47:25,624][INFO ][logstash.javapipeline ] Pipeline main is configured with pipeline.ecs_compatibility: v8 setting. All plugins in this pipeline will default to ecs_compatibility => v8 unless explicitly configured otherwise.
Configuration OK
[2026-06-19T13:47:25,837][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

When i do the following command,

sudo /usr/share/logstash/bin/logstash --debug --path.settings /etc/logstash -f /etc/logstash/conf.d/apache.conf

I have the following result,

[2026-06-19T13:58:00,883][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>"LogStash::ConfigurationError", :error=>"Invalid data stream configuration: ["index"]", :stacktrace=>"/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.15-java/lib/logstash/outputs/elasticsearch/data_stream_support.rb:76:in check_enabled_data_stream_config!'\n/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.15-java/lib/logstash/outputs/elasticsearch/data_stream_support.rb:59:in check_data_stream_config!'\n/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.15-java/lib/logstash/outputs/elasticsearch/data_stream_support.rb:32:in data_stream_config?'\n/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.15-java/lib/logstash/outputs/elasticsearch.rb:305:in register'\norg/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:69:in register'\n/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:243:in block in register_plugins'\norg/jruby/RubyArray.java:1981:in each'\n/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:242:in register_plugins'\n/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:610:in maybe_setup_out_plugins'\n/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:255:in start_workers'\n/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:196:in run'\n/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:148:in block in start'", "pipeline.sources"=>["/etc/logstash/conf.d/apache.conf"], :thread=>"#<Thread:0x17ecf057 /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:138 run>"}
[2026-06-19T13:58:00,895][INFO ][logstash.javapipeline ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2026-06-19T13:58:01,090][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil}
[2026-06-19T13:58:01,102][DEBUG][logstash.agent ] Shutting down all pipelines {:pipelines_count=>0}
[2026-06-19T13:58:01,126][DEBUG][logstash.agent ] Converging pipelines state {:actions_count=>0}
[2026-06-19T13:58:01,129][DEBUG][logstash.instrument.periodicpoller.os] Stopping
[2026-06-19T13:58:01,130][DEBUG][logstash.instrument.periodicpoller.jvm] Stopping
[2026-06-19T13:58:01,130][DEBUG][logstash.instrument.periodicpoller.persistentqueue] Stopping
[2026-06-19T13:58:01,134][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] Stopping
[2026-06-19T13:58:01,198][DEBUG][logstash.instrument.periodicpoller.flowrate] Stopping
[2026-06-19T13:58:01,236][DEBUG][logstash.agent ] API WebServer has stopped running
[2026-06-19T13:58:01,241][INFO ][logstash.runner ] Logstash shut down.
[2026-06-19T13:58:01,332][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:924)
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:883)
at usr.share.logstash.lib.bootstrap.environment.(/usr/share/logstash/lib/bootstrap/environment.rb:90)

6

Remove the index configuration from your output, you have set data_stream to true and all other data_stream_* settings, the index configuration cannot be used when data_stream is true.

Try changing to this:

output {
elasticsearch {
hosts => ["https://localhost:9200"]
data_stream => true
data_stream_type => "logs" # or metrics, etc.
data_stream_dataset => "app"
data_stream_namespace => "default"
}
}
7

What do I do in Kibana to view the logs?
Since I'm using the index, I configure the same index in Kibana that I configured in Logstash.

8

Data streams sit on top of indexes, so you view them in the same way. If you can't see your logs in Kibana discover it's because you need to create a data view on top.

9

I still don't see any logs, as you can see from the screenshot.

a
a1172×534 68.1 KB

10

I think these are completely different things.

In your original question you asked this:

How did you try to configure it and what is the name of the index you have in Elastic? Can you see your Index when you go into Stack management > Index management?

Please share some screenshots showing if you have any index starting with apache-*

11

You said to remove the index entry in Logstash.

Otherwise, here's the configuration I had:

output {
elasticsearch {
hosts => ["https://localhost:9200"]
index => "apache-access-%{+YYYY.MM.dd}"
data_stream => true
data_stream_type => "logs" # or metrics, etc.
data_stream_dataset => "app"
data_stream_namespace => "default"
}
}

12

This configuration will not work, you cannot have both index and data_stream => true at the same time.

If you had only index => "apache-access-%{+YYYY.MM.dd}" your log should go into an index named something like apache-2026-06-19

And in Kibana in need to create a Data View looking to apache-*.

If you use something like this:

data_stream => true
data_stream_type => "logs" # or metrics, etc.
data_stream_dataset => "app"
data_stream_namespace => "default"

Then your data will be on logs-app-default and your Data View needs to look at logs-app-*.

In your first question you had only the index configuration, with this did you had any apache-* index? After changing it to write into data stream did you had any logs-app-* index written?

You can check going into Stack Management > Index Management and looking both in the index and data streams tab.

13

Here is the result shown in the screenshot when I try to configure the index on Kibana:

b
b957×423 29.9 KB