Define default index pattern from a yml file

Elastic Stack Kibana
1

Hi All
we are running ELK (version 8.7.0) over docker containers.
Whenever we are doing a restart to those containers (manually or due to some failure) the index pattern we defined in the kibana is getting reset and the next time we log in to the kibana we have to set it again

i'm trying to figure out if there is some configuration we can define in our kibana.yml \ elasticsearch.yml files that will automatically set the initial index pattern

when consulting chat gpt it suggested defining in the elasticsearch.yml:

# Elasticsearch Configuration
cluster.name: my_cluster
node.name: node-1
network.host: 0.0.0.0
discovery.type: single-node

# Custom Index Pattern
index_patterns:
  - name: all_indices
    pattern: "*-*"

and in the kibana.yml:

# Kibana Configuration
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://elasticsearch:9200"]

# Custom Index Pattern
kibana.index: ".kibana"
kibana.defaultAppId: "discover"
kibana.indexPatterns: '["all_indices"]'

however, this leads to an error when trying to run the kibana container since it won't recognize the kibana.indexPatterns configuration.

so, is there any way to do so?

2

You mean, the data views? This is stored in Elasticsearch, if it is gone every time you restart your containers something is not right in the way you are running it.

Are you using persistent storage, right? Please share your docker-compose.yml.

None of those settins exists, Chat GPT is unreliable as it can hallucinate and invent settings and commands that do not exist, I do not recommend applying any setting suggested by chat gpt without checking the documentation, depend on what you change it may break your cluster or lead to data loss.

3

The elastic part inside our docker-compose.yml :

version: '3.5'
services:
    elasticsearch:
        container_name: elasticsearch
        hostname: elasticsearch
        image: 'docker.elastic.co/elasticsearch/elasticsearch:8.7.0'
        restart: unless-stopped
        environment:
            - cluster.name=docker-cluster
            - bootstrap.memory_lock=true
        ulimits:
            memlock: {soft: -1, hard: -1}
        volumes:
            - '${NANOLOCK_HOME}/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:z'
            - '${NANOLOCK_HOME}/elasticsearch/jvm.options:/usr/share/elasticsearch/config/jvm.options:z'
            - '${NANOLOCK_HOME}/elasticsearch/data:/usr/share/elasticsearch/data:z'
            - '${NANOLOCK_HOME}/elasticsearch/logs:/usr/share/elasticsearch/logs:z'
        healthcheck:
            test: [CMD-SHELL, 'wget -q -O - http://localhost:9200 || exit 1']
            interval: 10m
            timeout: 60s  
        networks:
          nanolock:
            ipv4_address: XXX.XX.X.XXX

as you can see we have a volume with elasticsearch.yml file:

path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
network.host: 0.0.0.0
transport.host: 127.0.0.1
http.host: 0.0.0.0
script.allowed_types: inline
xpack.monitoring.templates.enabled: false
#xpack.monitoring.enabled: false
xpack.security.enabled: true
4

You need to share the entire docker-compose.yml, not just the elasticsearch part.

5

Hi @Amir_Dar ! Welcome to the community!

Advanced settings (including the default index pattern) can be customized in the following format in kibana.yml:

uiSettings:
  overrides:
    "defaultIndex": "<id here>"
    <... other settings>
7

thanks for the quick reply.
just to make sure I get it, does the "" mean the actual index pattern I want to use? for example, "logs-*"?

8

Not sure where Chat GPT came up with that...

Also word of Caution using ChatGPT specifically with Elasticsearch the Data Set it used in several years old and Elasticsearch has changed significantly since then, so use it with caution I would always cross-reference the suggestions against our official docs.

1 Like
9

I don't think this has any relation with your issue, you still need to share your entire docker-compose.yml to make it clear how you are running your stack.

10

If your were referring to my suggestion with

uiSettings:
  overrides:
    "defaultIndex": "<id here>"
    <... other settings>

then no, it should be id of Index Pattern saved object (or Data View saved object in recent kibana versions) and not a pattern itself.

11

How do we save index pattern\data view in order to see the ID, so we can use it as you suggested?

12

You can define and save it from Stack Management > Index Patterns (Data Views) page. The ID will be in the URL.

13

but won't this data be lost if i restart my kibana container?
it's not very presistent

1 Like
14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.