Issues with ELK

Hi All,
I have the template siem_alarm created in my kibana index pattern but i have this error "Error: No indices match pattern "siem_alarms" at url/bundles/commons.bundle.js:3:1337196"

Does anyone know what could be responsible for this.
thanks
pero

Hi @pero. What part of Kibana were you accessing when this error came up? Can you run GET /_cat/indices in your Kibana Dev Tools and share the results?

Hi @nickpeihl,
Am getting this from when I click on "index pattern, followed by me selecting the specific pattern then I click on the fresh Icon which then open up the 'refresh field list box' then click on refresh button, this will popup an 'error fetching fields with a 'see the full error' button.
if I run 'GET /_cat/indices' I get

  "error": "Incorrect HTTP method for uri [/_cat/indices?pretty] and method [POST], allowed: [GET]",
  "status": 405

In which version are you? Can you share a screenshot of that page?

This is an weird error for something in Kibana to make a POST request for _cat/indices.

Hi @nickpeihl /@leandrojmp,

Here are screenshots that can be helpful






when I run 'GET /_cat/indices' on the Kibana Dev Tools this is what I get





Thanks for the help

And where is this from? You didn't share anything like this.

Also, from what you shared you do not have any indice named siem_alarms anymore, you have an indice named siem_events-*, but I do not see any siem_alarms.

Please run GET _cat/indices again and share the result but as plain text using the preformatted text button, the </> button.

The first index pattern you see as default is 'siem_alarms' but when I run GET _cat/indices I do not see the 'siem_alarms' and 'siem_alarm_events-*'

Hi @leandrojmp,
when I run GET /_cat/indices it get this

green  open siem_events-2023.07.05                                  CiM3kFJfSPi_2kisxalL8Q 1 0 288433   0 100.6mb 100.6mb
green  open auditbeat-7.4.1-2023.06.30                              PUMFLxGIQkmVBc6_S6GrxQ 1 0 334040   0  84.2mb  84.2mb
green  open auditbeat-7.4.1-2023.06.27                              O6-LaJ9bSQ2FkbCWldNbPA 1 0 297846   0  70.6mb  70.6mb
green  open auditbeat-7.4.1-2023.06.26                              beqNST1_QSGkQO4iYtppzQ 1 0  63423   0    16mb    16mb
green  open auditbeat-7.4.1-2023.06.25                              PJgbYcnGR1GIn1lcjVtwNA 1 0      0   0    283b    283b
yellow open %{[@metadata][beat]}-%{[@metadata][version]}-2023.06.29 gWoZN_IuShiwjoWtI43HRA 1 1      6   0 226.6kb 226.6kb
green  open auditbeat-2023.07.05                                    -gKf1Cq2QlCZe3iMP982JA 1 0 332985   0   176mb   176mb
green  open auditbeat-2023.07.04                                    R_-I9uO7RV-lSHYsgl3rsA 1 0 406238   0 230.4mb 230.4mb
green  open siem_events-2023.07.01                                  YH15ewWYQnusW4SYQtXklg 1 0 323862   0  92.1mb  92.1mb
yellow open %{[@metadata][beat]}-%{[@metadata][version]}-2023.06.27 EOyW0xuURlm1KpO7crAeZA 1 1      4   0 110.4kb 110.4kb
green  open siem_events-2023.07.02                                  KhJkRX3ySNyqnGlWEzehPw 1 0 317982   0  90.4mb  90.4mb
green  open .kibana_task_manager_1                                  02WrViHKT_KVpoL1UfR2Rw 1 0      2   0  13.5kb  13.5kb
green  open siem_events-2023.07.03                                  jYLTD6v1SVKmJHt42WLFSQ 1 0 311485   0  88.9mb  88.9mb
green  open siem_events-2023.07.04                                  DfGozML6RY-hLYnDgpBZvg 1 0 219720   0  62.9mb  62.9mb
yellow open %{[@metadata][beat]}-%{[@metadata][version]}-2023.06.26 admFJphwTDqBBxiDHoJgNg 1 1      9   0    56kb    56kb
green  open auditbeat-2023.06.30                                    ct1Cq917TEC1gjsRNqWaXA 1 0 599131   0 324.8mb 324.8mb
green  open auditbeat-2023.07.03                                    CQH4f02jRoek6hEj7mgS4w 1 0 565842   0 313.9mb 313.9mb
green  open auditbeat-2023.07.02                                    PBsPBlrcS1yFoFGqH5-sTw 1 0 576752   0   314mb   314mb
green  open siem_events-2023.06.30                                  Btjlox98QOuOEBmPWOm_hg 1 0 334040   0  95.7mb  95.7mb
green  open auditbeat-2023.07.01                                    LWF5MzgZQdyVUFJWKYsK7A 1 0 586212   0   316mb   316mb
yellow open %{[@metadata][beat]}-%{[@metadata][version]}-2023.05.30 70KyapVHRpa0VhzfJjbDqA 1 1      2   0 127.6kb 127.6kb
green  open auditbeat-7.4.1-2023.06.29                              eVHINF0WTiiKgx8LNjSa1Q 1 0 323585   0  81.9mb  81.9mb
green  open auditbeat-7.4.1-2023.06.28                              1C4RviUwQr-2BUMqVgAfJg 1 0 468926   0 114.8mb 114.8mb
green  open siem_events-2023.06.29                                  S0mjQNVrTiKUUtd6IKddog 1 0 323585   0  92.9mb  92.9mb
green  open siem_events-2023.06.28                                  lLICYO8IQX2lVHxmq0hXMg 1 0 468926   0 133.7mb 133.7mb
green  open siem_events-2023.06.27                                  5BEn2fZ7RZugDeP0Zls2Fw 1 0 297846   0  83.4mb  83.4mb
green  open siem_events-2023.06.26                                  rswSpbDPS8ub4Ku49UbEXw 1 0  63423   0  18.3mb  18.3mb
green  open .apm-agent-configuration                                mMiItdNzTHOG9d49CNMtjQ 1 0      0   0    283b    283b
green  open .kibana_1                                               5wZ9F7aVRq-Vj0ENYTX_HA 1 0   1031 101   512kb   512kb
green  open auditbeat-2023.06.27                                    eQ_rvQ8pRrCRpd0tloQBbw 1 0 348909   0   179mb   179mb
green  open auditbeat-2023.06.28                                    e1QzxPWrSZ-lPX1GvzjMoQ 1 0 615016   0 338.4mb 338.4mb
green  open auditbeat-7.4.1-2023.07.05                              gK1Xi86LRL2RsqlgxFUEuQ 1 0 288419   0  70.5mb  70.5mb
green  open filebeat-2023.06.23                                     N_iFrRl3Rdal9EWB-OT7sw 1 0   1706   0 765.2kb 765.2kb
green  open auditbeat-7.4.1-2023.07.04                              a6rjMzrzSeyhmMz628ziDQ 1 0 219720   0  55.9mb  55.9mb
green  open auditbeat-2023.06.25                                    7tgoi0FvRmOSGfxgpaDqDQ 1 0  12316   0   7.3mb   7.3mb
green  open auditbeat-2023.06.26                                    cPyoN8ovRAm_1bZKP1P6Yg 1 0 520260   0 266.4mb 266.4mb
green  open auditbeat-7.4.1-2023.07.03                              k54x8kpPQz-UeaU4msiTVg 1 0 311485   0  78.3mb  78.3mb
green  open auditbeat-7.4.1-2023.07.02                              CGGczPy9SVyqXMneR-Fy4Q 1 0 317982   0  79.4mb  79.4mb
green  open auditbeat-7.4.1-2023.07.01                              RvFYqN4YTqiYhZGNMs9HOA 1 0 323862   0    81mb    81mb
green  open auditbeat-2023.06.29                                    ulJ1y7ipQuK0fj8hg3i78w 1 0 572598   0   321mb   321mb
green  open siem_events-2023.06.25                                  tQQVKlXwQiadk_cmK-vyoQ 1 0      0   0    283b    283b
yellow open %{[@metadata][beat]}-%{[@metadata][version]}-2023.07.03 fPMo-M5vQXy_GNaD40iyDw 1 1      5   0 363.8kb 363.8kb
green  open filebeat-7.4.0-2023.06.23-000001                        O_j3Rof0STKkHwR6qCEV9A 1 0      0   0    283b    283b
green  open siem_events-2023.06.22                                  p59TvpB4Q5W3uXOZi-oNmg 1 0 192714   0  55.1mb  55.1mb
yellow open %{[@metadata][beat]}-%{[@metadata][version]}-2023.06.30 BHHsslugSv6QI8HgA69qOg 1 1      3   0 120.5kb 120.5kb
green  open auditbeat-2023.06.23                                    8EDAy-K0RgWo1iDy4Y30xg 1 0 520740   0 297.2mb 297.2mb
green  open auditbeat-2023.06.24                                    E1BaHTRFRti1_4JE7LBQDA 1 0 338259   0 199.2mb 199.2mb
yellow open localhost;9200                                          Bq81Ji1fRCmeM2MKtGvzTA 1 1      0   0    283b    283b

This means that someone with access to your Elasticsearch cluster created the index pattern siem_alarms and set it as the default index pattern when there was at least one index that matched that index pattern.

For some reason you do not have any index that matches this index pattern anymore, but only you can answer why, maybe someone deleted it or those indices are not being created anymore, or it was an alias that it is not present on your data anymore, you will need to investigate what happened.

Do you have an index pattern for the indices siem_events-*? If not you just need to create one and set it as default.

Also, 7.4 is pretty old, you should look into update to 7.17 and after date upgrade to 8.8.2.

Hi @leandrojmp, @nickpeihl
this was an error from something else please. so ignore it
["error": "Incorrect HTTP method for uri [/_cat/indices?pretty] and method [POST], allowed: [GET]",
"status": 405"]

No problem.

But as I said, there is no issue in Elasticsearch in this case.

You just have a default index pattern and your cluster does not have any indices that matches that index pattern anymore, so the message is expected.

Hi @leandrojmp,

I have the index pattern for the 'siem_alarms and siem_alarm_events created in elasticsearch and I can actually see the logs of this two patterns. Am very new to ELK. And am trying to integrate it with Dsiem

Can you show where you are seeing it? From what you shared you do not have any index named siem_alarms, so you do not have any index that matches that index pattern.

@nickpeihl, @leandrojmp,

The logs from both the 'siem_alarms and siem_alarm_events' are in the dsiem application.


![alarm_events|574x207](upload://1RLLAsm6Vkzsmgo2nX3N4xIokmP.jpeg)


![alarm_logs1|690x446](upload://3IIiezv5Ou5nO47ml7Gf6hmR2iY.jpeg)

 
![logs|442x43](upload://vRvoYCZPiI1TT1qcTHbu1qEN4JT.jpeg)

As you can see, these are being index to elasticsearch I believe

I can't see the images and I do not know what is dsiem.

Also, do not tag people, there is no need, if they are in a conversation they will be already notified.

Hi @leandrojmp; @nickpeihl ,

here are the patterns using GET /_template/siem_alarms-*

{
  "siem_alarms-*" : {
    "order" : 0,
    "version" : 1,
    "index_patterns" : [
      "siem_alarms-*"
    ],
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "number_of_replicas" : "0",
        "refresh_interval" : "1s"
      }
    },
    "mappings" : {
      "dynamic_templates" : [
        {
          "string_as_keywords" : {
            "mapping" : {
              "norms" : false,
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              }
            },
            "match_mapping_type" : "string"
          }
        }
      ],
      "properties" : {
        "src_ips" : {
          "type" : "ip"
        },
        "dst_ips" : {
          "type" : "ip"
        }
      }
    },
    "aliases" : {
      "siem_alarms" : { },
      "siem_alarms_id_lookup" : { }
    }
  }
}

GET /_template/siem_alarms

{
  "siem_alarms" : {
    "order" : 0,
    "version" : 1,
    "index_patterns" : [
      "siem_alarms-*"
    ],
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "number_of_replicas" : "0",
        "refresh_interval" : "1s"
      }
    },
    "mappings" : {
      "dynamic_templates" : [
        {
          "strings_as_keywords" : {
            "mapping" : {
              "norms" : false,
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              }
            },
            "match_mapping_type" : "string"
          }
        }
      ],
      "properties" : {
        "src_ips" : {
          "type" : "ip"
        },
        "dst_ips" : {
          "type" : "ip"
        }
      }
    },
    "aliases" : {
      "siem_alarms" : { },
      "siem_alarms_id_lookup" : { }
    }
  }
}

GET /_template/siem_alarm_events

{
  "siem_alarm_events" : {
    "order" : 0,
    "version" : 1,
    "index_patterns" : [
      "siem_alarm_events-*"
    ],
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "number_of_replicas" : "0",
        "refresh_interval" : "1s"
      }
    },
    "mappings" : {
      "dynamic_templates" : [
        {
          "strings_as_keywords" : {
            "mapping" : {
              "norms" : false,
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              }
            },
            "match_mapping_type" : "string"
          }
        }
      ]
    },
    "aliases" : { }
  }
}

However I can see index patterns GET /_template/siem_alarms-* and GET /_template/siem_alarms
could that be the cause of the problem because I only need one

Dsiem is a correlation engine like Ossim

No, this is not the cause of the problem, the cause of the problem was already explained, you do not have any indice in your cluster that match your index pattern.

Those templates will be applied to any index named siem_alarms-* like siem_alarms-2023.07, and they have an alias to siem_alarms.

So if you have an index named siem_alarms-2023.07.05 the index pattern will match this indice because of the alias.

But you do not have any index named siem_alarms-* in Elasticsearch.

If you have this data in other tool you need to explain how you get the data from that other tool and send them to Elasticsearch, because currently you do not have this data in Elasticsearch.

So the main issue is, how do you get data from Dsiem into Elasticsearch? What tool do you use? This processos is probably not working anymore.

Also, please do not tag people as already asked.

will look at my logstash pipeline then. because am using logstash to send my data from filebeat.

I have it resolved. Am really grateful for the quick response from you all