Can I upgrade my existing ELK setup to 8.x?

[Ravi_Pattar]

@stephenb

If you do not have it try

./bin/elasticsearch-reset-password -u elastic

Hello,

I have tried with the reset password and it look like the util for the same in not present.

root@ELK-test:/usr/share/elasticsearch/bin# ./bin/elasticsearch-reset-password -u elastic
-bash: ./bin/elasticsearch-reset-password: No such file or directory
root@ELK-test:/usr/share/elasticsearch/bin# pwd
/usr/share/elasticsearch/bin

root@ELK-test:/usr/share/elasticsearch/bin# ls -l
total 3616
-rwxr-xr-x 1 root root    2985 Aug 31  2023 elasticsearch
-rwxr-xr-x 1 root root     501 Aug 31  2023 elasticsearch-certgen
-rwxr-xr-x 1 root root     493 Aug 31  2023 elasticsearch-certutil
-rwxr-xr-x 1 root root     996 Aug 31  2023 elasticsearch-cli
-rwxr-xr-x 1 root root     443 Aug 31  2023 elasticsearch-croneval
-rwxr-xr-x 1 root root    5496 Aug 31  2023 elasticsearch-env
-rwxr-xr-x 1 root root    1828 Aug 31  2023 elasticsearch-env-from-file
-rwxr-xr-x 1 root root     168 Aug 31  2023 elasticsearch-geoip
-rwxr-xr-x 1 root root     184 Aug 31  2023 elasticsearch-keystore
-rwxr-xr-x 1 root root     450 Aug 31  2023 elasticsearch-migrate
-rwxr-xr-x 1 root root     126 Aug 31  2023 elasticsearch-node
-rwxr-xr-x 1 root root     176 Aug 31  2023 elasticsearch-plugin
-rwxr-xr-x 1 root root     441 Aug 31  2023 elasticsearch-saml-metadata
-rwxr-xr-x 1 root root     439 Aug 31  2023 elasticsearch-service-tokens
-rwxr-xr-x 1 root root     448 Aug 31  2023 elasticsearch-setup-passwords
-rwxr-xr-x 1 root root     118 Aug 31  2023 elasticsearch-shard
-rwxr-xr-x 1 root root     483 Aug 31  2023 elasticsearch-sql-cli
-rwxr-xr-x 1 root root 3600932 Aug 31  2023 elasticsearch-sql-cli-7.17.13.jar
-rwxr-xr-x 1 root root     436 Aug 31  2023 elasticsearch-syskeygen
-rwxr-xr-x 1 root root     436 Aug 31  2023 elasticsearch-users
-rwxr-xr-x 1 root root     332 Aug 31  2023 systemd-entrypoint
-rwxr-xr-x 1 root root     356 Aug 31  2023 x-pack-env
-rwxr-xr-x 1 root root     364 Aug 31  2023 x-pack-security-env
-rwxr-xr-x 1 root root     363 Aug 31  2023 x-pack-watcher-env
root@ELK-test:/usr/share/elasticsearch/bin# 

Thanks,
Ravi

[stephenb (Stephen Brown)]

Ok sorry you did not make it clear that you are still on 7.17... I thought you upgraded

I am confused where you are in the process I feel like I am guessing what you are trying to do..

For 7.17

./bin/elasticsearch-setup-passwords auto

If you were trying to upgrade @dadoonet provided the docs

I am not sure what you are trying to accomplish

[Ravi_Pattar]

@stephenb

Ok sorry you did not make it clear that you are still on 7.17... I thought you upgraded

I am confused where you are in the process I feel like I am guessing what you are trying to do..

For 7.17

./bin/elasticsearch-setup-passwords auto

If you were trying to upgrade @dadoonet provided the docs

I am not sure what you are trying to accomplish

I am sorry if there was any confusion. I am into the process of upgrading from the existing 7.17 to 8.x. Hence I am facing these issues at the 1st step of the upgrade assistant > fix the depreciation issues.

With the latest command on setting up the password which has worked now. e.g. as below.

curl -u elastic:password http://localhost:9200

root@ELK-test:~# curl -u elastic:FnbEUnD0Q0jj6385Z0MO http://localhost:9200
{
  "name" : "ELK-test",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "8HvYlQx1TiGPOkGwFqVVgQ",
  "version" : {
    "number" : "7.17.13",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "2b211dbb8bfdecaf7f5b44d356bdfe54b1050c13",
    "build_date" : "2023-08-31T17:33:19.958690787Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Thanks,
Ravi

[stephenb (Stephen Brown)]

You should have got a new password for kibana_system now put that in kibana.yml

And restart Kibana

[Ravi_Pattar]

@stephenb

You should have got a new password for kibana_system now put that in kibana.yml

And restart Kibana

Yes, I have got a new password and after restarting the kibana service I could see the authorization is working now as it prompted me for the username/password when I accessed the kibana GUI. Now, I am able to get in for the GUI by using the login credentials of elastic.

With the above I am also able to see that the critical error under the Elasticsearch depreciation issues also have got cleared and now I am working on clearing the 2nd one from the list.

2. Warning

Disk watermarks do not treat single-node clusters differently in versions 8.0 and later.

Disk watermarks do not treat single-node clusters differently in versions 8.0 and later, which will affect the behavior of this cluster. Set [cluster.routing.allocation.disk.watermark.enable_for_single_data_node] to [true] to adopt the future behavior before upgrading. (nodes impacted: [ELK-test, ELK-test])

Would request you to please provide the docs on upgrading from 7.17 to 8.11.

Also the purpose of this upgrade plan on the main ELK production server is because we were seeing filebeat 8.x indices from the other client servers were being pushed and the ILM implemented was for filebeat 7.17 on the main ELK.

Hence the client requested to check if we can upgrade the main ELK to the 8.x so that the issue can be resolved instead of downgrading the other client servers to 7.17.

Just for reference on the above filebeat 8.x issue reported Question regarding filebeat indexes for version 8.11.x and ILM - #8 by Ravi_Pattar

Thanks,
Ravi

[stephenb (Stephen Brown)]

Hi @Ravi_Pattar

Upgrade to Elastic 8.12.2 | Elastic Installation and Upgrade Guide [8.12] |...

I don't know that we can step you through every step.

Keep working through the upgrade assistant.

Take the advice like the watermark advice set that setting.

If you have a production system and at this important, perhaps you should consider a paid support or some consulting.

Also in general we don't recommend running single node clusters for production You are at risk for data loss if there's a failure on that node... But that's a different discussion.

[Ravi_Pattar]

Hi @stephenb

Thank you and I understood on the points mentioned above. Sure, will discuss this internally and suggest the team. Also, will work on the upgrade assistant and will create a new topic in case of any issues faced.

Thank you very much for your support.

Thanks,
Ravi

[stephenb (Stephen Brown)]

Yes, come back and ask more questions...

The other thing perhaps to do is take a snapshot of your data. If you're concerned, that would be a good task to reduce the risk.

[Ravi_Pattar]

Hello,

After the last changes and updates where basic security was enabled and its working fine.

But we have hit another problem where the logs in Discover tab are not shown after this changes.

Just fyi that I created another topic on the forum.

No logs for current timestamp shown in Disover tab

Hello, I have verified the configurations and I am not getting where is the missing bit. Because I am still facing issues with the logstash because I couldn't see logs showing up in the Discover tab on the Kibana GUI for the latest. input { beats { port => 5044 } } output { elasticsearch { hosts => [ "localhost:9200" ] user => elastic password => wakfgtqwJYKgYdYsKkE8 } } To verify on the username and password mentioned above. I tried to run the curl com…

Thanks,
Ravi