Error "redirected too many time" in Kibana 5.0.2


(Tat Dat Pham) #1

Hi, i'm install 5.0.2 (ES,LS,Kibana) for testing.

I checked cluster status, it is green but when i start Kibana 5.0.2, i got error

And log when start Kibana

I try to install 5.0.1, Elastic and Kibana are working!.


Endless loop while logging to Kibana with X-pack
"No endpoint or operation is available at [_xpack]"
(Lee Drengenberg) #2

Hi,

Are you running Kibana 5.0.1 and 5.0.2 against the same version of Elasticsearch? Which version is it?

And do you have X-Pack installed on both version of Kibana as well as Elasticsearch?

Thanks,
Lee


(Lee Drengenberg) #3

Also, someone else reported this problem possibly related to CentOS kernel 3.10. And did not appear on later kernel version 3.19 through 4.8. Can you let us know what your OS/kernel version is?

Thanks,
Lee


(Tat Dat Pham) #4

No, i used Kibana 5.0.1 for ES 5.0.1 (x-pack 5.0.1) and Kibana 5.0.2 for ES 5.0.2 (x-pack 5.0.2)

Also, someone else reported this problem possibly related to CentOS kernel 3.10. And did not appear on later kernel version 3.19 through 4.8. Can you let us know what your OS/kernel version is?

I used Kernel 2.6. Today, i create new Vituarl machine and installed centos 7, upgrade Kernel to 4.8.12.-1.e17. After that, i try to install kibana 5.0.2, x-pack 5.0.2 but got same error ...


(Lee Drengenberg) #5

If you want to try again with 5.0.2, please make sure Elasticsearch is completely started before starting Kibana. There seems to be some issue with Kibana with X-Pack if it can't reach Elasticsearch when it starts and get the license check completed successfully.

There will be a new release coming out soon that addresses this problem, but I can't promise when. So if the steps above don't work on 5.0.2 for you, and 5.0.1 does work for you, you might have to stay with that until the next release comes out.

Regards,
Lee


(Lee Drengenberg) #6

I just got word of another potential way to fix this problem.

Someone else reported;
" I got them working by having them set server.host in kibana.yml to the FQDN to match how their users are accessing Kibana (using FQDN and not IP) (instead of setting server.host to IP),"

Let is know if you try this and it helps or not.

Thanks,
Lee


(Tat Dat Pham) #7

If you want to try again with 5.0.2, please make sure Elasticsearch is completely started before starting Kibana. There seems to be some issue with Kibana with X-Pack if it can't reach Elasticsearch when it starts and get the license check completed successfully.

Yes, i checked and healthy status is green.

Someone else reported;
" I got them working by having them set server.host in kibana.yml to the FQDN to match how their users are accessing Kibana (using FQDN and not IP) (instead of setting server.host to IP),"

It not work for me :slight_smile:


(Niall) #8

Hi,

I am also having this issue with Kibana 5.1.1 running on Ubuntu 16.04. I have tried the things you have suggested but nothing seems to help.

The logs show that kibana is just sending out 301 responses to any URL it is given.

It is probably also worth noting that I do have X-Pack installed and my cluster was up and green when I started Kibana.


(Lee Drengenberg) #9

Hi Phạm and Niall,

Are you both installing the packages (.deb or .rpm) and starting the services? In the first screenshot from Phạm it looks like you're installing the package but then running it as the root user. While that should work, I'm just trying to figure out what you're doing differently than me.

Here's the steps I just ran with 5.1.1 and Kibana is up and working for me.

# all of this as root user
# cleanup steps
 service kibana stop
 service elasticsearch stop
 service elasticsearch stop
 apt-get purge -y kibana
 apt-get purge -y elasticsearch
 rm -rf /usr/share/kibana
 rm -rf /usr/share/elasticsearch
 rm -rf /etc/kibana
 rm -rf /etc/elasticsearch
 rm -rf /var/lib/kibana
 rm -rf /var/lib/elasticsearch

# I'm currently on 32-bit Ubuntu so use the right builds for your platform 
 wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.1.deb
 wget https://artifacts.elastic.co/downloads/kibana/kibana-5.1.1-i386.deb
 dpkg -i elasticsearch-5.1.1.deb 
 dpkg -i kibana-5.1.1-i386.deb 

# -b for the x-pack install is batch mode with no prompt
 /usr/share/elasticsearch/bin/elasticsearch-plugin install -b x-pack

# this takes pretty long.  Make sure it completes successfully
 /usr/share/kibana/bin/kibana-plugin install x-pack

 service elasticsearch start
 service kibana start

Next I open my browser to http://localhost:5601
I get the login screen
Login as user elastic, password changeme
It worked and redirected me to http://localhost:5601/app/kibana#/management/kibana/index/?_g=() because I don't yet have a default index pattern created in Kibana.

I logged out and back in. I restarted Kibana service and logged back in. Everything is working for me.

Please let me know what, if anything, you think is different. I've done this on CentOS 7 64-bit and Ubuntu 64-bit in the past so it doesn't seem like that's the problem.

If I leave Kibana in my browser and logged in, and I service elasticsearch stop, then click logout in Kibana, then I get;
The page isn’t redirecting properly Firefox has detected that the server is redirecting the request for this address in a way that will never complete.

Then I service elasticsearch start, and keep refreshing the browser. After about 1 minute, the refresh works and I'm back on the login page and can successfully log in again.

So based on that, it would seem that your Kibana isn't reaching your Elasticsearch, or has some kind of unusual connection issue.

Or its a difference between running the services vs. downloading the tar.gz archives, unzipping, and running them that way. I'll go through those steps to see if I can reproduce.

Regards,
Lee


(Lee Drengenberg) #10

I did the same steps again except used the .tar.gz archives. Again, I used 32-bit kibana in this case.

cd ~/Downloads/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.1.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.1.1-linux-x86.tar.gz
tar -xvf elasticsearch-5.1.1.tar.gz 
tar -xvf kibana-5.1.1-linux-x86.tar.gz 
./elasticsearch-5.1.1/bin/elasticsearch-plugin install x-pack
./kibana-5.1.1-linux-x86/bin/kibana-plugin install x-pack
./elasticsearch-5.1.1/bin/elasticsearch

# in another shell;
./kibana-5.1.1-linux-x86/bin/kibana

I was able to login and out of Kibana.
I stopped and restarted Kibana and was able to login and out of Kibana.
If I stop elasticsearch, then in Kibana I get The page isn’t redirecting properly.
But when I start elasticsearch again, within about 20 seconds I can refresh Kibana and log in again.

Regards,
Lee


(Niall) #11

Hi Lee,

Thanks for the extensive support which has been very helpful. Unfortunately I was still experiencing the error after doing everything detailed above.

I eventually got Kibana working again after some more digging. It turns out that there is a bug in elasticsearch 5.1.1 that causes it to bind to an IPv6 socket using an IPv4 address.

I had all of my nodes setup to use static IPs resolved from the local hosts file. After upgrading from 5.0.1 to 5.1.1 when running netstat -l it shows that despite the fact that elasticsearch is binding to the correct hostname/IP the socket type is tcp6.

I fixed the issue by completely disabling IPv6 on all of the nodes and then restarting each node. After doing this the cluster came back up and Kibana started working again.

Disabling IPv6 may not always be a viable option in a production environment however, so this bug could cause issues for some people.

Best Regards,
Niall


(Lee Drengenberg) #12

Hi Niall,

Could you please check this and see if it explains what you're seeing regarding
".. bind to an IPv6 socket using an IPv4 address .."

Regards,
Lee


(Jason Tedor) #13

Would you please explain what you mean by this?


(Tat Dat Pham) #14

Hi,

This problem fixed in newest version (Kibana 5.1.1.)
Thank for support :slight_smile:


(system) #15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.