Kibana 7.7.0 Basic version: management tab missing Security panel when started from docker

Elastic Stack Elasticsearch
21

Sorry, I didn't understand your suggestion. If I set up indices as read only how can Logsstash write on it? Above it, I am pretty sure if I remove xpack.security logstash will save normally. I can give a try increasing disk space but it is strange to me since there is almost no data save on such elasticsearch and it works without xpack.

22

PS.: I can execute your second command using elastic user.

>docker exec -it dockercomposelogs_logstash_1 bash
bash-4.2$ curl -X POST -H "Content-Type: application/json" http://192.168.99.100:9200/logstash-test/_doc/1 -d'{"test":1}' -u elastic:e12345 -vvv
* About to connect() to 192.168.99.100 port 9200 (#0)
*   Trying 192.168.99.100...
* Connected to 192.168.99.100 (192.168.99.100) port 9200 (#0)
* Server auth using Basic with user 'elastic'
> POST /logstash-test/_doc/1 HTTP/1.1
> Authorization: Basic ZWxhc3RpYzplMTIzNDU=
> User-Agent: curl/7.29.0
> Host: 192.168.99.100:9200
> Accept: */*
> Content-Type: application/json
> Content-Length: 10
>
* upload completely sent off: 10 out of 10 bytes
23

Please, compare last email working properly with Docker Machine IP address and elastic user with this failling when I simply replace the IP Address by container alias.

bash-4.2$ curl -X POST -H "Content-Type: application/json" http://elasticsearch:9200/logstash-test/_doc/1 -d'{"test":2}' -u elastic:e12345 -vvv
* About to connect() to elasticsearch port 9200 (#0)
*   Trying 172.18.0.2...
* Connected to elasticsearch (172.18.0.2) port 9200 (#0)
* Server auth using Basic with user 'elastic'
> POST /logstash-test/_doc/1 HTTP/1.1
> Authorization: Basic ZWxhc3RpYzplMTIzNDU=
> User-Agent: curl/7.29.0
> Host: elasticsearch:9200
> Accept: */*
> Content-Type: application/json
> Content-Length: 10
>
* upload completely sent off: 10 out of 10 bytes
< HTTP/1.1 429 Too Many Requests
< content-type: application/json; charset=UTF-8
< content-length: 319
<
* Connection #0 to host elasticsearch left intact
{"error":{"root_cause":[{"type":"cluster_block_exception","reason":"index [logstash-test] blocked by: [TOO_MANY_REQUESTS/12/index read-only / allow delete (api)];"}],"type":"cluster_block_exception","reason":"index [logstash-test] blocked by: [TOO_MANY_REQUESTS/12/index read-only / allow delete (api)];"},"status":429}bash-4.2$
24

Kindly, do you know any trick to force logstash to use ip address? I created this question

25

Hello @jimisdrpc,

I can help but we're encountering several problems.

If your docker is using a single volume both for data and logs, the disk space can fill up quite fast. This is not necessarily related to the indices but, for example, logs.

This is for sure due to the fact Elasticsearch has not enough disk free.
You can check the cat API cat nodes API | Elasticsearch Guide [8.11] | Elastic or any other monitoring errors.

I encourage to not cross post with stack overflow... Now where should we follow up?

Please:

  1. Ensure you have enough disk space on Elasticsearch containers
  2. If you're on Elasticsearch 7.4+ if the disk space is freed, the index block will be removed (Disk-based shard allocation | Elasticsearch Guide [8.11] | Elastic)
  3. Test again the curl requests to ensure the document is written.
  4. Check if the Logstash pipeline configuration is misconfigured as the last logs shown a problem [2020-05-21T12:41:16,538][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \t\r\n], "#", "input", "filter", "output" at line 1, column 1 (byte 1)", :backtrace=> - this is a usual error if you've edited the file using notepad. Ensure the file encoding is correct.

Once we're at this point, we can easily resume investigating any error.

26

I missed the question about using the up instead of the hostnames. You're using docker and the best approach is to use the service names instead of IPs.

27

Thank you so much. Honestly, you have already answered the topic question. Now I can see Security panel. I will restart from beggining and post a new question if necessary. As a tip for future readers, there is indeed some limetation because I am using Docker Toolbox with Virtual Box instead of Docker For Windows with HyperV. It is not the first time I face some weird connection issue. Thanks again

28

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.