Kibana 7.7.0 Basic version: management tab missing Security panel when started from docker

[jimisdrpc (Jimis)]

Sorry, I didn't understand your suggestion. If I set up indices as read only how can Logsstash write on it? Above it, I am pretty sure if I remove xpack.security logstash will save normally. I can give a try increasing disk space but it is strange to me since there is almost no data save on such elasticsearch and it works without xpack.

[jimisdrpc (Jimis)]

PS.: I can execute your second command using elastic user.

>docker exec -it dockercomposelogs_logstash_1 bash
bash-4.2$ curl -X POST -H "Content-Type: application/json" http://192.168.99.100:9200/logstash-test/_doc/1 -d'{"test":1}' -u elastic:e12345 -vvv
* About to connect() to 192.168.99.100 port 9200 (#0)
*   Trying 192.168.99.100...
* Connected to 192.168.99.100 (192.168.99.100) port 9200 (#0)
* Server auth using Basic with user 'elastic'
> POST /logstash-test/_doc/1 HTTP/1.1
> Authorization: Basic ZWxhc3RpYzplMTIzNDU=
> User-Agent: curl/7.29.0
> Host: 192.168.99.100:9200
> Accept: */*
> Content-Type: application/json
> Content-Length: 10
>
* upload completely sent off: 10 out of 10 bytes

[jimisdrpc (Jimis)]

Please, compare last email working properly with Docker Machine IP address and elastic user with this failling when I simply replace the IP Address by container alias.

bash-4.2$ curl -X POST -H "Content-Type: application/json" http://elasticsearch:9200/logstash-test/_doc/1 -d'{"test":2}' -u elastic:e12345 -vvv
* About to connect() to elasticsearch port 9200 (#0)
*   Trying 172.18.0.2...
* Connected to elasticsearch (172.18.0.2) port 9200 (#0)
* Server auth using Basic with user 'elastic'
> POST /logstash-test/_doc/1 HTTP/1.1
> Authorization: Basic ZWxhc3RpYzplMTIzNDU=
> User-Agent: curl/7.29.0
> Host: elasticsearch:9200
> Accept: */*
> Content-Type: application/json
> Content-Length: 10
>
* upload completely sent off: 10 out of 10 bytes
< HTTP/1.1 429 Too Many Requests
< content-type: application/json; charset=UTF-8
< content-length: 319
<
* Connection #0 to host elasticsearch left intact
{"error":{"root_cause":[{"type":"cluster_block_exception","reason":"index [logstash-test] blocked by: [TOO_MANY_REQUESTS/12/index read-only / allow delete (api)];"}],"type":"cluster_block_exception","reason":"index [logstash-test] blocked by: [TOO_MANY_REQUESTS/12/index read-only / allow delete (api)];"},"status":429}bash-4.2$

[jimisdrpc (Jimis)]

Kindly, do you know any trick to force logstash to use ip address? I created this question

How to force LogStash to connect using Ip Address instead of DNS names

In my local environment, I must use IP Address instead of Localhost or other nickname. My settup is Docker over Virtual Box instead of Docker For Windows. It is not part of my question discuss why I am using Virtual Box instead of HyperV but only for superficial explanation I use also Minishift for other projects (free OpenShift) and it depends on Virtualbox. I have noted that I must use Docker Machine IP address in some places that most of Hello World tutorials just use localhost, 127.0.0.1 or…

[Luca_Belluccini]

Hello @jimisdrpc,

I can help but we're encountering several problems.

If your docker is using a single volume both for data and logs, the disk space can fill up quite fast. This is not necessarily related to the indices but, for example, logs.

@jimisdrpc

TOO_MANY_REQUESTS/12/index read-only / allow delete (api)]

This is for sure due to the fact Elasticsearch has not enough disk free.
You can check the cat API cat nodes API | Elasticsearch Guide [8.11] | Elastic or any other monitoring errors.

@jimisdrpc

Kindly, do you know any trick to force logstash to use ip address? I created this question

I encourage to not cross post with stack overflow... Now where should we follow up?

Please:

1. Ensure you have enough disk space on Elasticsearch containers
2. If you're on Elasticsearch 7.4+ if the disk space is freed, the index block will be removed (Disk-based shard allocation | Elasticsearch Guide [8.11] | Elastic)
3. Test again the curl requests to ensure the document is written.
4. Check if the Logstash pipeline configuration is misconfigured as the last logs shown a problem [2020-05-21T12:41:16,538][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \t\r\n], "#", "input", "filter", "output" at line 1, column 1 (byte 1)", :backtrace=> - this is a usual error if you've edited the file using notepad. Ensure the file encoding is correct.

Once we're at this point, we can easily resume investigating any error.

[Luca_Belluccini]

I missed the question about using the up instead of the hostnames. You're using docker and the best approach is to use the service names instead of IPs.

[jimisdrpc (Jimis)]

Thank you so much. Honestly, you have already answered the topic question. Now I can see Security panel. I will restart from beggining and post a new question if necessary. As a tip for future readers, there is indeed some limetation because I am using Docker Toolbox with Virtual Box instead of Docker For Windows with HyperV. It is not the first time I face some weird connection issue. Thanks again