Unauthorized logstash role for bulk writes

Elastic Stack Logstash
1

Here are my current role that is attached to the user logstash

Screen Shot 2017-03-20 at 5.09.42 PM.png1092×862 50.9 KB

I know create_index and create aren't necessary but I figured I would give that at try.

I had originally followed this guide https://www.elastic.co/guide/en/x-pack/current/logstash.html but it makes no mention of how to handle this. I thought having the write permission would be enough.

2

If you try to do that manually from cURL using the same user, does it manage to create the index?

curl --user logstash:YourPasswordHere -XPUT 'localhost:9200/error-test?pretty' -H 'Content-Type: application/json' -d'
{
"settings" : {
"index" : {
"number_of_shards" : 3,
"number_of_replicas" : 2
}
}
}
'

3

When I run that command with my credentials and cluster url instead of localhost

{
  "acknowledged" : true,
  "shards_acknowledged" : true
}

If I go to Kibana dev console

GET error-test
{
  "error-test": {
    "aliases": {},
    "mappings": {},
    "settings": {
      "index": {
        "creation_date": "1490208027717",
        "number_of_shards": "3",
        "number_of_replicas": "2",
        "uuid": <someuuid>,
        "version": {
          "created": "5010299"
        },
        "provided_name": "error-test"
      }
    }
  }
}

I've been able to send logs that did NOT use the bulk API for a while now.

4

Hmm, that's weird. If there was an issue with the permissions, it should not have been able to auth and create the index via cURL.
Can you paste your Logstash .conf file?
Also, it would be nice to have the auditing log event for that specific "access_denied".
This is how you enable auditing.
https://www.elastic.co/guide/en/x-pack/current/auditing.html

5

Correct me if I'm wrong, but I'm under the impression that auditing cannot be enabled when using Elastic cloud. Whenever I add the following, I am unable to save my configurations due to the "Illegal user settings" error message.

xpack:
  security:
    audit:
      enabled: true

curl  --user <username>:<password> https://<clusterid>.us-east-1.aws.found.io:9243/error-test/logs/_bulk?pretty=True -XPOST -d '
{"index": {}}
{"title": "Two", "tags": ["ruby", "python"] }
{"index": {}}
{"title": "Three", "tags": ["java"] }
{"index": {}}
{"title": "Four", "tags": ["ruby", "php"] }
'
{
  "took" : 23,
  "errors" : false,
  "items" : [
    {
      "index" : {
        "_index" : "error-test",
        "_type" : "logs",
        "_id" : "AVr3y1VO7ApZg6ubVQzj",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 3,
          "successful" : 1,
          "failed" : 0
        },
        "created" : true,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "error-test",
        "_type" : "logs",
        "_id" : "AVr3y1VO7ApZg6ubVQzk",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 3,
          "successful" : 1,
          "failed" : 0
        },
        "created" : true,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "error-test",
        "_type" : "logs",
        "_id" : "AVr3y1VO7ApZg6ubVQzl",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 3,
          "successful" : 1,
          "failed" : 0
        },
        "created" : true,
        "status" : 201
      }
    }
  ]
}

I'm throughly confused. If I curl a bulk request, I seem to have no issue submitting those writes despite the issue I'm having in the OP.

6

This is quite strange. You seem to be doing everything correctly, but something isn't working right.

To help with debugging can you try the following and provide the results:

    curl --user logstash "https://<clusterid>.us-east-1.aws.found.io:9243/_xpack/security/_authenticate?pretty"

    curl --user elastic "https://<clusterid>.us-east-1.aws.found.io:9243/_xpack/security/role/<your-logstash-role-name>?pretty"
7

Running the first command:

{
  "username" : "logstash",
  "roles" : [
    "logstash_writer",
    "logstash_reader"
  ],
  "full_name" : "Internal Logstash User",
  "email" : "someemail@email.com",
  "metadata" : { },
  "enabled" : true
}

Running the second command

{
  "logstash_writer" : {
    "cluster" : [
      "manage_index_templates",
      "monitor"
    ],
    "indices" : [
      {
        "names" : [
          "info-*",
          "debug-*",
          "critical-*",
          "error-*",
          "warn-*"
        ],
        "privileges" : [
          "write",
          "delete",
          "create_index",
          "create"
        ]
      }
    ],
    "run_as" : [ ],
    "metadata" : { }
  }
}

Looks like what I've put in.

8

Is it perhaps possible to disable bulk from logstash?

9

No, write grants bulk and once it's granted it isn't supposed to be possible to disable it.

I'll try and find some spare time to dig in and see if I can work out what's going on here, because you seem to be doing everything correctly.

Can you tell me the exact version of elasticsearch that you're running? If you don't know, then you can access it from the root page of your cluster (curl --user elastic "https://<clusterid>.us-east-1.aws.found.io:9243/").

10

I am using 5.1.2 of Elasticsearch on Elastic Cloud.

11

For fun I went back just now and tried running curl --user elastic:<password> "https://<clusterid>.us-east-1.aws.found.io:9243/
along with curl --user logstash:<password> "https://<clusterid>.us-east-1.aws.found.io/9243" and have found that both commands hangs whenever I am experiencing this issue that I had in a previous thread I posted. It seems that issue still is occurring on my end and I have not been able to get the error to consistently occur.

In addition, if I run a command like curl -L google.com (-L for redirecting since omitting the -L will just result in a 301) I receive html as expected.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.