Kibana Indices Privilege query

[pk.241011]

Hi,

I have created a user to be used for creating the monthly logs in Elasticsearch cluster we have.

I just want this user to create a new index every month and push data into it. I do not want it to update or delete index and documents.

Keeping this in mind I gave this user create_index (for creating indices) and create privilege (to index documents). I skipped index privilege because it also allows user to update documents.

This is not working. It is not creating index and putting data into the cluster.

However when I give it write privilege along with create_index and create privilege, it works !!!
I do not want to give it write privilege because it includes permission to index, update, and delete documents as well as performing bulk operations.

I think I am making some silly mistake here. Can experts point out what privilege i should give?

_Additional information: The data is sent from the application by the serilog elasticsearch sink. And it looks like that it invokes Bulk API to push data.

This is old link but here is what it says:

Write : Privilege to perform all write operations on documents, including the ability to index, update, and delete documents as well as perform bulk operations. If write is granted on the .scripts index, it includes the ability to put and delete indexed scripts.

Looks like Bulk API operations does come with Write privilege only. Will like experts to chip in.

More information:
I am able to put an index into the cluster via curl:

curl -u TonyStark:Pepper143 -XPUT 'http://StarkServer9200/TopSecret-01-2017.09/Coffee/1?pretty' -H 'Content-Type: application/json' -d' {"user" : "kimchy","post_date" : "2009-11-15T14:12:12","message" : "Blah"}'

Elasticsearch version: 5.2.1
Kibana : 5.2.1

I am using serilog elasticsearch sink to push data from my application to the elasticsearch instance.

[pk.241011]

I think I found why !!
As per this link : "Create, index, and delete requests are write operations".

So does that mean if write privilege is not given then create_index and create privilege do not hold?

And the issue remains. This user can create indexes, push documents in. This user cannot delete indexes but can delete documents in them.

[Brandon_Kobel]

@pk.241011 you were initially correct in assigning the create_index and create privileges, and I've verified that those two privileges allow the user to create an index and index documents. Are you able to verify the behavior you're seeing using curl?

[pk.241011]

Hi @Brandon_Kobel,

I just confirmed. I kept the permissions to create and create_index. Through the application it failed. But on command line via curl it succeeded.

curl -u tony:pepper123 -XPUT 'http://starkindustry:9200/ironman-01-2017.03/today/2?pretty' -H 'Content-Type: application/json' -d' {"user" : "kimchy","post_date" : "2009-11-15T14:12:12","message" : "trying out Elasticsearch"}'

More context (Though not sure if it helps since it works when I add write privilege to the user):
Our serilog app setting are like this:
<add key="serilog:write-to:Elasticsearch.nodeUris" value="http://tony:pepper123@starkindustry:9200">
<add key="serilog:write-to:Elasticsearch.indexFormat" value="ironman-01-{0:yyyy.MM}" />

This means that depending on the month and year it will keep creating indexes in the format ironman-01-2017.03

This is the regex I have kept in indices field for this particular user to that he does not get access to any system level indices:
/.*-20[0-9][0-9]\.[0-1][0-9].*/ to match all the indices like blah-2017.03

[pk.241011]

@Brandon_Kobel I just added more information to the question. Just to summarise, the sink is sending data to Elasticsearch via Bulk API. Looks like create and create_index do not include the bulk data upload privileges. Just wanted to know if my guess is right. And if so then are there any workarounds other than giving write privileges.

[Brandon_Kobel]

@pk.241011 you are correct that the bulk operations require the write privilege, so if serilog is using the bulk apis that user will require the write privilege.