How do you set default index with x-pack installed?

mathias · November 24, 2017, 11:26am

Hi,

Without X-Pack I can set default index with the following curl command:
curl --verbose -POST http://kibana:5601/api/saved_objects/index-pattern -H "kbn-version: $KIBANA_VERSION" -H 'Content-Type: application/json;charset=UTF-8' -H 'User-Agent: Mozilla/5.0' -H 'Accept: Application/json, text/plain, /' -H 'DNT:1' -d '{"attributes":{"title":"events-*","timeFieldName":"@timestamp","notExpandable":true}}'

I tried the same with X-Pack installed and it does not work.

I also tried the following but it does not change the index:

curl --verbose -XPOST http://user:pwd@kibana:5601/api/kibana/settings/defaultIndex -H 'kbn-version:5.6.3' -H 'User-Agent: Mozilla/5.0' -H 'Content-Type: application/json' -H 'kbn-name:kibana' -d '{"value":"events-*"}'

Note: Unnecessary use of -X or --request, POST is already inferred.

Trying 10.68.32.177...

TCP_NODELAY set

Connected to kibana (10.68.32.177) port 5601 (#0)

Server auth using Basic with user 'ericsson'

POST /api/kibana/settings/defaultIndex HTTP/1.1
Host: kibana:5601
Authorization: Basic ZXJpY3Nzb246ZXJpY3Nzb24=
Accept: /
kbn-version:5.6.3
User-Agent: Mozilla
Content-Type: application/json
kbn-name:kibana
Content-Length: 20

upload completely sent off: 20 out of 20 bytes
< HTTP/1.1 200 OK
< kbn-name: kibana
< kbn-version: 5.6.3
< kbn-xpack-sig: 1028c92db5fd0f241c63cccd5da55232
< content-type: application/json; charset=utf-8
< cache-control: no-cache
< content-length: 153
< Date: Fri, 24 Nov 2017 11:19:11 GMT
< Connection: keep-alive
<

Connection #0 to host kibana left intact
{"settings":{"defaultIndex":{"userValue":"events-*"},"xPackMonitoring:allowReport":{"userValue":false},"xPackMonitoring:showBanner":{"userValue":false}}}

How to use curl and change default-index?

Br Mathias

ppisljar · November 24, 2017, 12:27pm

does this help you ? https://www.elastic.co/guide/en/x-pack/current/http-clients.html

mathias · November 24, 2017, 12:33pm

Not really,

I have created a user with password which I can log into Kibana with.
I am creating documents and store them in elasticsearch using the same credentials.
An I can retrieve the documents as well.

What I can not do is to set the default index at startup of kibana.
Setting it manually via the gui works but that does not help me.
I want it configured at start.

Is it possible to curl in the default index?

What headers and data are required?
curl -XPOST http://user:pwd@kibana:5601/api/kibana/settings/defaultIndex ........

Anything else I need to do?

Kind regards
Mathias

ppisljar · November 24, 2017, 12:45pm

providing username and password with curl --user username:password also does not work ?

whats the response from

curl --verbose --user username:password -POST http://kibana:5601/api/saved_objects/index-pattern -H "kbn-version: $KIBANA_VERSION" -H 'Content-Type: application/json;charset=UTF-8' -H 'User-Agent: Mozilla/5.0' -H 'Accept: Application/json, text/plain, /' -H 'DNT:1' -d '{"attributes":{"title":"events-*","timeFieldName":"@timestamp","notExpandable":true}}'

?

mathias · November 24, 2017, 12:53pm

The command executes without problems:

Trying 10.68.32.177...
TCP_NODELAY set
Connected to kibana (10.68.32.177) port 5601 (#0)
Server auth using Basic with user 'ericsson'

POST /api/saved_objects/index-pattern HTTP/1.1
Host: kibana:5601
Authorization: Basic ZXJpY3Nzb246ZXJpY3Nzb24=
kbn-version: 5.6.3
Content-Type: application/json;charset=UTF-8
User-Agent: Mozilla/5.0
Accept: Application/json, text/plain, /
DNT:1
Content-Length: 85

upload completely sent off: 85 out of 85 bytes
< HTTP/1.1 200 OK
< kbn-name: kibana
< kbn-version: 5.6.3
< kbn-xpack-sig: 1028c92db5fd0f241c63cccd5da55232
< content-type: application/json; charset=utf-8
< cache-control: no-cache
< content-length: 148
< Date: Fri, 24 Nov 2017 12:47:46 GMT
< Connection: keep-alive
<
Connection #0 to host kibana left intact
{"id":"AV_uEc_V2io_oD0oo2If","type":"index-pattern","version":1,"attributes":{"title":"events-*","timeFieldName":"@timestamp","notExpandable":true}}

But when doublechecking in gui, I do not see that default index has updated to events-*.
Instead the old default index is no longer default:
I manually have to create a new one:

Br Mathias

ppisljar · November 24, 2017, 1:05pm

ok thats weird ... i'll ask somebody to look into this

chrisronline · November 27, 2017, 4:38pm

Hi @mathias,

I'm not sure I'm quite following.

I don't think there is any code in Kibana server that will automatically set a default index pattern. We do, however, have code in Kibana client that will do that.

Two sources:

After creating an index pattern in the UI: https://github.com/elastic/kibana/blob/6.0/src/core_plugins/kibana/public/management/sections/indices/create_index_pattern/create_index_pattern.js#L221
After loading Kibana when there is no default set, but an index pattern exists: https://github.com/elastic/kibana/blob/6.0/src/ui/public/index_patterns/route_setup/load_default.js#L36 (Keep in mind that this code path is only invoked if there is only one existing index pattern)

Are you sure that a default index pattern is created after this curl command, and not after loading Kibana? You can easily verify by removing all existing index patterns, running this curl command, then running:
curl -XGET localhost:5601/api/kibana/settings and see if you see the defaultIndex field.

Thanks,
Chris

mathias · November 28, 2017, 8:42am

Before I try to set default index it looks like this:

curl -s -XGET user:pass@kibana:5601/api/kibana/settings | jq .
{
"settings": {
"buildNum": {
"userValue": 15554
},
"xPackMonitoring:allowReport": {
"userValue": true
},
"xPackMonitoring:showBanner": {
"userValue": false
}
}
}

An index pattern exists:

curl --silent --head 'http://user:pass@localhost:5601/api/index_patterns/_fields_for_wildcard?pattern=events-*'
HTTP/1.1 200 OK
kbn-name: kibana
kbn-version: 5.6.3
kbn-xpack-sig: 1028c92db5fd0f241c63cccd5da55232
content-type: application/json; charset=utf-8
cache-control: no-cache
Date: Tue, 28 Nov 2017 08:23:42 GMT
Connection: keep-alive

The GUI looks like this:

Before loading the X-Pack I managed to set default index this way:
Maybe it was pure luck but it worked every time.

curl --verbose -POST http://user:pass@kibana:5601/api/saved_objects/index-pattern -H "kbn-version: $KIBANA_VERSION" -H 'Content-Type: application/json;charset=UTF-8' -H 'User-Agent: Mozilla/5.0' -H 'Accept: Application/json, text/plain, /' -H 'DNT:1' -d '{"attributes":{"title":"events-*","timeFieldName":"@timestamp","notExpandable":true}}'

After loading X-pack it looks like this:

curl --verbose -POST http://user:pass@kibana:5601/api/saved_objects/index-pattern -H "kbn-version: $KIBANA_VERSION" -H 'Content-Type: application/json;charset=UTF-8' -H 'User-Agent: Mozilla/5.0' -H 'Accept: Application/json, text/plain, /' -H 'DNT:1' -d '{"attributes":{"title":"events-*","timeFieldName":"@tim
estamp","notExpandable":true}}'

Trying 10.68.32.177...

TCP_NODELAY set

Connected to kibana (10.68.32.177) port 5601 (#0)

Server auth using Basic with user 'user'

POST /api/saved_objects/index-pattern HTTP/1.1
Host: kibana:5601
Authorization: Basic ZXJpY3Nzb246ZXJpY3Nzb24=
kbn-version: 5.6.3
Content-Type: application/json;charset=UTF-8
User-Agent: Mozilla/5.0
Accept: Application/json, text/plain, /
DNT:1
Content-Length: 85

upload completely sent off: 85 out of 85 bytes
< HTTP/1.1 200 OK
< kbn-name: kibana
< kbn-version: 5.6.3
< kbn-xpack-sig: 1028c92db5fd0f241c63cccd5da55232
< content-type: application/json; charset=utf-8
< cache-control: no-cache
< content-length: 148
< Date: Tue, 28 Nov 2017 08:32:21 GMT
< Connection: keep-alive
<

Connection #0 to host kibana left intact
{"id":"AWABwWq42io_oD0oyceB","type":"index-pattern","version":1,"attributes":{"title":"events-*","timeFieldName":"@timestamp","notExpandable":true}}/ #
/ #
/ #
/ #
/ #
/ # curl -s -XGET user:pass@kibana:5601/api/kibana/settings | jq .
{
"settings": {
"buildNum": {
"userValue": 15554
},
"xPackMonitoring:allowReport": {
"userValue": true
},
"xPackMonitoring:showBanner": {
"userValue": false
}
}
}

Another index pattern is created but no default index pattern.

I tried to see what the GUI send when setting default index pattern.

curl --verbose -XPOST http://user:pwd@kibana:5601/api/kibana/settings/defaultIndex -H 'kbn-version:5.6.3' -H 'User-Agent: Mozilla/5.0' -H 'Content-Type: application/json' -H 'kbn-name:kibana' -d '{"value":"events-*"}'
Note: Unnecessary use of -X or --request, POST is already inferred.

Trying 10.68.32.177...

TCP_NODELAY set

Connected to kibana (10.68.32.177) port 5601 (#0)

Server auth using Basic with user 'user'

POST /api/kibana/settings/defaultIndex HTTP/1.1
Host: kibana:5601
Authorization: Basic ZXJpY3Nzb246ZXJpY3Nzb24=
Accept: /
kbn-version:5.6.3
User-Agent: Mozilla/5.0
Content-Type: application/json
kbn-name:kibana
Content-Length: 20

upload completely sent off: 20 out of 20 bytes
< HTTP/1.1 200 OK
< kbn-name: kibana
< kbn-version: 5.6.3
< kbn-xpack-sig: 1028c92db5fd0f241c63cccd5da55232
< content-type: application/json; charset=utf-8
< cache-control: no-cache
< content-length: 183
< Date: Tue, 28 Nov 2017 08:39:25 GMT
< Connection: keep-alive
<

Connection #0 to host kibana left intact
{"settings":{"buildNum":{"userValue":15554},"xPackMonitoring:allowReport":{"userValue":true},"xPackMonitoring:showBanner":{"userValue":false},"defaultIndex":{"userValue":"events-"}}}/ #
/ #
/ #
/ #
/ #
/ # curl -s -XGET user:pass@kibana:5601/api/kibana/settings | jq .
{
"settings": {
"buildNum": {
"userValue": 15554
},
"xPackMonitoring:allowReport": {
"userValue": true
},
"xPackMonitoring:showBanner": {
"userValue": false
},
"defaultIndex": {
"userValue": "events-"
}
}
}

But the GUI looks the same. No index pattern is set:

Just noticed that after setting the default index in GUI the last curl command changes to an id instead of a name.

curl -s -XGETuser:pass@kibana:5601/api/kibana/settings | jq .
{
"settings": {
"buildNum": {
"userValue": 15554
},
"xPackMonitoring:allowReport": {
"userValue": true
},
"xPackMonitoring:showBanner": {
"userValue": false
},
"defaultIndex": {
"userValue": "AWABwWq42io_oD0oyceB"
}
}
}

Br Mathias

mathias · November 28, 2017, 10:43am

Without X-Pack it looks like this when I have a working default index set with the first curl command above:

curl -s -XGET kibana:5601/api/kibana/settings | jq .
{
"settings": {
"defaultIndex": {
"userValue": "events-*"
},
"metrics:max_buckets": {
"userValue": "300000"
}
}
}

Br Mathias

chrisronline · November 28, 2017, 4:02pm

Hi @mathias,

Which version of Kibana are you running in both situations, the one with X-Pack and the one without?

Thanks,
Chris

mathias · November 28, 2017, 7:21pm

Hi Chris,

Kibana 5.6.3 in both cases.

The only major difference is that I use an alpine docker image for the non x-pack case.
When using x-pack I was forced to use debian docker image.

Br Mathias

chrisronline · November 28, 2017, 8:39pm

Hi @mathias,

Sorry for the back and forth. I just want to make sure I'm using the right steps in reproducing this.

On a fresh install of 5.6.3, this is what I'm doing:

Starting ES
Starting Kibana (and not loading Kibana in the browser)
Verifying no default index pattern:

curl -XGET <kibana>/api/kibana/settings
-> 
{"settings":{"buildNum":{"userValue":8467}}}

Creating an index pattern:

curl -XPOST <kibana>/api/saved_objects/index-pattern -H 'kbn-version: 5.6.3' -H 'Content-Type: application/json' -d '{"attributes":{"title": "logstash-*", "timeFieldName": "@timestamp", "notExpandable": true}}'

Verifying it exists:

curl -XGET <kibana>/api/saved_objects/index-pattern
-> 
{"saved_objects":[{"id":"AWAEVmfI_cDw8Ov6vitD","type":"index-pattern","version":1,"attributes":{"title":"logstash-*","timeFieldName":"@timestamp","notExpandable":true}}],"total":1,"per_page":20,"page":1}

Checking for a default index:

curl -XGET <kibana>/api/kibana/settings
-> 
{"settings":{"buildNum":{"userValue":8467}}}

Am I correct in assuming that you're saying that step 4 actually creates the default index pattern and you'd see it in step 6?

mathias · November 28, 2017, 8:42pm

Yes, that is correct.

I used inspect inside the browser to see what what sent when I set default index from the GUI.
I did the same thing using curl and it worked.

Br Mathias

chrisronline · November 28, 2017, 9:18pm

Oh, so if I understand this correctly, you're saying that you did an extra curl command? Specifically the one you found that the browser made to set the default index?

Assuming that's true, you're saying that these steps work fine without x-pack but not with it, right?

chrisronline · November 28, 2017, 9:47pm

I think the core problem is that you're using the index pattern title instead of the id. That changed in 5.6. Instead of posting -d '{"value": "events-*"}', try using the index pattern id, which you can see from the /saved_objects/index-pattern endpoint.

However, this behavior should be consistent between x-pack and non x-pack

system · December 26, 2017, 9:47pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.