Truncating String in Data Table

[wwalker (Walker)]

I have a data table visualization where I am counting the top hits on a list of URLs. All the URLs have http://www.example.com/index. I am counting top hits on the part after /index. Is it possible to truncate the root portion of the URL within the Kibana visualization without modifying the Elasticsearch stored data?

[thomasneirynck]

hi @wwalker,

you'll want to look at scripted fields in Kibana, https://www.elastic.co/guide/en/kibana/current/scripted-fields.html. This will allow you to truncate the url on the-fly, without changing the underlying storage.

[wwalker (Walker)]

Looks exactly like what I would want....except that I don't know any programming language to get it to do what I want, lol. I'll see what I can learn about painless.

[wwalker (Walker)]

OK, working with the example documentation, I've gotten close...I think. However, when I run the preview, I get the below error. I think it has to do with the field I am working with having a space in it, but I'm not sure what the syntax is that I am supposed to use. I've tried single, double, and triple quotes around the field path but all result in the same error.

Painless Script:
"'ctx._source.Full URL'" = /https:\/\/www.example.com\/government\/elections*\/.matcher("'ctx._source.Full URL'").replaceAll('');

Resulting error
"shard": 0, "index": "cloudflare-2018.10.31", "node": "pPGsP09BSNeNANYO6A9KEQ", "reason": { "type": "script_exception", "reason": "compile error", "script_stack": [], "script": "\"'ctx._source.Full URL'\" = /https:\\/\\/www.example.com\\/government\\/elections*\\/.matcher(\"'ctx._source.Full URL'\").replaceAll('');", "lang": "painless", "caused_by": { "type": "illegal_argument_exception", "reason": "Left-hand side cannot be assigned a value."

[wwalker (Walker)]

Figured the syntax out...and fixed my jacked up regex.

ctx._source['Full URL'] = /https:\/\/example\.com\/government\/elections\//.matcher(ctx._source['Full URL']).replaceAll('')

The field was accepted and created but now I get "1 of 2 shards failed" and no results in the Discover regardless of the time picker range I select. If I delete the scripted field, Discover shows everything again.

Any ideas on what I did wrong?

[wwalker (Walker)]

In addition to Discover not showing any data, if I create a visualization and select the newly created field, I get the following error:

{
	"error": {
		"root_cause": [
			{
				"type": "script_exception",
				"reason": "runtime error",
				"script_stack": [
					"ctx._source['Full URL'] == /https:\\/\\/example\\.com\\/government\\/elections\\//i.matcher(ctx._source['Full URL']).replaceAll('')",
					" ^---- HERE"
				],
				"script": "ctx._source['Full URL'] == /https:\\/\\/example\\.com\\/government\\/elections\\//i.matcher(ctx._source['Full URL']).replaceAll('')",
				"lang": "painless"
			},
			{
				"type": "script_exception",
				"reason": "runtime error",
				"script_stack": [
					"ctx._source['Full URL'] == /https:\\/\\/example\\.com\\/government\\/elections\\//i.matcher(ctx._source['Full URL']).replaceAll('')",
					" ^---- HERE"
				],
				"script": "ctx._source['Full URL'] == /https:\\/\\/example\\.com\\/government\\/elections\\//i.matcher(ctx._source['Full URL']).replaceAll('')",
				"lang": "painless"
			}
		],
		"type": "search_phase_execution_exception",
		"reason": "all shards failed",
		"phase": "query",
		"grouped": true,
		"failed_shards": [
			{
				"shard": 0,
				"index": "cloudflare-2018.10.30",
				"node": "pPGsP09BSNeNANYO6A9KEQ",
				"reason": {
					"type": "script_exception",
					"reason": "runtime error",
					"script_stack": [
						"ctx._source['Full URL'] == /https:\\/\\/example\\.com\\/government\\/elections\\//i.matcher(ctx._source['Full URL']).replaceAll('')",
						" ^---- HERE"
					],
					"script": "ctx._source['Full URL'] == /https:\\/\\/example\\.com\\/government\\/elections\\//i.matcher(ctx._source['Full URL']).replaceAll('')",
					"lang": "painless",
					"caused_by": {
						"type": "null_pointer_exception",
						"reason": null
					}
				}
			},
			{
				"shard": 0,
				"index": "cloudflare-2018.10.31",
				"node": "pPGsP09BSNeNANYO6A9KEQ",
				"reason": {
					"type": "script_exception",
					"reason": "runtime error",
					"script_stack": [
						"ctx._source['Full URL'] == /https:\\/\\/example\\.com\\/government\\/elections\\//i.matcher(ctx._source['Full URL']).replaceAll('')",
						" ^---- HERE"
					],
					"script": "ctx._source['Full URL'] == /https:\\/\\/example\\.com\\/government\\/elections\\//i.matcher(ctx._source['Full URL']).replaceAll('')",
					"lang": "painless",
					"caused_by": {
						"type": "null_pointer_exception",
						"reason": null
					}
				}
			}
		]
	},
	"status": 500
}