Quick question, Scripted Field hostname + url

Elastic Stack Kibana
1

How would someone do to get a "complete hostname with url" field

using : hostname.raw and url.raw

hostname is shown in log as : hostname="cm.g.doubleclick.net"
url is shown as : url="/pixel?google_nid=eyereturn&g=fde2328d-182e-49fa-a232-8d182ef9fafe&google_hm=eIyjRguSfqiMo0YLvn6_g&google"

so at the end, I would like to get :

cm.g.doubleclick.net/pixel?google_nid=eyereturn&g=fde2328d-182e-49fa-a232-8d182ef9fafe&google_hm=eIyjRguSfqiMo0YLvn6_g&google as a field.

I assume I can do it in Scripted Field, but I dont see much information about that... anyone already succeed in doing it ?

Thank you !

2

What version of Kibana and Elasticsearch are you using?

3

latest

kibana 4.4 latest stable build, same for elasticsearch latest build avail on repo

4

curl -XGET 'localhost:9200'
{
"name" : "Beyonder",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "2.3.4",
"build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
"build_timestamp" : "2016-06-30T11:24:31Z",
"build_snapshot" : false,
"lucene_version" : "5.5.0"
},
"tagline" : "You Know, for Search"

5

So, there currently a limitation with scripted fields that does not allow string manipulation. It's a limitation of Lucene expressions.
My suggestion would be to do the string concatenation when you index the documents so you have three fields. hostname, url and full_url

6

sorry to ask .. but im sort of a newbie limit script kiddy in term of ELK stack, would you mind giving me an exemple ?

plus I assume, I would then have to re-index the whole thing ? or start from scratch ..

right ?

7

What are you using to index your documents?

8

I'm not sure I understand ?

I use a logstash input file 10-network.conf that fetch logs from /var/log/network.log using kv filter and some mutate. then output using 50-output.conf

to my elasticsearch..

9

My logstash ability is almost non-existent... but a quick google search turned this up:

filter {
  mutate {
    add_field => ["field3", "%{field1} %{field2}"]
  }
}

You'll also want to make sure that the field that you're adding is not-analyzed.
Elasticsearch mapping-intro Link

And AFAIK, you will have to re-index your data.

I've also been told that when 5.0 launches, you'll be able to create a scripted field that concatenates strings, but that doesn't help you right now.