Kibana is dividing the urls

[Leooel (Leonardo)]

Recently I created a grok to parse logs of Squid3, and all work normally to send all the data to elasticsearch.

I can see everything normally on discover part, but when I create a graphic visualization, to analize the urls requisitions, Kiana is dividing the urls.

EX: "443", "wp", "www86", "rf", "ss", and more things like that

[LeeDr (Lee Drengenberg)]

Hi Leonardo,

You're using an analyzed field in your visualization. Strings like your URL that are sent to the standard analyzer are separated into terms by punctuation marks like the .s in your URLs. Depending on how you are getting your logs into Elasticsearch you could have both an analyzed and not analyzed (or raw) field, and in this case you would want that field. Discover tab in Kibana doesn't show these fields, but they should be available in Visualizations if you have the fields. If you don't have the field, you would need to modify the mapping in Elasticsearch to store it in your index. If you're using Logstash to get the logs into Elasticsearch you could look at that documentation.

Here's one reference How to query the stored, un-analyzed, form of an analyzed field?

If that one's not clear, search around for "not analyzed" or "raw" and you should find lots of examples.

Regards,
Lee

[LeeDr (Lee Drengenberg)]

Here's another related thread;

How can I updated the a sub-field within an index to be not_analyzed?

I have an index with a sub-filed that I want to set to be not analyzed? How can I change this? Additionally, this indexes are patterned for the day using Filebeat -- is there a way to ensure when new indices are created by Filebeat, the same not analyzed configuration takes place? Here is my index myrecords-YYYY-MM-DD: { "myrecords-2016.10.25": { "aliases": {}, "mappings": { "log": { "properties": { "@timestamp": { "type": "date", "format": "dateOptionalTime" …

[Leooel (Leonardo)]

Hii Lee,

This is the output parsed to elasticsearch:

{
"message" => "12.12.12.12 - - [21/Oct/2016:13:56:00 -0300] "CONNECT some.site.com HTTP/1.1" 503 0 TCP_MISS:HIER_NONE",
"@timestamp" => "2016-10-28T11:03:54.191Z",
"source" => "/var/log/access.log",
"offset" => 148843,
"host" => "logs",
"IP" => "12.12.12.12",
"ident" => "-",
"auth" => "-",
"Data" => "21/Oct/2016:13:56:00 -0300",
"Metodo" => "CONNECT",
"requisicao" => "some.site.com",
"httpversion" => "1.1",
"response" => "503",
"bytes" => "0",
"tcptype" => "TCP_MISS:HIER_NONE"
}

I tried theese thigs:

• Change the field to raw
• Change to not_analized
• Change to not_analized + raw

And happen the following:

When raw, the URL is divided

When not_analized, don't appear the field of URL on graphic

When not_analized + raw, don't appear the field of URL on graphic

When indexed: false, don't appear the field of URL on graphic

I connect to the elasticsearch with dejavù app of chrome.

Here the schemma:

{
"title": "squid3",
"timeFieldName": "@timestamp",
"fields": "[{"name":"Metodo","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"host","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"bytes","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"_source","type":"_source","count":0,"scripted":false,"indexed":false,"analyzed":false,"doc_values":false},{"name":"IP","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"_index","type":"string","count":0,"scripted":false,"indexed":false,"analyzed":false,"doc_values":false},{"name":"requisicao","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"message","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"response","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"httpversion","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"@timestamp","type":"date","count":0,"scripted":false,"indexed":true,"analyzed":false,"doc_values":true},{"name":"Data","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"source","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"ident","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"tcptype","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"offset","type":"number","count":0,"scripted":false,"indexed":true,"analyzed":false,"doc_values":true},{"name":"auth","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},{"name":"_id","type":"string","count":0,"scripted":false,"indexed":false,"analyzed":false,"doc_values":false},{"name":"_type","type":"string","count":0,"scripted":false,"indexed":false,"analyzed":false,"doc_values":false},{"name":"_score","type":"number","count":0,"scripted":false,"indexed":false,"analyzed":false,"doc_values":false}]"
}
CTRL+F field of URL = requisicao
||
V

{"name":"requisicao","type":"string","count":0,"scripted":false,"indexed":true,"analyzed":true,"doc_values":false},

The fields changed: requisicao to requisicao.raw
indexed:true to indexed:false
analyzed:true to analyzed:false

Did I made something wrong?