Hi Everyone,
I am using
kibana 4.5.4
elasticsearch 2.3.3
i am, trying to plot a bar or line graph for file types across the timestamp but unable to filter the file types
we have field called "url" in our index..
so from that field we need to combine all the similar types like .png or .jpg to images ...in the same way videos.
sample url:/abc/cde/efg/test.png
iam plotiing graph in the following way..
y-axis: count
x-axis: timestamp
split-lines: selected sub-aggregation as filters..
Now clicked the advanced and given the JSON input as below
{
"bool": {
"should": [
{
"wildcard": {
"url": {
"value": "*jpg"
}
}
},
{
"wildcard": {
"url": {
"value": "*png"
}
}
}
]
}
this above filter is not effecting when and giving error as "bool" is unknown key..
But when i give in elasticsearch in the below way i n the sense iam getting
GET xyz/_search
{
"filter": {
"bool": {
"should": [
{
"wildcard": {
"url": {
"value": "*jpg"
}
}
},
{
"wildcard": {
"url": {
"value": "*png"
}
}
}
]
}
}
}
Please anyone Help Me Out How add this in KIbana
Thank You
Balu
Hi Balu,
I would consider using a scripted field here:
I assume your field values are like a path. something like:
my/path/to/file1.png
my/path/to/file2.png
my/path/to/file1.jpg
You can use a scripted field to strip off the file type extension.
With that, you can use the same visualization configuration you have now.
- In Kibana, create a scripted field with (Management > Indices > Inde Patterns > Scripted Fields).
You're script would look something like. Give it a type of 'string'.
Suppose we call that "extensiondyn".
- Then you can use that field to split our line chart, the same way you were doing earlier.
Hope this helps!
HI thomas,
Thanx a lot for your reply.....But please help me iam still getting error.
I have fields like "url": "/get/files/images/duck.jpg"
I have given as below
doc['url.raw'].value.lastIndexOf('.') > 0 ?
doc['url.raw'].value.substring(doc['url.raw'].value.lastIndexOf('.')) : 'foobar'
Now when i redirect to the discover tab iam getting error
now edited and given in this below one iam getting same
doc['url'].value.lastIndexOf('.') > 0 ?
doc['url'].value.substring(doc['url'].value.lastIndexOf('.')) : 'foobar'
...
...
..
..
..
..
EVEN i tried in this way....
..
..
..
And i have seed somewhere in doumentation like kibana4 will now allow scripted fields?...Is that true?......PLease suggest me
As iam new to this ,unable to debug this....Please help me what is the issue here
Hi Balu,
Thanks for checking into it. I jumped the gun here. You are right that this is not working in 4.6. It will be possible in the 5.0 release of Kibana.
In 4.6, you can't create scripted fields that have a string data-type in Kibana. This explains the errors you are seeing. The script I showed uses 'Painless' as the scripting language, where you can perform string operations using the Java String APIs. In Kibana 4.6 you can only use scripts to evaluate numeric expressions.
This will be possible in Kibana 5 with the following improvement: https://github.com/elastic/kibana/pull/7700). The beta for the 5 release, which will include this improvement, is coming out very shortly.
Another option in the meantime is to reindex your source data, where you strip off the file type extension up front.
Okay thanx.............but does this work in 4.4 and less versions ...please suggest
Hi Balu,
The Kibana integration to write scripts in Painless will only available in Kibana 5.0 and higher (so not 4.4 or less).
Reindexing your data is something you could do now.
Hi Thomas,
okay fine ....okay i will reindex.....Please suggest how i need to re-index to reach this requirement i.e to filter all the images types(.jpg,png....etx) As a image category.......and all the Video types as the video category...........
I am sharing you the rough sketch,please help what i need to do to get this...
..
..
..
please help me out....
for now index we have fields..like
timestamp:
url:/abc/bcd/ngp/sample.jpg
filesize:10kb
please tel how we need to re-index and filter in kibana to reach the requirement ...
Hi Balu,
I believe this is something you will have to do manually. You will have to update your documents to include this new field (file type) with the truncated filename.
I would ask this question on the logstash forum (https://discuss.elastic.co/c/logstash), since that product is used for this exact sort of thing (ie parsing text and extracting new fields from it).
