I am new to ELk .I am not able to find a word which is seprated by (.) for example: javax.ejb.duplicatekeyexception
so when i only search duplicatekeyexception . kibana does not show any result , kindly please help me in resolving this issue .
the standard text analyzer of elasticsearch will not split on .
to find it you would have to search for the full exception(fieldname:"javax.ejb.duplicatekeyexception")
if you want to search for the single word you would have to include a custom analyzer for the searched field in the index mapping
thanks for the reply sir,
i searched alot about custom analyzer but i didn't get where to write all those put,get api request and how to see its output on kibana .could you please explain me the steps of making an analyzer using elasticsearch , logstash and kibana.
The analyzer is only on elasticsearch site and can/must be set for each index.
It can only be set on new indices or in templates for new indices.
-
create new index with analyzer
1.1 go to the dev tools in kibana
1.2 execute th following(replace <indexname>, <type> and <fieldname>:PUT <indexname> { "settings": { "analysis": { "analyzer": { "point_analyzer": { "type": "pattern", "pattern": "\\.|\\s" } } } }, "mappings": { "<type>": { "properties": { "<fieldname>": { "type": "text", "analyzer": "point_analyzer" } } } } }1.3 index a document in the new index
next post template creation for automatic index creation
create template for indices that are automatically created
- got to dev tools in kibana
- execute the following(replace all that is in <>)
PUT _template/<some template name>
{
"order": 0,
"template": "<pattern for indexname e.g. elk-*>",
"settings": {
"analysis": {
"analyzer": {
"point_analyzer": {
"type": "pattern",
"pattern": "\\.|\\s"
}
}
}
},
"mappings": {
"<type>": {
"properties": {
"<fieldname>": {
"type": "text",
"analyzer": "point_analyzer"
}
}
}
}
}
- let logstash create an index with a name in the new index pattern
thanks for the help!
i have completed all the steps but after this also m not able to get the result in kibana.
message:"duplicatekeyexception" i am seraching and still it is not giving any output.
this my logstash.conf file:
input {
file {
path => "xyz/s.log"
start_position => "beginning"
sincedb_path => "xyz/loginput1.sincedb"
ignore_older => 0
codec => multiline {
pattern => "^\s"
what => "previous"
}
}
}
filter
{
date {
match => ["timestamp" , "yyyy-MM-dd'T'HH:mm:ss.SSSZ"]
target => "@timestamp"
add_field => { "debug" => "timestampMatched"}
}
}
output {
stdout {
codec => rubydebug
}
if "xyz1" in [message] {
email {
to => 'technical@logstash.net'
from => 'monitor@logstash.net'
subject => 'Alert - %{title}'
body => "Tags: %{tags}\n\Content:\n%{message}"
domain => 'mail.logstash.net'
port => 25
}
}
elasticsearch {
action => "index"
hosts => "localhost"
index => "logdemo12"
}
}
this is what i am exceuting in kibana dev tool:
PUT customindex1
{
"settings": {
"analysis": {
"analyzer": {
"point_analyzer": {
"type": "pattern",
"pattern": "\.|\s"
}
}
}
},
"mappings": {
"dotanlayzer": {
"properties": {
"message": {
"type": "text",
"analyzer": "point_analyzer"
}
}
}
}
}
PUT _template/customtemplate
{
"order": 0,
"template": "logdemo1-*",
"settings": {
"analysis": {
"analyzer": {
"point_analyzer": {
"type": "pattern",
"pattern": "\.|\s"
}
}
}
},
"mappings": {
"dotanlayzer": {
"properties": {
"message": {
"type": "text",
"analyzer": "point_analyzer"
}
}
}
}
}
with your current logstash configuration you would have to change the template with the following command (in dev-tool):
PUT _template/customtemplate
{
"order":0,
"template":"logdemo12",
"settings":{
"analysis":{
"analyzer":{
"point_analyzer":{
"type":"pattern",
"pattern":"\\.|\\s"
}
}
}
},
"mappings":{
"_default_":{
"properties":{
"message":{
"type":"text",
"analyzer":"point_analyzer"
}
}
}
}
}
and
DELETE logdemo12
than you could try again
Thanks for the help..
I am able to see the results when i use GET method but when i search on kibana UI then it gives me nothing. so for that what i have to do?
If you see it with a GET Request to ES and not in Kibana the only thing that comes to mind is the refresh of the configured index in Kibana
-
Management -> Index Patterns -
Delete old index pattern -
Create a new Index Pattern with the new indexname logdemo12
I did this also still not working.
steps which i am doing is:
- i executed (put/indexname , put/customindex) in dev tool
- i executed (put/templatename, put/customtemplate) in dev tool
- i ran the logstash.conf file with above configurations.
- i searched on kibana and did'nt get the result!
Thanks in advance and sorry for troubling u alot
can you post the response to the GET request from Elasticsearch for the document that you get back
{
"took": 114,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 27,
"max_score": 8.557227,
"hits": [
{
"_index": "logdemo12",
"_type": "logs",
"_id": "xbzvzh",
"_score": 8.557227,
"_source": {
"@timestamp": "2017-09-11T06:12:09.251Z",
"message": "[7/25/17 7:49:12:583 CDT] xxxxxxxxxxxxxxxxxxxxx com.xxx.NullPointerException
"@version": "1",
"path": "D:/b.log",
"host": "abcdef"
}
},
okay i cann't see any problem why it shouldn't be visible in Kibana
the obvious thing would be that the timeframe in kibana is to short but other than that... 
What does raw field means?
thanks for the help christian it worked for few of the text not for all 
any idea what is the difference between the messages that are shown and the ones that are not shown?
can't think of anything
Now it is working ! after adding this line "pattern": "\.|\s|\:|\_"
Thanks alot Christian 
!
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.