and it works perfect in https://grokdebug.herokuapp.com/, see below screenshot but however it returns "tags" => [
[0] "_grokparsefailure"
] and i don't see the extracted fields totalSize and someId
^{%{QUOTEDSTRING}:%{INT},%{QUOTEDSTRING}:%{DATA:totalSize},%{DATA}{%{QUOTEDSTRING}:{%{QUOTEDSTRING}:%{INT:someId},%{DATA}}$
#I use quoted string and remove unecessary parentheses.
Your input respect the json format, use the json plugin should be the easiest way to index your values.
Yes you are right, I think I should use json plugin, but from the entire json I'm interested in only one key value and that json path is not static in nature, how can I solve this, any thoughts?
According to the data you show us, term have a nested field named someId not busId.
And to access to a specific array position, you have to use [must][0] instead of [must[0]]
Here the final line: add_field => { "json_someId" => "%{[query][bool][must][0][term][someId][value]}" }
I'm trying to calculate the time taken to parse by JSON filter but the given field is not being added in the output, can you please see if there is anything wrong with my code
from the above output, My understanding is, the event json took total 2 records and parsed/transformed/processed it in 39 milliseconds which is basically 39/2 = 19.5 milliseconds for each document , Can you please correct me if my understanding is not correct?
Also, can you please tell me the difference between these
I need help on json filter plugin, the thing is I've the capture a value from one json path but some times the json pattern changes so, how can we achieve that do we have any fallback for that?
Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 69, column 16 (byte 2013) after filter
and in the line number 69 , there is a if block statement which is if "es_someId" =~ "parsedJson" { as shown in above code example.
I think here you want to check the content of es_someId. In a condition, to access to a field value you have to use this syntax : if [es_someId] =~ "parsedJson" {
Thanks for that, can you look at the below poblem please
Problem: Trying to extract the key value pair from a json BUT the JSON is not static in nature it may have number of patterns like expected key pair value json path will be changed dynamically, so in grok we use
break_on_match=>true
it helps us to check the match against the number of patterns and if matched one of the pattern then it wont check for another one, similarly do we have anything to match against different jsonPaths for a value?
Yeah finally I've moved to ruby instead JSON plugin as the JSON is not static in nature and below is the logic I used , let me know if there is any better way.
if "_grokparsefailure" in [tags]{
ruby{
code => 'event.get("message")
if event.get("message").include? "someId"
someIdIndexStarts=event.get("message").index("someId")
someIdValueIndexStart=someIdIndexStarts+19
lengthOfTotalString=event.get("message").length
for i in someIdValueIndexStart..lengthOfTotalString
if event.get("message").slice(i) == ","
endOfSomeIdValueIndex=i
event.set("es_someId",event.get("message").slice(someIdValueIndexStart..endOfSomeIdValueIndex-1));
break;
end
end
end'
}
}
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.