I am adding filter to remove a plain text as it causes error while JSON parsing.
file {
id => "my_lt_log"
path => "/logs/logtransformer.log"
type => "log"
start_position => "beginning"
}
if [type] == "log" {
date {
match => [ "timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSZ" ]
target => "timestamp"
}
if [message] =~ /.*JAVA_TOOL_OPTIONS.*/{
drop { }
}
json {
skip_on_invalid_json => true
source => "message"
target => "json"
remove_field=>["message", "kubernetes"]
}
I will not get any log in Elasticsearch which is fine, but the input file /logs/logtransformer.log shows one error logs:
{"version": "1.1.0", "timestamp": "2023-02-10T13:14:48.467Z", "severity": "warning", "service_id": "eric-log-transformer", "metadata" : {"namespace": "zyadros", "pod_name": "eric-log-transformer-56b7fdbc6f-4hn45", "node_name": "node-10-63-142-147", "pod_uid": "bfe6ffa6-397f-4785-b91e-54c2b6c37cb5", "container_name": "logtransformer"}, "message": "Error parsing json {:source=>'message', :raw=>'Picked up JAVA_TOOL_OPTIONS: -XX:InitialRAMPercentage=50.000000 -XX:MaxRAMPercentage=75.000000', :exception=>#<LogStash::Json::ParserError: Unrecognized token 'Picked': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
at [Source: (byte[])'Picked up JAVA_TOOL_OPTIONS: -XX:InitialRAMPercentage=50.000000 -XX:MaxRAMPercentage=75.000000'; line: 1, column: 8]>}"}
{"version": "1.1.0", "timestamp": "2023-02-10T13:14:49.382Z", "severity": "warning", "service_id": "eric-log-transformer", "metadata" : {"namespace": "zyadros", "pod_name": "eric-log-transformer-56b7fdbc6f-4hn45", "node_name": "node-10-63-142-147", "pod_uid": "bfe6ffa6-397f-4785-b91e-54c2b6c37cb5", "container_name": "logtransformer"}, "message": "Could not index event to OpenSearch. {:status=>400, :action=>['index', {:_id=>nil, :_index=>'adp-app-logs-2023.02.10', :routing=>nil}, {'@version'=>'1', 'timestamp'=>'%{[json][timestamp]}', 'service_id'=>'%{[json][service_id]}', 'tags'=>['_jsonparsefailure'], 'version'=>'%{[json][version]}', 'message'=>['Picked up JAVA_TOOL_OPTIONS: -XX:InitialRAMPercentage=50.000000 -XX:MaxRAMPercentage=75.000000', '%{[json][message]}'], 'severity'=>'%{[json][severity]}', 'metadata'=>{'container_name'=>'%{[json][metadata][container_name]}', 'node_name'=>'node-10-63-142-147', 'namespace'=>'zyadros', 'pod_name'=>'eric-log-transformer-56b7fdbc6f-4hn45', 'pod_uid'=>'bfe6ffa6-397f-4785-b91e-54c2b6c37cb5'}, 'filename'=>'/logs/metrics.log', '@timestamp'=>2023-02-10T13:14:47.800974Z, 'logplane'=>'adp-app-logs'}], :response=>{'index'=>{'_index'=>'adp-app-logs-2023.02.10', '_type'=>'_doc', '_id'=>'3Q52O4YBCoUR1nQu1R4i', 'status'=>400, 'error'=>{'type'=>'mapper_parsing_exception', 'reason'=>'failed to parse field [timestamp] of type [date] in document with id '3Q52O4YBCoUR1nQu1R4i'. Preview of field's value: '%{[json][timestamp]}'', 'caused_by'=>{'type'=>'illegal_argument_exception', 'reason'=>'failed to parse date field [%{[json][timestamp]}] with format [strict_date_optional_time||epoch_millis]', 'caused_by'=>{'type'=>'date_time_parse_exception', 'reason'=>'date_time_parse_exception: Failed to parse with all enclosed parsers'}}}}}}"}
And this actual message without filter is:
Picked up JAVA_TOOL_OPTIONS: -XX:InitialRAMPercentage=50.000000 -XX:MaxRAMPercentage=75.000000
)