Hello everyone.
I am trying to modify some parameters of the logs that come from fleet with the "custom logs" integration.
I have created the following pipeline:
LOG LINE:
2023-02-28 09:04:01,937 ERROR [org.jboss.remoting.transport.http.HTTPClientInvoker] (http-172.17.119.148-8180-14) Error creating SSL Socket Factory for client invoker: Error initializing socket factory SSL context: Can not find truststore url.
custom-logs-pipelines
[
{
"dissect": {
"field": "message",
"pattern": "%{date} %{hour} %{log.level} %{origen} %{url} %{message}"
}
}
]
TEST PIPELINE OK:
{
"docs": [
{
"doc": {
"_index": "_index",
"_id": "_id",
"_version": "-3",
"_source": {
"date": "2023-02-28",
"origen": "[org.jboss.remoting.transport.http.HTTPClientInvoker]",
"message": "Error creating SSL Socket Factory for client invoker: Error initializing socket factory SSL context: Can not find truststore url.",
"hour": "09:04:01,937",
"log": {
"level": "ERROR"
},
"url": "(http-172.17.119.148-8180-14)"
},
"_ingest": {
"timestamp": "2023-03-14T14:30:47.06430719Z"
}
}
}
]
}
integration configuration:
I also have created a component template
custom-logs@mapping
{
"properties": {
"date": {
"eager_global_ordinals": false,
"norms": false,
"index": true,
"store": false,
"type": "keyword",
"split_queries_on_whitespace": false,
"index_options": "docs",
"doc_values": true
},
"hour": {
"eager_global_ordinals": false,
"norms": false,
"index": true,
"store": false,
"type": "keyword",
"index_options": "docs",
"split_queries_on_whitespace": false,
"doc_values": true
},
"log": {
"type": "object",
"properties": {
"level": {
"type": "keyword"
}
}
},
"origen": {
"type": "keyword"
},
"message": {
"type": "text"
},
"url": {
"type": "keyword"
}
}
}
and finally my index template
This is the message I get after all the configuration
11:47:15.059
elastic_agent.filebeat
[elastic_agent.filebeat][error] failed to publish events: temporary bulk send failure
11:47:16.883
elastic_agent.filebeat
[elastic_agent.filebeat][error] failed to publish events: temporary bulk send failure
11:47:18.433
elastic_agent.filebeat
[elastic_agent.filebeat][error] failed to publish events: temporary bulk send failure
11:47:19.858
elastic_agent.filebeat
[elastic_agent.filebeat][error] failed to publish events: temporary bulk send failure
11:47:21.299
elastic_agent.filebeat
[elastic_agent.filebeat][error] failed to publish events: temporary bulk send failure
I really don't know what I'm failing, any help would be very useful at this time.