Dear Community,
I currently have the problem that query and click data are not being included in ES. It turns out that this is due to Filebeat not being able to read the log file correctly. An error is thrown:
**Preview of field’s value: ‘[0:0:0:0:0:0:0:1]‘\“,**“caused_by\“:{\“type\“:\“illegal_argument_exception\“,\“reason\“:\“‘[0:0:0:0:0:0:0:1]’ is not an IP string literal.\“}}, dropping event!“
Full:
{
“log.level”: “warn”,
“@timestamp”: “2023-07-25T09:43:25.060Z”,
“log.logger”: “elasticsearch”,
“log.origin”: {
“file.name”: “elasticsearch/client.go”,
“file.line”: 429
},
“message”: “Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Date(2023, time.July, 25, 9, 43, 20, 320000000, time.Local), Meta:null, Fields:{\“agent\“:{\“ephemeral_id\“:\“92934042-b05d-4045-996d-2503cacdab64\“,\“id\“:\“6da624bf-2e98-4eca-b15b-4c4760c9f0ab\“,\“name\“:\“woehler-sql1\“,\“type\“:\“filebeat\“,\“version\“:\“8.7.0\“},\“data_stream\“:{\“dataset\“:\“app_search.analytics\“,\“namespace\“:\“default\“,\“type\“:\“logs\“},\“ecs\“:{\“version\“:\“1.7.0\“},\“event\“:{\“action\“:\“loco_moco_search\“,\“category\“:\“app-search-analytics\“,\“created\“:\“2023-07-25T09:43:20Z\“,\“dataset\“:\“app-search-analytics\“,\“document_ids\“:null,\“loco_moco_search_request_id\“:\“dd9e4608-8f7b-44cb-9a7c-192b733a3961\“,\“query_string\“:\“\”,\“tags\“:[\“swiftype_documents_api\“]},\“host\“:{\“name\“:\“woehler-sql1\“},\“input\“:{\“type\“:\“log\“},\“labels\“:{\“engine_id\“:\“64bf9890f9bc6b592f0379fc\“,\“index_date\“:\“2023.07.25\“,\“lm_account_id\“:\“64bf96adf9bc6b41d07557e7\“},\“log\“:{\“file\“:{\“path\“:\“/var/log/enterprise-search/filebeat.log\“},\“offset\“:631840},\“related\“:{\“ip\“:\“[0:0:0:0:0:0:0:1]\“}}, Private:file.State{Id:\“native::791127-65024\“, PrevId:\“\”, Finished:false, Fileinfo:(*os.fileStat)(0xc000179ad0), Source:\“/var/log/enterprise-search/filebeat.log\“, Offset:632431, Timestamp:time.Date(2023, time.July, 25, 9, 43, 23, 929801211, time.Local), TTL:-1, Type:\“log\“, Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0xc1257, Device:0xfe00}, IdentifierName:\“native\“}, TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:mapstr.M(nil)}} (status=400): {\“type\“:\“document_parsing_exception\“,\“reason\“:\“[1:393] failed to parse field [related.ip] of type [ip] in document with id ‘8WZujIkBkKjv5knQlm4S’. **Preview of field’s value: ‘[0:0:0:0:0:0:0:1]‘\“,\**“caused_by\“:{\“type\“:\“illegal_argument_exception\“,\“reason\“:\“‘[0:0:0:0:0:0:0:1]’ is not an IP string literal.\“}}, dropping event!“,
“service.name”: “filebeat”,
“ecs.version”: “1.6.0"
}
When I force access via IPv4, the error does not occur. It is exclusively the case with IPv6, as it does not seem to fit the mapping pattern of Filebeat.
Does anyone have any idea what this could be due to? Is this a bug?
Many greetings
Frederik