When I ingest json logs over a pipeline with a json parser, I get a document_parsing_exception error:
(status = 400): {
"type": "document_parsing_exception",
"reason": "[1:15872] failed to parse field [gitlab.params.value] of type [keyword] in document with id 'LcFzYIsBEzyvtGgrvrS5'. Preview of field's value: '{variables=[FILTERED], query=query pathLastCommit($projectPath: ID!, $path: String, $ref: String!, $refType: RefType) {\n project(fullPath: $projectPath) {\n __typename\n id\n repository {\n __typename\n paginatedTree(path: $path, ref: $ref, refType: $refType) {\n __typename\n nodes {\n __typename\n lastCommit {\n __typename\n id\n sha\n title\n titleHtml\n descriptionHtml\n message\n webPath\n authoredDate\n authorName\n authorGravatar\n author {\n __typename\n id\n name\n avatarUrl\n webPath\n }\n signature {\n __typename\n ... on GpgSignature {\n gpgKeyPrimaryKeyid\n verificationStatus\n __typename\n }\n ... on X509Signature {\n verificationStatus\n x509Certificate {\n id\n subject\n subjectKeyIdentifier\n x509Issuer {\n id\n subject\n subjectKeyIdentifier\n __typename\n }\n __typename\n }\n __typename\n }\n ... on SshSignature {\n verificationStatus\n keyFingerprintSha256\n __typename\n }\n }\n pipelines(ref: $ref, first: 1) {\n __typename\n edges {\n __typename\n node {\n __typename\n id\n detailedStatus {\n __typename\n id\n detailsPath\n icon\n tooltip\n text\n group\n }\n }\n }\n }\n }\n }\n }\n }\n }\n}\n, operationName=pathLastCommit}'",
"caused_by": {
"type": "illegal_state_exception",
"reason": "Can't get text on a START_OBJECT at 1:13668"
}
}
I write the parsed fields to the target field gitlab.
The problem is the content of the parsed field params.value. The content "[FILTERED]" is interpreted as array I think.
But where I simulate this pipeline with the same document, the json is parsed without error.
The ignore_failure flag is true, but this don't help.
Any ideas to solve this?
Here is my simulate request:
"pipeline": {
"description": "parse json",
"processors": [
{
"json": {
"field": "message",
"target_field": "gitlab",
"ignore_failure": false,
"strict_json_parsing": false
}
}
]
},
"docs": [
{
"_source": {
"agent": {
"ephemeral_id": "d8c092e5-5b6d-48d2-a0c4-97e770e0fb65",
"id": "6fb7162d-8150-4969-8a32-f1526e6e0db7",
"name": "progit2-1",
"type": "filebeat",
"version": "8.10.2"
},
"container": {
"id": "gitlab-rails"
},
"data_stream": {
"dataset": "gitlab",
"namespace": "pws",
"type": "logs"
},
"ecs": {
"version": "8.0.0"
},
"elastic_agent": {
"id": "6fb7162d-8150-4969-8a32-f1526e6e0db7",
"snapshot": false,
"version": "8.10.2"
},
"event": {
"dataset": "gitlab"
},
"host": {
"architecture": "x86_64",
"containerized": false,
"hostname": "progit2-1",
"id": "23df0f2fa04b415182d58c2dbe5798b8",
"ip": [
"10.210.1.112",
"fe80::250:56ff:fe97:4aaa",
"172.17.0.193",
"fe80::42:13ff:feee:4587",
"172.17.0.1",
"fe80::8075:14ff:fef6:707b"
],
"mac": [
"00-50-56-97-4A-AA",
"02-42-13-EE-45-87",
"02-42-56-AE-7B-1E",
"82-75-14-F6-70-7B"
],
"name": "progit2-1",
"os": {
"codename": "bullseye",
"family": "debian",
"kernel": "5.10.0-26-amd64",
"name": "Debian GNU/Linux",
"platform": "debian",
"type": "linux",
"version": "11 (bullseye)"
}
},
"input": {
"type": "log"
},
"log": {
"file": {
"path": "/var/lib/gitlab/logs/gitlab-rails/production_json.log"
},
"offset": 7082597
},
"message": "{\"method\":\"POST\",\"path\":\"/api/graphql\",\"format\":\"*/*\",\"controller\":\"GraphqlController\",\"action\":\"execute\",\"status\":200,\"time\":\"2023-10-24T06:51:18.708Z\",\"params\":[{\"key\":\"operationName\",\"value\":\"pathLastCommit\"},{\"key\":\"variables\",\"value\":\"[FILTERED]\"},{\"key\":\"query\",\"value\":\"query pathLastCommit($projectPath: ID!, $path: String, $ref: String!, $refType: RefType) {\\n project(fullPath: $projectPath) {\\n __typename\\n id\\n repository {\\n __typename\\n paginatedTree(path: $path, ref: $ref, refType: $refType) {\\n __typename\\n nodes {\\n __typename\\n lastCommit {\\n __typename\\n id\\n sha\\n title\\n titleHtml\\n descriptionHtml\\n message\\n webPath\\n authoredDate\\n authorName\\n authorGravatar\\n author {\\n __typename\\n id\\n name\\n avatarUrl\\n webPath\\n }\\n signature {\\n __typename\\n ... on GpgSignature {\\n gpgKeyPrimaryKeyid\\n verificationStatus\\n __typename\\n }\\n ... on X509Signature {\\n verificationStatus\\n x509Certificate {\\n id\\n subject\\n subjectKeyIdentifier\\n x509Issuer {\\n id\\n subject\\n subjectKeyIdentifier\\n __typename\\n }\\n __typename\\n }\\n __typename\\n }\\n ... on SshSignature {\\n verificationStatus\\n keyFingerprintSha256\\n __typename\\n }\\n }\\n pipelines(ref: $ref, first: 1) {\\n __typename\\n edges {\\n __typename\\n node {\\n __typename\\n id\\n detailedStatus {\\n __typename\\n id\\n detailsPath\\n icon\\n tooltip\\n text\\n group\\n }\\n }\\n }\\n }\\n }\\n }\\n }\\n }\\n }\\n}\\n\"},{\"key\":\"graphql\",\"value\":{\"operationName\":\"pathLastCommit\",\"variables\":\"[FILTERED]\",\"query\":\"query pathLastCommit($projectPath: ID!, $path: String, $ref: String!, $refType: RefType) {\\n project(fullPath: $projectPath) {\\n __typename\\n id\\n repository {\\n __typename\\n paginatedTree(path: $path, ref: $ref, refType: $refType) {\\n __typename\\n nodes {\\n __typename\\n lastCommit {\\n __typename\\n id\\n sha\\n title\\n titleHtml\\n descriptionHtml\\n message\\n webPath\\n authoredDate\\n authorName\\n authorGravatar\\n author {\\n __typename\\n id\\n name\\n avatarUrl\\n webPath\\n }\\n signature {\\n __typename\\n ... on GpgSignature {\\n gpgKeyPrimaryKeyid\\n verificationStatus\\n __typename\\n }\\n ... on X509Signature {\\n verificationStatus\\n x509Certificate {\\n id\\n subject\\n subjectKeyIdentifier\\n x509Issuer {\\n id\\n subject\\n subjectKeyIdentifier\\n __typename\\n }\\n __typename\\n }\\n __typename\\n }\\n ... on SshSignature {\\n verificationStatus\\n keyFingerprintSha256\\n __typename\\n }\\n }\\n pipelines(ref: $ref, first: 1) {\\n __typename\\n edges {\\n __typename\\n node {\\n __typename\\n id\\n detailedStatus {\\n __typename\\n id\\n detailsPath\\n icon\\n tooltip\\n text\\n group\\n }\\n }\\n }\\n }\\n }\\n }\\n }\\n }\\n }\\n}\\n\"}}],\"correlation_id\":\"01HDG77E63G7QGRWH8Y70Z9G9X\",\"meta.caller_id\":\"GraphqlController#execute\",\"meta.remote_ip\":\"10.212.134.35\",\"meta.feature_category\":\"groups_and_projects\",\"meta.user\":\"allgaier\",\"meta.user_id\":18,\"meta.client_id\":\"user/18\",\"graphql\":[{\"depth\":10,\"complexity\":77,\"used_fields\":[\"Project.__typename\",\"Project.id\",\"Repository.__typename\",\"TreeConnection.__typename\",\"Tree.__typename\",\"Commit.__typename\",\"Commit.id\",\"Commit.sha\",\"Commit.title\",\"Commit.titleHtml\",\"Commit.descriptionHtml\",\"Commit.message\",\"Commit.webPath\",\"Commit.authoredDate\",\"Commit.authorName\",\"Commit.authorGravatar\",\"UserCore.__typename\",\"UserCore.id\",\"UserCore.name\",\"UserCore.avatarUrl\",\"UserCore.webPath\",\"Commit.author\",\"CommitSignature.__typename\",\"GpgSignature.gpgKeyPrimaryKeyid\",\"GpgSignature.verificationStatus\",\"GpgSignature.__typename\",\"X509Signature.verificationStatus\",\"X509Certificate.id\",\"X509Certificate.subject\",\"X509Certificate.subjectKeyIdentifier\",\"X509Issuer.id\",\"X509Issuer.subject\",\"X509Issuer.subjectKeyIdentifier\",\"X509Issuer.__typename\",\"X509Certificate.x509Issuer\",\"X509Certificate.__typename\",\"X509Signature.x509Certificate\",\"X509Signature.__typename\",\"SshSignature.verificationStatus\",\"SshSignature.keyFingerprintSha256\",\"SshSignature.__typename\",\"Commit.signature\",\"PipelineConnection.__typename\",\"PipelineEdge.__typename\",\"Pipeline.__typename\",\"Pipeline.id\",\"DetailedStatus.__typename\",\"DetailedStatus.id\",\"DetailedStatus.detailsPath\",\"DetailedStatus.icon\",\"DetailedStatus.tooltip\",\"DetailedStatus.text\",\"DetailedStatus.group\",\"Pipeline.detailedStatus\",\"PipelineEdge.node\",\"PipelineConnection.edges\",\"Commit.pipelines\",\"Tree.lastCommit\",\"TreeConnection.nodes\",\"Repository.paginatedTree\",\"Project.repository\",\"Query.project\"],\"used_deprecated_fields\":[],\"variables\":\"{\\\"projectPath\\\"=\u003e\\\"infrastructure-as-code/iac-rabbitmq\\\", \\\"ref\\\"=\u003e\\\"master\\\", \\\"refType\\\"=\u003e\\\"HEADS\\\", \\\"path\\\"=\u003e\\\"scripts/deploy.sh\\\"}\",\"operation_name\":\"pathLastCommit\"}],\"remote_ip\":\"10.212.134.35\",\"user_id\":18,\"username\":\"allgaier\",\"ua\":\"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0\",\"queue_duration_s\":0.006815,\"request_urgency\":\"low\",\"target_duration_s\":5,\"gitaly_calls\":2,\"gitaly_duration_s\":0.01107,\"redis_calls\":8,\"redis_allowed_cross_slot_calls\":1,\"redis_duration_s\":0.002509,\"redis_read_bytes\":491,\"redis_write_bytes\":969,\"redis_cache_calls\":1,\"redis_cache_duration_s\":0.001522,\"redis_cache_read_bytes\":109,\"redis_cache_write_bytes\":131,\"redis_repository_cache_calls\":2,\"redis_repository_cache_duration_s\":0.000288,\"redis_repository_cache_read_bytes\":202,\"redis_repository_cache_write_bytes\":137,\"redis_sessions_calls\":3,\"redis_sessions_allowed_cross_slot_calls\":1,\"redis_sessions_duration_s\":0.000518,\"redis_sessions_read_bytes\":180,\"redis_sessions_write_bytes\":595,\"redis_shared_state_calls\":2,\"redis_shared_state_duration_s\":0.000181,\"redis_shared_state_write_bytes\":106,\"db_count\":12,\"db_write_count\":0,\"db_cached_count\":0,\"db_replica_count\":0,\"db_primary_count\":12,\"db_main_count\":10,\"db_ci_count\":2,\"db_main_replica_count\":0,\"db_ci_replica_count\":0,\"db_replica_cached_count\":0,\"db_primary_cached_count\":0,\"db_main_cached_count\":0,\"db_ci_cached_count\":0,\"db_main_replica_cached_count\":0,\"db_ci_replica_cached_count\":0,\"db_replica_wal_count\":0,\"db_primary_wal_count\":0,\"db_main_wal_count\":0,\"db_ci_wal_count\":0,\"db_main_replica_wal_count\":0,\"db_ci_replica_wal_count\":0,\"db_replica_wal_cached_count\":0,\"db_primary_wal_cached_count\":0,\"db_main_wal_cached_count\":0,\"db_ci_wal_cached_count\":0,\"db_main_replica_wal_cached_count\":0,\"db_ci_replica_wal_cached_count\":0,\"db_replica_duration_s\":0.0,\"db_primary_duration_s\":0.023,\"db_main_duration_s\":0.011,\"db_ci_duration_s\":0.012,\"db_main_replica_duration_s\":0.0,\"db_ci_replica_duration_s\":0.0,\"cpu_s\":0.062784,\"mem_objects\":32232,\"mem_bytes\":2056312,\"mem_mallocs\":10556,\"mem_total_bytes\":3345592,\"pid\":338000,\"worker_id\":\"puma_1\",\"rate_limiting_gates\":[],\"db_duration_s\":0.02142,\"view_duration_s\":0.00034,\"duration_s\":0.10015}"
}
}
]
}