Json codec failed with data bigger then 4k size

Elastic Stack Logstash
1

I am trying to read the JSON files using logstash input plugin. For this I used codec as JSON, it is working fine until it encounters a file containing a size of more than 4k. It is giving a parsing error for this. I didn't find any configuration to specify a bigger size than 4k.

Error:

:exception=>#<LogStash::Json::ParserError: Unexpected end-of-input in field name
 at [Source: (byte[])" [truncated 3595 bytes]; line: 1, column: 4096]>}

You can reproduce this easily. Just use stdin data bigger than 4k size.

configuration is:

input {
stdin {
    id => "my_plugin_id" 
  }
}

filter {
  json {
        source => "message"       
   }
}
  
output {
      stdout { codec => rubydebug }    
    }

Can anyone help with this?

2

Can anyone please help with this? Is there any workaround for this?

3

Which input plugin are you using? Please share the config you are using and sample of the data you are trying to ingest and is giving you error.

4

Thanks for the reply,

I saw a Github issue for the same, it is having sample data as well.

I tried this and it is failing for me. I am not getting which config should i update to increase the limit.

5

Share the logstash pipeline you are trying to use and the message that is failing so it is possible to try to replicate the error with your message.

6

I do not think you can control it. It is an intermittent problem that is probably down in Jackson-core. Logstash 8.7 ships with jrjackson-1.2.34.jar and jackson-core-2.14.1.jar.

I just tried reading a 17,550 byte line from a file using stdin {} and a json filter had no problem parsing it.

7

I am trying to read data from the s3 bucket. But to reproduce I used stdin input. I am getting the same error in both places. when I reduce the input size below 4092 Char then it works but when it crossed the 4092 Char size then it fails.

input {
stdin {}
}
filter {

json {
source => "message"
}
}
output {
stdout { codec=>rubydebug }
}

Example JSON I am using is:

{"0": "d49d666e-8b6c-11e9-92bb-000ec6bbcc56", "1": "d49e503a-8b6c-11e9-a67b-000ec6bbcc56", "2": "d49e503b-8b6c-11e9-927c-000ec6bbcc56", "3": "d49e503c-8b6c-11e9-9a53-000ec6bbcc56", "4": "d49e503d-8b6c-11e9-b1da-000ec6bbcc56", "5": "d49e503e-8b6c-11e9-99a1-000ec6bbcc56", "6": "d49e503f-8b6c-11e9-9364-000ec6bbcc56", "7": "d49e5040-8b6c-11e9-8d42-000ec6bbcc56", "8": "d49e5041-8b6c-11e9-8fdd-000ec6bbcc56", "9": "d49e5042-8b6c-11e9-b1a2-000ec6bbcc56", "10": "d49e5043-8b6c-11e9-a4f6-000ec6bbcc56", "11": "d49e5044-8b6c-11e9-9d49-000ec6bbcc56", "12": "d49e5045-8b6c-11e9-8b05-000ec6bbcc56", "13": "d49e5046-8b6c-11e9-883c-000ec6bbcc56", "14": "d49e5047-8b6c-11e9-8535-000ec6bbcc56", "15": "d49e5048-8b6c-11e9-9a24-000ec6bbcc56", "16": "d49e5049-8b6c-11e9-bd86-000ec6bbcc56", "17": "d49e7730-8b6c-11e9-a17c-000ec6bbcc56", "18": "d49e7731-8b6c-11e9-82ee-000ec6bbcc56", "19": "d49e7732-8b6c-11e9-8eaa-000ec6bbcc56", "20": "d49e7733-8b6c-11e9-a5a4-000ec6bbcc56", "21": "d49e7734-8b6c-11e9-93f6-000ec6bbcc56", "22": "d49e7735-8b6c-11e9-a842-000ec6bbcc56", "23": "d49e7736-8b6c-11e9-be30-000ec6bbcc56", "24": "d49e7737-8b6c-11e9-9fac-000ec6bbcc56", "25": "d49e7738-8b6c-11e9-ad55-000ec6bbcc56", "26": "d49e7739-8b6c-11e9-9062-000ec6bbcc56", "27": "d49e773a-8b6c-11e9-95e8-000ec6bbcc56", "28": "d49e773b-8b6c-11e9-90b6-000ec6bbcc56", "29": "d49e773c-8b6c-11e9-839d-000ec6bbcc56", "30": "d49e773d-8b6c-11e9-b444-000ec6bbcc56", "31": "d49e773e-8b6c-11e9-bb52-000ec6bbcc56", "32": "d49e773f-8b6c-11e9-acf9-000ec6bbcc56", "33": "d49e7740-8b6c-11e9-b5c2-000ec6bbcc56", "34": "d49e7741-8b6c-11e9-8a29-000ec6bbcc56", "35": "d49e7742-8b6c-11e9-90b4-000ec6bbcc56", "36": "d49e7743-8b6c-11e9-9625-000ec6bbcc56", "37": "d49e7744-8b6c-11e9-b195-000ec6bbcc56", "38": "d49e7745-8b6c-11e9-978a-000ec6bbcc56", "39": "d49e7746-8b6c-11e9-a8e7-000ec6bbcc56", "40": "d49e7747-8b6c-11e9-8c81-000ec6bbcc56", "41": "d49e7748-8b6c-11e9-bd0b-000ec6bbcc56", "42": "d49e7749-8b6c-11e9-8335-000ec6bbcc56", "43": "d49e774a-8b6c-11e9-bfe1-000ec6bbcc56", "44": "d49e774b-8b6c-11e9-afe2-000ec6bbcc56", "45": "d49e774c-8b6c-11e9-a014-000ec6bbcc56", "46": "d49e774d-8b6c-11e9-8c07-000ec6bbcc56", "47": "d49e774e-8b6c-11e9-8e5e-000ec6bbcc56", "48": "d49e774f-8b6c-11e9-b0c7-000ec6bbcc56", "49": "d49e7750-8b6c-11e9-8e66-000ec6bbcc56", "50": "d49e7751-8b6c-11e9-adb8-000ec6bbcc56", "51": "d49e7752-8b6c-11e9-a828-000ec6bbcc56", "52": "d49e7753-8b6c-11e9-97e1-000ec6bbcc56", "53": "d49e7754-8b6c-11e9-83f7-000ec6bbcc56", "54": "d49e7755-8b6c-11e9-a902-000ec6bbcc56", "55": "d49e7756-8b6c-11e9-bce9-000ec6bbcc56", "56": "d49e7757-8b6c-11e9-a206-000ec6bbcc56", "57": "d49e7758-8b6c-11e9-8359-000ec6bbcc56", "58": "d49e7759-8b6c-11e9-95f6-000ec6bbcc56", "59": "d49e775a-8b6c-11e9-a425-000ec6bbcc56", "60": "d49e775b-8b6c-11e9-9ce6-000ec6bbcc56", "61": "d49e775c-8b6c-11e9-aaab-000ec6bbcc56", "62": "d49e775d-8b6c-11e9-9c08-000ec6bbcc56", "63": "d49e775e-8b6c-11e9-8e39-000ec6bbcc56", "64": "d49e775f-8b6c-11e9-8d00-000ec6bbcc56", "65": "d49e7760-8b6c-11e9-a567-000ec6bbcc56", "66": "d49e7761-8b6c-11e9-8f70-000ec6bbcc56", "67": "d49e7762-8b6c-11e9-b1e5-000ec6bbcc56", "68": "d49e7763-8b6c-11e9-b91a-000ec6bbcc56", "69": "d49e7764-8b6c-11e9-8000-000ec6bbcc56", "70": "d49e7765-8b6c-11e9-a66c-000ec6bbcc56", "71": "d49e7766-8b6c-11e9-8563-000ec6bbcc56", "72": "d49e7767-8b6c-11e9-b2de-000ec6bbcc56", "73": "d49e7768-8b6c-11e9-9c79-000ec6bbcc56", "74": "d49e7769-8b6c-11e9-b3bd-000ec6bbcc56", "75": "d49e776a-8b6c-11e9-8b7c-000ec6bbcc56", "76": "d49e776b-8b6c-11e9-8942-000ec6bbcc56", "77": "d49e776c-8b6c-11e9-860b-000ec6bbcc56", "78": "d49e776d-8b6c-11e9-b5fd-000ec6bbcc56", "79": "d49e776e-8b6c-11e9-a4d9-000ec6bbcc56", "80": "d49e776f-8b6c-11e9-a11f-000ec6bbcc56", "81": "d49e7770-8b6c-11e9-b7d0-000ec6bbcc56", "82": "d49e7771-8b6c-11e9-82d2-000ec6bbcc56", "83": "d49e7772-8b6c-11e9-9d28-000ec6bbcc56", "84": "d49e7773-8b6c-11e9-a777-000ec6bbcc56", "85": "d49e7774-8b6c-11e9-ab26-000ec6bbcc56", "86": "d49e7775-8b6c-11e9-91a1-000ec6bbcc56", "87": "d49e7776-8b6c-11e9-a115-000ec6bbcc56", "88": "d49e7777-8b6c-11e9-9859-000ec6bbcc56", "89": "d49e7778-8b6c-11e9-a193-000ec6bbcc56"}

You can easily reproduce this with the above-mentioned config and Json. it will give parsing error as

[WARN ] 2023-04-25 19:12:12.015 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>"{\"0\": \"d49d666e-8b6c-11e9-92bb-000ec6bbcc56\", \"1\": \"d49e503a-8b6c-11e9-a67b-000ec6bbcc56\", \"2\": \"d49e503b-8b6c-11e9-927c-000ec6bbcc56\", \"3\": \"d49e503c-8b6c-11e9-9a53-000ec6bbcc56\", \"4\": \"d49e503d-8b6c-11e9-b1da-000ec6bbcc56\", \"5\": \"d49e503e-8b6c-11e9-99a1-000ec6bbcc56\", \"6\": \"d49e503f-8b6c-11e9-9364-000ec6bbcc56\", \"7\": \"d49e5040-8b6c-11e9-8d42-000ec6bbcc56\", \"8\": \"d49e5041-8b6c-11e9-8fdd-000ec6bbcc56\", \"9\": \"d49e5042-8b6c-11e9-b1a2-000ec6bbcc56\", \"10\": \"d49e5043-8b6c-11e9-a4f6-000ec6bbcc56\", \"11\": \"d49e5044-8b6c-11e9-9d49-000ec6bbcc56\", \"12\": \"d49e5045-8b6c-11e9-8b05-000ec6bbcc56\", \"13\": \"d49e5046-8b6c-11e9-883c-000ec6bbcc56\", \"14\": \"d49e5047-8b6c-11e9-8535-000ec6bbcc56\", \"15\": \"d49e5048-8b6c-11e9-9a24-000ec6bbcc56\", \"16\": \"d49e5049-8b6c-11e9-bd86-000ec6bbcc56\", \"17\": \"d49e7730-8b6c-11e9-a17c-000ec6bbcc56\", \"18\": \"d49e7731-8b6c-11e9-82ee-000ec6bbcc56\", \"19\": \"d49e7732-8b6c-11e9-8eaa-000ec6bbcc56\", \"20\": \"d49e7733-8b6c-11e9-a5a4-000ec6bbcc56\", \"21\": \"d49e7734-8b6c-11e9-93f6-000ec6bbcc56\", \"22\": \"d49e7735-8b6c-11e9-a842-000ec6bbcc56\", \"23\": \"d49e7736-8b6c-11e9-be30-000ec6bbcc56\", \"24\": \"d49e7737-8b6c-11e9-9fac-000ec6bbcc56\", \"25\": \"d49e7738-8b6c-11e9-ad55-000ec6bbcc56\", \"26\": \"d49e7739-8b6c-11e9-9062-000ec6bbcc56\", \"27\": \"d49e773a-8b6c-11e9-95e8-000ec6bbcc56\", \"28\": \"d49e773b-8b6c-11e9-90b6-000ec6bbcc56\", \"29\": \"d49e773c-8b6c-11e9-839d-000ec6bbcc56\", \"30\": \"d49e773d-8b6c-11e9-b444-000ec6bbcc56\", \"31\": \"d49e773e-8b6c-11e9-bb52-000ec6bbcc56\", \"32\": \"d49e773f-8b6c-11e9-acf9-000ec6bbcc56\", \"33\": \"d49e7740-8b6c-11e9-b5c2-000ec6bbcc56\", \"34\": \"d49e7741-8b6c-11e9-8a29-000ec6bbcc56\", \"35\": \"d49e7742-8b6c-11e9-90b4-000ec6bbcc56\", \"36\": \"d49e7743-8b6c-11e9-9625-000ec6bbcc56\", \"37\": \"d49e7744-8b6c-11e9-b195-000ec6bbcc56\", \"38\": \"d49e7745-8b6c-11e9-978a-000ec6bbcc56\", \"39\": \"d49e7746-8b6c-11e9-a8e7-000ec6bbcc56\", \"40\": \"d49e7747-8b6c-11e9-8c81-000ec6bbcc56\", \"41\": \"d49e7748-8b6c-11e9-bd0b-000ec6bbcc56\", \"42\": \"d49e7749-8b6c-11e9-8335-000ec6bbcc56\", \"43\": \"d49e774a-8b6c-11e9-bfe1-000ec6bbcc56\", \"44\": \"d49e774b-8b6c-11e9-afe2-000ec6bbcc56\", \"45\": \"d49e774c-8b6c-11e9-a014-000ec6bbcc56\", \"46\": \"d49e774d-8b6c-11e9-8c07-000ec6bbcc56\", \"47\": \"d49e774e-8b6c-11e9-8e5e-000ec6bbcc56\", \"48\": \"d49e774f-8b6c-11e9-b0c7-000ec6bbcc56\", \"49\": \"d49e7750-8b6c-11e9-8e66-000ec6bbcc56\", \"50\": \"d49e7751-8b6c-11e9-adb8-000ec6bbcc56\", \"51\": \"d49e7752-8b6c-11e9-a828-000ec6bbcc56\", \"52\": \"d49e7753-8b6c-11e9-97e1-000ec6bbcc56\", \"53\": \"d49e7754-8b6c-11e9-83f7-000ec6bbcc56\", \"54\": \"d49e7755-8b6c-11e9-a902-000ec6bbcc56\", \"55\": \"d49e7756-8b6c-11e9-bce9-000ec6bbcc56\", \"56\": \"d49e7757-8b6c-11e9-a206-000ec6bbcc56\", \"57\": \"d49e7758-8b6c-11e9-8359-000ec6bbcc56\", \"58\": \"d49e7759-8b6c-11e9-95f6-000ec6bbcc56\", \"59\": \"d49e775a-8b6c-11e9-a425-000ec6bbcc56\", \"60\": \"d49e775b-8b6c-11e9-9ce6-000ec6bbcc56\", \"61\": \"d49e775c-8b6c-11e9-aaab-000ec6bbcc56\", \"62\": \"d49e775d-8b6c-11e9-9c08-000ec6bbcc56\", \"63\": \"d49e775e-8b6c-11e9-8e39-000ec6bbcc56\", \"64\": \"d49e775f-8b6c-11e9-8d00-000ec6bbcc56\", \"65\": \"d49e7760-8b6c-11e9-a567-000ec6bbcc56\", \"66\": \"d49e7761-8b6c-11e9-8f70-000ec6bbcc56\", \"67\": \"d49e7762-8b6c-11e9-b1e5-000ec6bbcc56\", \"68\": \"d49e7763-8b6c-11e9-b91a-000ec6bbcc56\", \"69\": \"d49e7764-8b6c-11e9-8000-000ec6bbcc56\", \"70\": \"d49e7765-8b6c-11e9-a66c-000ec6bbcc56\", \"71\": \"d49e7766-8b6c-11e9-8563-000ec6bbcc56\", \"72\": \"d49e7767-8b6c-11e9-b2de-000ec6bbcc56\", \"73\": \"d49e7768-8b6c-11e9-9c79-000ec6bbcc56\", \"74\": \"d49e7769-8b6c-11e9-b3bd-000ec6bbcc56\", \"75\": \"d49e776a-8b6c-11e9-8b7c-000ec6bbcc56\", \"76\": \"d49e776b-8b6c-11e9-8942-000ec6bbcc56\", \"77\": \"d49e776c-8b6c-11e9-860b-000ec6bbcc56\", \"78\": \"d49e776d-8b6c-11e9-b5fd-000ec6bbcc56\", \"79\": \"d49e776e-8b6c-11e9-a4d9-000ec6bbcc56\", \"80\": \"d49e776f-8b6c-11e9-a11f-000ec6bbcc56\", \"81\": \"d49e7770-8b6c-11e9-b7d0-000ec6bbcc56\", \"82\": \"d49e7771-8b6c-11e9-82d2-000ec6bbcc56\", \"83\": \"d49e7772-8b6c-11e9-9d28-000ec6bbcc56\", \"84\": \"d49e7773-8b6c-11e9-a777-000ec6bbcc56\", \"85\": \"d49e7774-8b6c-11e9-ab26-000ec6bbcc56\", \"86\": \"d49e7775-8b6c-11e9-91a1-000ec6bbcc56\", \"87\": \"d49e7776-8b6c-11e9-a115-000ec6bbcc56\", \"88\": \"d49e7777-8b6c-11e9-9859-000ec6bbcc56\", \"89\": \"d49", :exception=>#<LogStash::Json::ParserError: Unexpected end-of-input in VALUE_STRING
 at [Source: (byte[])"{"0": "d49d666e-8b6c-11e9-92bb-000ec6bbcc56", "1": "d49e503a-8b6c-11e9-a67b-000ec6bbcc56", "2": "d49e503b-8b6c-11e9-927c-000ec6bbcc56", "3": "d49e503c-8b6c-11e9-9a53-000ec6bbcc56", "4": "d49e503d-8b6c-11e9-b1da-000ec6bbcc56", "5": "d49e503e-8b6c-11e9-99a1-000ec6bbcc56", "6": "d49e503f-8b6c-11e9-9364-000ec6bbcc56", "7": "d49e5040-8b6c-11e9-8d42-000ec6bbcc56", "8": "d49e5041-8b6c-11e9-8fdd-000ec6bbcc56", "9": "d49e5042-8b6c-11e9-b1a2-000ec6bbcc56", "10": "d49e5043-8b6c-11e9-a4f6-000ec6bbcc56", "11"[truncated 3595 bytes]; line: 1, column: 4096]>}

With below mentioned JSON it will work because i reduced the size to 3992 Chars.

{"0": "d49d666e-8b6c-11e9-92bb-000ec6bbcc56", "1": "d49e503a-8b6c-11e9-a67b-000ec6bbcc56", "2": "d49e503b-8b6c-11e9-927c-000ec6bbcc56", "3": "d49e503c-8b6c-11e9-9a53-000ec6bbcc56", "4": "d49e503d-8b6c-11e9-b1da-000ec6bbcc56", "5": "d49e503e-8b6c-11e9-99a1-000ec6bbcc56", "6": "d49e503f-8b6c-11e9-9364-000ec6bbcc56", "7": "d49e5040-8b6c-11e9-8d42-000ec6bbcc56", "8": "d49e5041-8b6c-11e9-8fdd-000ec6bbcc56", "9": "d49e5042-8b6c-11e9-b1a2-000ec6bbcc56", "10": "d49e5043-8b6c-11e9-a4f6-000ec6bbcc56", "11": "d49e5044-8b6c-11e9-9d49-000ec6bbcc56", "12": "d49e5045-8b6c-11e9-8b05-000ec6bbcc56", "13": "d49e5046-8b6c-11e9-883c-000ec6bbcc56", "14": "d49e5047-8b6c-11e9-8535-000ec6bbcc56", "15": "d49e5048-8b6c-11e9-9a24-000ec6bbcc56", "16": "d49e5049-8b6c-11e9-bd86-000ec6bbcc56", "17": "d49e7730-8b6c-11e9-a17c-000ec6bbcc56", "18": "d49e7731-8b6c-11e9-82ee-000ec6bbcc56", "19": "d49e7732-8b6c-11e9-8eaa-000ec6bbcc56", "20": "d49e7733-8b6c-11e9-a5a4-000ec6bbcc56", "21": "d49e7734-8b6c-11e9-93f6-000ec6bbcc56", "22": "d49e7735-8b6c-11e9-a842-000ec6bbcc56", "23": "d49e7736-8b6c-11e9-be30-000ec6bbcc56", "24": "d49e7737-8b6c-11e9-9fac-000ec6bbcc56", "25": "d49e7738-8b6c-11e9-ad55-000ec6bbcc56", "26": "d49e7739-8b6c-11e9-9062-000ec6bbcc56", "27": "d49e773a-8b6c-11e9-95e8-000ec6bbcc56", "28": "d49e773b-8b6c-11e9-90b6-000ec6bbcc56", "29": "d49e773c-8b6c-11e9-839d-000ec6bbcc56", "30": "d49e773d-8b6c-11e9-b444-000ec6bbcc56", "31": "d49e773e-8b6c-11e9-bb52-000ec6bbcc56", "32": "d49e773f-8b6c-11e9-acf9-000ec6bbcc56", "33": "d49e7740-8b6c-11e9-b5c2-000ec6bbcc56", "34": "d49e7741-8b6c-11e9-8a29-000ec6bbcc56", "35": "d49e7742-8b6c-11e9-90b4-000ec6bbcc56", "36": "d49e7743-8b6c-11e9-9625-000ec6bbcc56", "37": "d49e7744-8b6c-11e9-b195-000ec6bbcc56", "38": "d49e7745-8b6c-11e9-978a-000ec6bbcc56", "39": "d49e7746-8b6c-11e9-a8e7-000ec6bbcc56", "40": "d49e7747-8b6c-11e9-8c81-000ec6bbcc56", "41": "d49e7748-8b6c-11e9-bd0b-000ec6bbcc56", "42": "d49e7749-8b6c-11e9-8335-000ec6bbcc56", "43": "d49e774a-8b6c-11e9-bfe1-000ec6bbcc56", "44": "d49e774b-8b6c-11e9-afe2-000ec6bbcc56", "45": "d49e774c-8b6c-11e9-a014-000ec6bbcc56", "46": "d49e774d-8b6c-11e9-8c07-000ec6bbcc56", "47": "d49e774e-8b6c-11e9-8e5e-000ec6bbcc56", "48": "d49e774f-8b6c-11e9-b0c7-000ec6bbcc56", "49": "d49e7750-8b6c-11e9-8e66-000ec6bbcc56", "50": "d49e7751-8b6c-11e9-adb8-000ec6bbcc56", "51": "d49e7752-8b6c-11e9-a828-000ec6bbcc56", "52": "d49e7753-8b6c-11e9-97e1-000ec6bbcc56", "53": "d49e7754-8b6c-11e9-83f7-000ec6bbcc56", "54": "d49e7755-8b6c-11e9-a902-000ec6bbcc56", "55": "d49e7756-8b6c-11e9-bce9-000ec6bbcc56", "56": "d49e7757-8b6c-11e9-a206-000ec6bbcc56", "57": "d49e7758-8b6c-11e9-8359-000ec6bbcc56", "58": "d49e7759-8b6c-11e9-95f6-000ec6bbcc56", "59": "d49e775a-8b6c-11e9-a425-000ec6bbcc56", "60": "d49e775b-8b6c-11e9-9ce6-000ec6bbcc56", "61": "d49e775c-8b6c-11e9-aaab-000ec6bbcc56", "62": "d49e775d-8b6c-11e9-9c08-000ec6bbcc56", "63": "d49e775e-8b6c-11e9-8e39-000ec6bbcc56", "64": "d49e775f-8b6c-11e9-8d00-000ec6bbcc56", "65": "d49e7760-8b6c-11e9-a567-000ec6bbcc56", "66": "d49e7761-8b6c-11e9-8f70-000ec6bbcc56", "67": "d49e7762-8b6c-11e9-b1e5-000ec6bbcc56", "68": "d49e7763-8b6c-11e9-b91a-000ec6bbcc56", "69": "d49e7764-8b6c-11e9-8000-000ec6bbcc56", "70": "d49e7765-8b6c-11e9-a66c-000ec6bbcc56", "71": "d49e7766-8b6c-11e9-8563-000ec6bbcc56", "72": "d49e7767-8b6c-11e9-b2de-000ec6bbcc56", "73": "d49e7768-8b6c-11e9-9c79-000ec6bbcc56", "74": "d49e7769-8b6c-11e9-b3bd-000ec6bbcc56", "75": "d49e776a-8b6c-11e9-8b7c-000ec6bbcc56", "76": "d49e776b-8b6c-11e9-8942-000ec6bbcc56", "77": "d49e776c-8b6c-11e9-860b-000ec6bbcc56", "78": "d49e776d-8b6c-11e9-b5fd-000ec6bbcc56", "79": "d49e776e-8b6c-11e9-a4d9-000ec6bbcc56", "80": "d49e776f-8b6c-11e9-a11f-000ec6bbcc56", "81": "d49e7770-8b6c-11e9-b7d0-000ec6bbcc56", "82": "d49e7771-8b6c-11e9-82d2-000ec6bbcc56", "83": "d49e7772-8b6c-11e9-9d28-000ec6bbcc56", "84": "d49e7773-8b6c-11e9-a777-000ec6bbcc56", "85": "d49e7774-8b6c-11e9-ab26-000ec6bbcc56", "86": "d49e7775-8b6c-11e9-91a1-000ec6bbcc56"}
8

@Badger/ @leandrojmp :
Is there any work around to fix this?

9

Which Logstash version are you using? I had no issue parsing the first json you shared using logstash 8.6.2.

This is part of my output where you can see that the json was parsed and the last object, the field 89 is present.

[2023-04-25T16:21:50,366][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
{
            "28" => "d49e773b-8b6c-11e9-90b6-000ec6bbcc56",
            "33" => "d49e7740-8b6c-11e9-b5c2-000ec6bbcc56",
          "host" => "stout",
            "70" => "d49e7765-8b6c-11e9-a66c-000ec6bbcc56",
            "79" => "d49e776e-8b6c-11e9-a4d9-000ec6bbcc56",
            "83" => "d49e7772-8b6c-11e9-9d28-000ec6bbcc56",
      "@version" => "1",
            "45" => "d49e774c-8b6c-11e9-a014-000ec6bbcc56",
            "53" => "d49e7754-8b6c-11e9-83f7-000ec6bbcc56",
            "22" => "d49e7735-8b6c-11e9-a842-000ec6bbcc56",
            "43" => "d49e774a-8b6c-11e9-bfe1-000ec6bbcc56",
            "84" => "d49e7773-8b6c-11e9-a777-000ec6bbcc56",
            "89" => "d49e7778-8b6c-11e9-a193-000ec6bbcc56",
10

This is really strange, I am using 8.7.0.

I am pasting all information that I am using in reproducing this.

root@ip-10-197-102-49:/usr/share/logstash/bin# ./logstash --version
Using bundled JDK: /usr/share/logstash/jdk
logstash 8.7.0
root@ip-10-197-102-49:/usr/share/logstash/bin# cat /etc/logstash/conf.d/logstash.conf.try
input {
stdin {
    id => "my_plugin_id"
    ecs_compatibility => "disabled"
  }
}

filter {
  json {
      source => "message"
  }
}

output {
      stdout { codec => rubydebug }
    }
root@ip-10-197-102-49:/usr/share/logstash/bin# ./logstash -f /etc/logstash/conf.d/logstash.conf.try
Using bundled JDK: /usr/share/logstash/jdk
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2023-04-25 19:38:14.176 [main] runner - NOTICE: Running Logstash as superuser is not recommended and won't be allowed in the future. Set 'allow_superuser' to 'false' to avoid startup errors in future releases.
[INFO ] 2023-04-25 19:38:14.189 [main] runner - Starting Logstash {"logstash.version"=>"8.7.0", "jruby.version"=>"jruby 9.3.10.0 (2.6.8) 2023-02-01 107b2e6697 OpenJDK 64-Bit Server VM 17.0.6+10 on 17.0.6+10 +indy +jit [x86_64-linux]"}
[INFO ] 2023-04-25 19:38:14.192 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[WARN ] 2023-04-25 19:38:14.416 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2023-04-25 19:38:15.286 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[INFO ] 2023-04-25 19:38:15.739 [Converge PipelineAction::Create<main>] Reflections - Reflections took 173 ms to scan 1 urls, producing 132 keys and 462 values
[INFO ] 2023-04-25 19:38:16.104 [Converge PipelineAction::Create<main>] javapipeline - Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[INFO ] 2023-04-25 19:38:16.123 [[main]-pipeline-manager] json - ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[INFO ] 2023-04-25 19:38:16.138 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["/etc/logstash/conf.d/logstash.conf.try"], :thread=>"#<Thread:0x5bc9b336@/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[INFO ] 2023-04-25 19:38:16.757 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>0.62}
[INFO ] 2023-04-25 19:38:16.782 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
The stdin plugin is now waiting for input:
[INFO ] 2023-04-25 19:38:16.795 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
{"0": "d49d666e-8b6c-11e9-92bb-000ec6bbcc56", "1": "d49e503a-8b6c-11e9-a67b-000ec6bbcc56", "2": "d49e503b-8b6c-11e9-927c-000ec6bbcc56", "3": "d49e503c-8b6c-11e9-9a53-000ec6bbcc56", "4": "d49e503d-8b6c-11e9-b1da-000ec6bbcc56", "5": "d49e503e-8b6c-11e9-99a1-000ec6bbcc56", "6": "d49e503f-8b6c-11e9-9364-000ec6bbcc56", "7": "d49e5040-8b6c-11e9-8d42-000ec6bbcc56", "8": "d49e5041-8b6c-11e9-8fdd-000ec6bbcc56", "9": "d49e5042-8b6c-11e9-b1a2-000ec6bbcc56", "10": "d49e5043-8b6c-11e9-a4f6-000ec6bbcc56", "11": "d49e5044-8b6c-11e9-9d49-000ec6bbcc56", "12": "d49e5045-8b6c-11e9-8b05-000ec6bbcc56", "13": "d49e5046-8b6c-11e9-883c-000ec6bbcc56", "14": "d49e5047-8b6c-11e9-8535-000ec6bbcc56", "15": "d49e5048-8b6c-11e9-9a24-000ec6bbcc56", "16": "d49e5049-8b6c-11e9-bd86-000ec6bbcc56", "17": "d49e7730-8b6c-11e9-a17c-000ec6bbcc56", "18": "d49e7731-8b6c-11e9-82ee-000ec6bbcc56", "19": "d49e7732-8b6c-11e9-8eaa-000ec6bbcc56", "20": "d49e7733-8b6c-11e9-a5a4-000ec6bbcc56", "21": "d49e7734-8b6c-11e9-93f6-000ec6bbcc56", "22": "d49e7735-8b6c-11e9-a842-000ec6bbcc56", "23": "d49e7736-8b6c-11e9-be30-000ec6bbcc56", "24": "d49e7737-8b6c-11e9-9fac-000ec6bbcc56", "25": "d49e7738-8b6c-11e9-ad55-000ec6bbcc56", "26": "d49e7739-8b6c-11e9-9062-000ec6bbcc56", "27": "d49e773a-8b6c-11e9-95e8-000ec6bbcc56", "28": "d49e773b-8b6c-11e9-90b6-000ec6bbcc56", "29": "d49e773c-8b6c-11e9-839d-000ec6bbcc56", "30": "d49e773d-8b6c-11e9-b444-000ec6bbcc56", "31": "d49e773e-8b6c-11e9-bb52-000ec6bbcc56", "32": "d49e773f-8b6c-11e9-acf9-000ec6bbcc56", "33": "d49e7740-8b6c-11e9-b5c2-000ec6bbcc56", "34": "d49e7741-8b6c-11e9-8a29-000ec6bbcc56", "35": "d49e7742-8b6c-11e9-90b4-000ec6bbcc56", "36": "d49e7743-8b6c-11e9-9625-000ec6bbcc56", "37": "d49e7744-8b6c-11e9-b195-000ec6bbcc56", "38": "d49e7745-8b6c-11e9-978a-000ec6bbcc56", "39": "d49e7746-8b6c-11e9-a8e7-000ec6bbcc56", "40": "d49e7747-8b6c-11e9-8c81-000ec6bbcc56", "41": "d49e7748-8b6c-11e9-bd0b-000ec6bbcc56", "42": "d49e7749-8b6c-11e9-8335-000ec6bbcc56", "43": "d49e774a-8b6c-11e9-bfe1-000ec6bbcc56", "44": "d49e774b-8b6c-11e9-afe2-000ec6bbcc56", "45": "d49e774c-8b6c-11e9-a014-000ec6bbcc56", "46": "d49e774d-8b6c-11e9-8c07-000ec6bbcc56", "47": "d49e774e-8b6c-11e9-8e5e-000ec6bbcc56", "48": "d49e774f-8b6c-11e9-b0c7-000ec6bbcc56", "49": "d49e7750-8b6c-11e9-8e66-000ec6bbcc56", "50": "d49e7751-8b6c-11e9-adb8-000ec6bbcc56", "51": "d49e7752-8b6c-11e9-a828-000ec6bbcc56", "52": "d49e7753-8b6c-11e9-97e1-000ec6bbcc56", "53": "d49e7754-8b6c-11e9-83f7-000ec6bbcc56", "54": "d49e7755-8b6c-11e9-a902-000ec6bbcc56", "55": "d49e7756-8b6c-11e9-bce9-000ec6bbcc56", "56": "d49e7757-8b6c-11e9-a206-000ec6bbcc56", "57": "d49e7758-8b6c-11e9-8359-000ec6bbcc56", "58": "d49e7759-8b6c-11e9-95f6-000ec6bbcc56", "59": "d49e775a-8b6c-11e9-a425-000ec6bbcc56", "60": "d49e775b-8b6c-11e9-9ce6-000ec6bbcc56", "61": "d49e775c-8b6c-11e9-aaab-000ec6bbcc56", "62": "d49e775d-8b6c-11e9-9c08-000ec6bbcc56", "63": "d49e775e-8b6c-11e9-8e39-000ec6bbcc56", "64": "d49e775f-8b6c-11e9-8d00-000ec6bbcc56", "65": "d49e7760-8b6c-11e9-a567-000ec6bbcc56", "66": "d49e7761-8b6c-11e9-8f70-000ec6bbcc56", "67": "d49e7762-8b6c-11e9-b1e5-000ec6bbcc56", "68": "d49e7763-8b6c-11e9-b91a-000ec6bbcc56", "69": "d49e7764-8b6c-11e9-8000-000ec6bbcc56", "70": "d49e7765-8b6c-11e9-a66c-000ec6bbcc56", "71": "d49e7766-8b6c-11e9-8563-000ec6bbcc56", "72": "d49e7767-8b6c-11e9-b2de-000ec6bbcc56", "73": "d49e7768-8b6c-11e9-9c79-000ec6bbcc56", "74": "d49e7769-8b6c-11e9-b3bd-000ec6bbcc56", "75": "d49e776a-8b6c-11e9-8b7c-000ec6bbcc56", "76": "d49e776b-8b6c-11e9-8942-000ec6bbcc56", "77": "d49e776c-8b6c-11e9-860b-000ec6bbcc56", "78": "d49e776d-8b6c-11e9-b5fd-000ec6bbcc56", "79": "d49e776e-8b6c-11e9-a4d9-000ec6bbcc56", "80": "d49e776f-8b6c-11e9-a11f-000ec6bbcc56", "81": "d49e7770-8b6c-11e9-b7d0-000ec6bbcc56", "82": "d49e7771-8b6c-11e9-82d2-000ec6bbcc56", "83": "d49e7772-8b6c-11e9-9d28-000ec6bbcc56", "84": "d49e7773-8b6c-11e9-a777-000ec6bbcc56", "85": "d49e7774-8b6c-11e9-ab26-000ec6bbcc56", "86": "d49e7775-8b6c-11e9-91a1-000ec6bbcc56", "87": "d49e7776-8b6c-11e9-a115-000ec6bbcc56", "88": "d49e7777-8b6c-11e9-9859-000ec6bbcc56", "89": "d49e7778-8b6c-11e9-a193-000ec6bbcc56"}
[WARN ] 2023-04-25 19:38:37.224 [[main]>worker0] json - Error parsing json {:source=>"message", :raw=>"{\"0\": \"d49d666e-8b6c-11e9-92bb-000ec6bbcc56\", \"1\": \"d49e503a-8b6c-11e9-a67b-000ec6bbcc56\", \"2\": \"d49e503b-8b6c-11e9-927c-000ec6bbcc56\", \"3\": \"d49e503c-8b6c-11e9-9a53-000ec6bbcc56\", \"4\": \"d49e503d-8b6c-11e9-b1da-000ec6bbcc56\", \"5\": \"d49e503e-8b6c-11e9-99a1-000ec6bbcc56\", \"6\": \"d49e503f-8b6c-11e9-9364-000ec6bbcc56\", \"7\": \"d49e5040-8b6c-11e9-8d42-000ec6bbcc56\", \"8\": \"d49e5041-8b6c-11e9-8fdd-000ec6bbcc56\", \"9\": \"d49e5042-8b6c-11e9-b1a2-000ec6bbcc56\", \"10\": \"d49e5043-8b6c-11e9-a4f6-000ec6bbcc56\", \"11\": \"d49e5044-8b6c-11e9-9d49-000ec6bbcc56\", \"12\": \"d49e5045-8b6c-11e9-8b05-000ec6bbcc56\", \"13\": \"d49e5046-8b6c-11e9-883c-000ec6bbcc56\", \"14\": \"d49e5047-8b6c-11e9-8535-000ec6bbcc56\", \"15\": \"d49e5048-8b6c-11e9-9a24-000ec6bbcc56\", \"16\": \"d49e5049-8b6c-11e9-bd86-000ec6bbcc56\", \"17\": \"d49e7730-8b6c-11e9-a17c-000ec6bbcc56\", \"18\": \"d49e7731-8b6c-11e9-82ee-000ec6bbcc56\", \"19\": \"d49e7732-8b6c-11e9-8eaa-000ec6bbcc56\", \"20\": \"d49e7733-8b6c-11e9-a5a4-000ec6bbcc56\", \"21\": \"d49e7734-8b6c-11e9-93f6-000ec6bbcc56\", \"22\": \"d49e7735-8b6c-11e9-a842-000ec6bbcc56\", \"23\": \"d49e7736-8b6c-11e9-be30-000ec6bbcc56\", \"24\": \"d49e7737-8b6c-11e9-9fac-000ec6bbcc56\", \"25\": \"d49e7738-8b6c-11e9-ad55-000ec6bbcc56\", \"26\": \"d49e7739-8b6c-11e9-9062-000ec6bbcc56\", \"27\": \"d49e773a-8b6c-11e9-95e8-000ec6bbcc56\", \"28\": \"d49e773b-8b6c-11e9-90b6-000ec6bbcc56\", \"29\": \"d49e773c-8b6c-11e9-839d-000ec6bbcc56\", \"30\": \"d49e773d-8b6c-11e9-b444-000ec6bbcc56\", \"31\": \"d49e773e-8b6c-11e9-bb52-000ec6bbcc56\", \"32\": \"d49e773f-8b6c-11e9-acf9-000ec6bbcc56\", \"33\": \"d49e7740-8b6c-11e9-b5c2-000ec6bbcc56\", \"34\": \"d49e7741-8b6c-11e9-8a29-000ec6bbcc56\", \"35\": \"d49e7742-8b6c-11e9-90b4-000ec6bbcc56\", \"36\": \"d49e7743-8b6c-11e9-9625-000ec6bbcc56\", \"37\": \"d49e7744-8b6c-11e9-b195-000ec6bbcc56\", \"38\": \"d49e7745-8b6c-11e9-978a-000ec6bbcc56\", \"39\": \"d49e7746-8b6c-11e9-a8e7-000ec6bbcc56\", \"40\": \"d49e7747-8b6c-11e9-8c81-000ec6bbcc56\", \"41\": \"d49e7748-8b6c-11e9-bd0b-000ec6bbcc56\", \"42\": \"d49e7749-8b6c-11e9-8335-000ec6bbcc56\", \"43\": \"d49e774a-8b6c-11e9-bfe1-000ec6bbcc56\", \"44\": \"d49e774b-8b6c-11e9-afe2-000ec6bbcc56\", \"45\": \"d49e774c-8b6c-11e9-a014-000ec6bbcc56\", \"46\": \"d49e774d-8b6c-11e9-8c07-000ec6bbcc56\", \"47\": \"d49e774e-8b6c-11e9-8e5e-000ec6bbcc56\", \"48\": \"d49e774f-8b6c-11e9-b0c7-000ec6bbcc56\", \"49\": \"d49e7750-8b6c-11e9-8e66-000ec6bbcc56\", \"50\": \"d49e7751-8b6c-11e9-adb8-000ec6bbcc56\", \"51\": \"d49e7752-8b6c-11e9-a828-000ec6bbcc56\", \"52\": \"d49e7753-8b6c-11e9-97e1-000ec6bbcc56\", \"53\": \"d49e7754-8b6c-11e9-83f7-000ec6bbcc56\", \"54\": \"d49e7755-8b6c-11e9-a902-000ec6bbcc56\", \"55\": \"d49e7756-8b6c-11e9-bce9-000ec6bbcc56\", \"56\": \"d49e7757-8b6c-11e9-a206-000ec6bbcc56\", \"57\": \"d49e7758-8b6c-11e9-8359-000ec6bbcc56\", \"58\": \"d49e7759-8b6c-11e9-95f6-000ec6bbcc56\", \"59\": \"d49e775a-8b6c-11e9-a425-000ec6bbcc56\", \"60\": \"d49e775b-8b6c-11e9-9ce6-000ec6bbcc56\", \"61\": \"d49e775c-8b6c-11e9-aaab-000ec6bbcc56\", \"62\": \"d49e775d-8b6c-11e9-9c08-000ec6bbcc56\", \"63\": \"d49e775e-8b6c-11e9-8e39-000ec6bbcc56\", \"64\": \"d49e775f-8b6c-11e9-8d00-000ec6bbcc56\", \"65\": \"d49e7760-8b6c-11e9-a567-000ec6bbcc56\", \"66\": \"d49e7761-8b6c-11e9-8f70-000ec6bbcc56\", \"67\": \"d49e7762-8b6c-11e9-b1e5-000ec6bbcc56\", \"68\": \"d49e7763-8b6c-11e9-b91a-000ec6bbcc56\", \"69\": \"d49e7764-8b6c-11e9-8000-000ec6bbcc56\", \"70\": \"d49e7765-8b6c-11e9-a66c-000ec6bbcc56\", \"71\": \"d49e7766-8b6c-11e9-8563-000ec6bbcc56\", \"72\": \"d49e7767-8b6c-11e9-b2de-000ec6bbcc56\", \"73\": \"d49e7768-8b6c-11e9-9c79-000ec6bbcc56\", \"74\": \"d49e7769-8b6c-11e9-b3bd-000ec6bbcc56\", \"75\": \"d49e776a-8b6c-11e9-8b7c-000ec6bbcc56\", \"76\": \"d49e776b-8b6c-11e9-8942-000ec6bbcc56\", \"77\": \"d49e776c-8b6c-11e9-860b-000ec6bbcc56\", \"78\": \"d49e776d-8b6c-11e9-b5fd-000ec6bbcc56\", \"79\": \"d49e776e-8b6c-11e9-a4d9-000ec6bbcc56\", \"80\": \"d49e776f-8b6c-11e9-a11f-000ec6bbcc56\", \"81\": \"d49e7770-8b6c-11e9-b7d0-000ec6bbcc56\", \"82\": \"d49e7771-8b6c-11e9-82d2-000ec6bbcc56\", \"83\": \"d49e7772-8b6c-11e9-9d28-000ec6bbcc56\", \"84\": \"d49e7773-8b6c-11e9-a777-000ec6bbcc56\", \"85\": \"d49e7774-8b6c-11e9-ab26-000ec6bbcc56\", \"86\": \"d49e7775-8b6c-11e9-91a1-000ec6bbcc56\", \"87\": \"d49e7776-8b6c-11e9-a115-000ec6bbcc56\", \"88\": \"d49e7777-8b6c-11e9-9859-000ec6bbcc56\", \"89\": \"d49", :exception=>#<LogStash::Json::ParserError: Unexpected end-of-input in VALUE_STRING
 at [Source: (byte[])"{"0": "d49d666e-8b6c-11e9-92bb-000ec6bbcc56", "1": "d49e503a-8b6c-11e9-a67b-000ec6bbcc56", "2": "d49e503b-8b6c-11e9-927c-000ec6bbcc56", "3": "d49e503c-8b6c-11e9-9a53-000ec6bbcc56", "4": "d49e503d-8b6c-11e9-b1da-000ec6bbcc56", "5": "d49e503e-8b6c-11e9-99a1-000ec6bbcc56", "6": "d49e503f-8b6c-11e9-9364-000ec6bbcc56", "7": "d49e5040-8b6c-11e9-8d42-000ec6bbcc56", "8": "d49e5041-8b6c-11e9-8fdd-000ec6bbcc56", "9": "d49e5042-8b6c-11e9-b1a2-000ec6bbcc56", "10": "d49e5043-8b6c-11e9-a4f6-000ec6bbcc56", "11"[truncated 3595 bytes]; line: 1, column: 4096]>}
{
          "tags" => [
        [0] "_jsonparsefailure"
    ],
       "message" => "{\"0\": \"d49d666e-8b6c-11e9-92bb-000ec6bbcc56\", \"1\": \"d49e503a-8b6c-11e9-a67b-000ec6bbcc56\", \"2\": \"d49e503b-8b6c-11e9-927c-000ec6bbcc56\", \"3\": \"d49e503c-8b6c-11e9-9a53-000ec6bbcc56\", \"4\": \"d49e503d-8b6c-11e9-b1da-000ec6bbcc56\", \"5\": \"d49e503e-8b6c-11e9-99a1-000ec6bbcc56\", \"6\": \"d49e503f-8b6c-11e9-9364-000ec6bbcc56\", \"7\": \"d49e5040-8b6c-11e9-8d42-000ec6bbcc56\", \"8\": \"d49e5041-8b6c-11e9-8fdd-000ec6bbcc56\", \"9\": \"d49e5042-8b6c-11e9-b1a2-000ec6bbcc56\", \"10\": \"d49e5043-8b6c-11e9-a4f6-000ec6bbcc56\", \"11\": \"d49e5044-8b6c-11e9-9d49-000ec6bbcc56\", \"12\": \"d49e5045-8b6c-11e9-8b05-000ec6bbcc56\", \"13\": \"d49e5046-8b6c-11e9-883c-000ec6bbcc56\", \"14\": \"d49e5047-8b6c-11e9-8535-000ec6bbcc56\", \"15\": \"d49e5048-8b6c-11e9-9a24-000ec6bbcc56\", \"16\": \"d49e5049-8b6c-11e9-bd86-000ec6bbcc56\", \"17\": \"d49e7730-8b6c-11e9-a17c-000ec6bbcc56\", \"18\": \"d49e7731-8b6c-11e9-82ee-000ec6bbcc56\", \"19\": \"d49e7732-8b6c-11e9-8eaa-000ec6bbcc56\", \"20\": \"d49e7733-8b6c-11e9-a5a4-000ec6bbcc56\", \"21\": \"d49e7734-8b6c-11e9-93f6-000ec6bbcc56\", \"22\": \"d49e7735-8b6c-11e9-a842-000ec6bbcc56\", \"23\": \"d49e7736-8b6c-11e9-be30-000ec6bbcc56\", \"24\": \"d49e7737-8b6c-11e9-9fac-000ec6bbcc56\", \"25\": \"d49e7738-8b6c-11e9-ad55-000ec6bbcc56\", \"26\": \"d49e7739-8b6c-11e9-9062-000ec6bbcc56\", \"27\": \"d49e773a-8b6c-11e9-95e8-000ec6bbcc56\", \"28\": \"d49e773b-8b6c-11e9-90b6-000ec6bbcc56\", \"29\": \"d49e773c-8b6c-11e9-839d-000ec6bbcc56\", \"30\": \"d49e773d-8b6c-11e9-b444-000ec6bbcc56\", \"31\": \"d49e773e-8b6c-11e9-bb52-000ec6bbcc56\", \"32\": \"d49e773f-8b6c-11e9-acf9-000ec6bbcc56\", \"33\": \"d49e7740-8b6c-11e9-b5c2-000ec6bbcc56\", \"34\": \"d49e7741-8b6c-11e9-8a29-000ec6bbcc56\", \"35\": \"d49e7742-8b6c-11e9-90b4-000ec6bbcc56\", \"36\": \"d49e7743-8b6c-11e9-9625-000ec6bbcc56\", \"37\": \"d49e7744-8b6c-11e9-b195-000ec6bbcc56\", \"38\": \"d49e7745-8b6c-11e9-978a-000ec6bbcc56\", \"39\": \"d49e7746-8b6c-11e9-a8e7-000ec6bbcc56\", \"40\": \"d49e7747-8b6c-11e9-8c81-000ec6bbcc56\", \"41\": \"d49e7748-8b6c-11e9-bd0b-000ec6bbcc56\", \"42\": \"d49e7749-8b6c-11e9-8335-000ec6bbcc56\", \"43\": \"d49e774a-8b6c-11e9-bfe1-000ec6bbcc56\", \"44\": \"d49e774b-8b6c-11e9-afe2-000ec6bbcc56\", \"45\": \"d49e774c-8b6c-11e9-a014-000ec6bbcc56\", \"46\": \"d49e774d-8b6c-11e9-8c07-000ec6bbcc56\", \"47\": \"d49e774e-8b6c-11e9-8e5e-000ec6bbcc56\", \"48\": \"d49e774f-8b6c-11e9-b0c7-000ec6bbcc56\", \"49\": \"d49e7750-8b6c-11e9-8e66-000ec6bbcc56\", \"50\": \"d49e7751-8b6c-11e9-adb8-000ec6bbcc56\", \"51\": \"d49e7752-8b6c-11e9-a828-000ec6bbcc56\", \"52\": \"d49e7753-8b6c-11e9-97e1-000ec6bbcc56\", \"53\": \"d49e7754-8b6c-11e9-83f7-000ec6bbcc56\", \"54\": \"d49e7755-8b6c-11e9-a902-000ec6bbcc56\", \"55\": \"d49e7756-8b6c-11e9-bce9-000ec6bbcc56\", \"56\": \"d49e7757-8b6c-11e9-a206-000ec6bbcc56\", \"57\": \"d49e7758-8b6c-11e9-8359-000ec6bbcc56\", \"58\": \"d49e7759-8b6c-11e9-95f6-000ec6bbcc56\", \"59\": \"d49e775a-8b6c-11e9-a425-000ec6bbcc56\", \"60\": \"d49e775b-8b6c-11e9-9ce6-000ec6bbcc56\", \"61\": \"d49e775c-8b6c-11e9-aaab-000ec6bbcc56\", \"62\": \"d49e775d-8b6c-11e9-9c08-000ec6bbcc56\", \"63\": \"d49e775e-8b6c-11e9-8e39-000ec6bbcc56\", \"64\": \"d49e775f-8b6c-11e9-8d00-000ec6bbcc56\", \"65\": \"d49e7760-8b6c-11e9-a567-000ec6bbcc56\", \"66\": \"d49e7761-8b6c-11e9-8f70-000ec6bbcc56\", \"67\": \"d49e7762-8b6c-11e9-b1e5-000ec6bbcc56\", \"68\": \"d49e7763-8b6c-11e9-b91a-000ec6bbcc56\", \"69\": \"d49e7764-8b6c-11e9-8000-000ec6bbcc56\", \"70\": \"d49e7765-8b6c-11e9-a66c-000ec6bbcc56\", \"71\": \"d49e7766-8b6c-11e9-8563-000ec6bbcc56\", \"72\": \"d49e7767-8b6c-11e9-b2de-000ec6bbcc56\", \"73\": \"d49e7768-8b6c-11e9-9c79-000ec6bbcc56\", \"74\": \"d49e7769-8b6c-11e9-b3bd-000ec6bbcc56\", \"75\": \"d49e776a-8b6c-11e9-8b7c-000ec6bbcc56\", \"76\": \"d49e776b-8b6c-11e9-8942-000ec6bbcc56\", \"77\": \"d49e776c-8b6c-11e9-860b-000ec6bbcc56\", \"78\": \"d49e776d-8b6c-11e9-b5fd-000ec6bbcc56\", \"79\": \"d49e776e-8b6c-11e9-a4d9-000ec6bbcc56\", \"80\": \"d49e776f-8b6c-11e9-a11f-000ec6bbcc56\", \"81\": \"d49e7770-8b6c-11e9-b7d0-000ec6bbcc56\", \"82\": \"d49e7771-8b6c-11e9-82d2-000ec6bbcc56\", \"83\": \"d49e7772-8b6c-11e9-9d28-000ec6bbcc56\", \"84\": \"d49e7773-8b6c-11e9-a777-000ec6bbcc56\", \"85\": \"d49e7774-8b6c-11e9-ab26-000ec6bbcc56\", \"86\": \"d49e7775-8b6c-11e9-91a1-000ec6bbcc56\", \"87\": \"d49e7776-8b6c-11e9-a115-000ec6bbcc56\", \"88\": \"d49e7777-8b6c-11e9-9859-000ec6bbcc56\", \"89\": \"d49",
    "@timestamp" => 2023-04-25T19:38:37.064782265Z,
         "event" => {
        "original" => "{\"0\": \"d49d666e-8b6c-11e9-92bb-000ec6bbcc56\", \"1\": \"d49e503a-8b6c-11e9-a67b-000ec6bbcc56\", \"2\": \"d49e503b-8b6c-11e9-927c-000ec6bbcc56\", \"3\": \"d49e503c-8b6c-11e9-9a53-000ec6bbcc56\", \"4\": \"d49e503d-8b6c-11e9-b1da-000ec6bbcc56\", \"5\": \"d49e503e-8b6c-11e9-99a1-000ec6bbcc56\", \"6\": \"d49e503f-8b6c-11e9-9364-000ec6bbcc56\", \"7\": \"d49e5040-8b6c-11e9-8d42-000ec6bbcc56\", \"8\": \"d49e5041-8b6c-11e9-8fdd-000ec6bbcc56\", \"9\": \"d49e5042-8b6c-11e9-b1a2-000ec6bbcc56\", \"10\": \"d49e5043-8b6c-11e9-a4f6-000ec6bbcc56\", \"11\": \"d49e5044-8b6c-11e9-9d49-000ec6bbcc56\", \"12\": \"d49e5045-8b6c-11e9-8b05-000ec6bbcc56\", \"13\": \"d49e5046-8b6c-11e9-883c-000ec6bbcc56\", \"14\": \"d49e5047-8b6c-11e9-8535-000ec6bbcc56\", \"15\": \"d49e5048-8b6c-11e9-9a24-000ec6bbcc56\", \"16\": \"d49e5049-8b6c-11e9-bd86-000ec6bbcc56\", \"17\": \"d49e7730-8b6c-11e9-a17c-000ec6bbcc56\", \"18\": \"d49e7731-8b6c-11e9-82ee-000ec6bbcc56\", \"19\": \"d49e7732-8b6c-11e9-8eaa-000ec6bbcc56\", \"20\": \"d49e7733-8b6c-11e9-a5a4-000ec6bbcc56\", \"21\": \"d49e7734-8b6c-11e9-93f6-000ec6bbcc56\", \"22\": \"d49e7735-8b6c-11e9-a842-000ec6bbcc56\", \"23\": \"d49e7736-8b6c-11e9-be30-000ec6bbcc56\", \"24\": \"d49e7737-8b6c-11e9-9fac-000ec6bbcc56\", \"25\": \"d49e7738-8b6c-11e9-ad55-000ec6bbcc56\", \"26\": \"d49e7739-8b6c-11e9-9062-000ec6bbcc56\", \"27\": \"d49e773a-8b6c-11e9-95e8-000ec6bbcc56\", \"28\": \"d49e773b-8b6c-11e9-90b6-000ec6bbcc56\", \"29\": \"d49e773c-8b6c-11e9-839d-000ec6bbcc56\", \"30\": \"d49e773d-8b6c-11e9-b444-000ec6bbcc56\", \"31\": \"d49e773e-8b6c-11e9-bb52-000ec6bbcc56\", \"32\": \"d49e773f-8b6c-11e9-acf9-000ec6bbcc56\", \"33\": \"d49e7740-8b6c-11e9-b5c2-000ec6bbcc56\", \"34\": \"d49e7741-8b6c-11e9-8a29-000ec6bbcc56\", \"35\": \"d49e7742-8b6c-11e9-90b4-000ec6bbcc56\", \"36\": \"d49e7743-8b6c-11e9-9625-000ec6bbcc56\", \"37\": \"d49e7744-8b6c-11e9-b195-000ec6bbcc56\", \"38\": \"d49e7745-8b6c-11e9-978a-000ec6bbcc56\", \"39\": \"d49e7746-8b6c-11e9-a8e7-000ec6bbcc56\", \"40\": \"d49e7747-8b6c-11e9-8c81-000ec6bbcc56\", \"41\": \"d49e7748-8b6c-11e9-bd0b-000ec6bbcc56\", \"42\": \"d49e7749-8b6c-11e9-8335-000ec6bbcc56\", \"43\": \"d49e774a-8b6c-11e9-bfe1-000ec6bbcc56\", \"44\": \"d49e774b-8b6c-11e9-afe2-000ec6bbcc56\", \"45\": \"d49e774c-8b6c-11e9-a014-000ec6bbcc56\", \"46\": \"d49e774d-8b6c-11e9-8c07-000ec6bbcc56\", \"47\": \"d49e774e-8b6c-11e9-8e5e-000ec6bbcc56\", \"48\": \"d49e774f-8b6c-11e9-b0c7-000ec6bbcc56\", \"49\": \"d49e7750-8b6c-11e9-8e66-000ec6bbcc56\", \"50\": \"d49e7751-8b6c-11e9-adb8-000ec6bbcc56\", \"51\": \"d49e7752-8b6c-11e9-a828-000ec6bbcc56\", \"52\": \"d49e7753-8b6c-11e9-97e1-000ec6bbcc56\", \"53\": \"d49e7754-8b6c-11e9-83f7-000ec6bbcc56\", \"54\": \"d49e7755-8b6c-11e9-a902-000ec6bbcc56\", \"55\": \"d49e7756-8b6c-11e9-bce9-000ec6bbcc56\", \"56\": \"d49e7757-8b6c-11e9-a206-000ec6bbcc56\", \"57\": \"d49e7758-8b6c-11e9-8359-000ec6bbcc56\", \"58\": \"d49e7759-8b6c-11e9-95f6-000ec6bbcc56\", \"59\": \"d49e775a-8b6c-11e9-a425-000ec6bbcc56\", \"60\": \"d49e775b-8b6c-11e9-9ce6-000ec6bbcc56\", \"61\": \"d49e775c-8b6c-11e9-aaab-000ec6bbcc56\", \"62\": \"d49e775d-8b6c-11e9-9c08-000ec6bbcc56\", \"63\": \"d49e775e-8b6c-11e9-8e39-000ec6bbcc56\", \"64\": \"d49e775f-8b6c-11e9-8d00-000ec6bbcc56\", \"65\": \"d49e7760-8b6c-11e9-a567-000ec6bbcc56\", \"66\": \"d49e7761-8b6c-11e9-8f70-000ec6bbcc56\", \"67\": \"d49e7762-8b6c-11e9-b1e5-000ec6bbcc56\", \"68\": \"d49e7763-8b6c-11e9-b91a-000ec6bbcc56\", \"69\": \"d49e7764-8b6c-11e9-8000-000ec6bbcc56\", \"70\": \"d49e7765-8b6c-11e9-a66c-000ec6bbcc56\", \"71\": \"d49e7766-8b6c-11e9-8563-000ec6bbcc56\", \"72\": \"d49e7767-8b6c-11e9-b2de-000ec6bbcc56\", \"73\": \"d49e7768-8b6c-11e9-9c79-000ec6bbcc56\", \"74\": \"d49e7769-8b6c-11e9-b3bd-000ec6bbcc56\", \"75\": \"d49e776a-8b6c-11e9-8b7c-000ec6bbcc56\", \"76\": \"d49e776b-8b6c-11e9-8942-000ec6bbcc56\", \"77\": \"d49e776c-8b6c-11e9-860b-000ec6bbcc56\", \"78\": \"d49e776d-8b6c-11e9-b5fd-000ec6bbcc56\", \"79\": \"d49e776e-8b6c-11e9-a4d9-000ec6bbcc56\", \"80\": \"d49e776f-8b6c-11e9-a11f-000ec6bbcc56\", \"81\": \"d49e7770-8b6c-11e9-b7d0-000ec6bbcc56\", \"82\": \"d49e7771-8b6c-11e9-82d2-000ec6bbcc56\", \"83\": \"d49e7772-8b6c-11e9-9d28-000ec6bbcc56\", \"84\": \"d49e7773-8b6c-11e9-a777-000ec6bbcc56\", \"85\": \"d49e7774-8b6c-11e9-ab26-000ec6bbcc56\", \"86\": \"d49e7775-8b6c-11e9-91a1-000ec6bbcc56\", \"87\": \"d49e7776-8b6c-11e9-a115-000ec6bbcc56\", \"88\": \"d49e7777-8b6c-11e9-9859-000ec6bbcc56\", \"89\": \"d49"
    },
      "@version" => "1",
          "host" => "ip-10-197-102-49"
}

I am getting an exception. Am i missing anything here?

11

Oh I see, you are using the stdin input and pasting the content of the log in the terminal, I just did that in the same way you did and got the same error.

The way I replicated before was pasting the log in a file and piping it to logstash or using an echo

cat samplelog | logstash with the pipeline config

or

echo 'your json log' | logstash with the pipeline config

Both of those worked for me, not sure why pasting it doesn't work, probably is some limitation of the terminal that adds something, but reading from a file or using an echo works.

What is your original input plugin? The s3 input?

12

I am using the s3 input plugin.

I tried with file or echo, and it worked. let me see if I am missing something in s3 side.

Really thanks for all the responses. I will update the thread.

13

With s3 input it is working, you are right, the issue was with the terminal only. Thanks for all your help.

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.