Json: cannot unmarshal string into Go value of type map[string]interface {}

sur1029 · November 20, 2019, 9:26am

Hi,
We are pushing json logs from application and have configured filebeat to push same logs to es.
I am getting below error in logs:
2019/11/20 06:12:59.971200 json .go:32: ERR Error decoding JSON: json : cannot unmarshal string into Go value of type map[string]interface {}
2019/11/20 06:14:02.714919 json .go:32: ERR Error decoding JSON: invalid character 'p' after object key
2019/11/20 06:14:02.714982 json .go:32: ERR Error decoding JSON: invalid character 'p' looking for beginning of value
2019/11/20 06:22:16.153381 json .go:32: ERR Error decoding JSON: invalid character 'p' after array element
2019/11/20 06:22:16.153478 json .go:32: ERR Error decoding JSON: invalid character 'D' looking for beginning of value
2019/11/20 06:28:04.885510 json .go:32: ERR Error decoding JSON: invalid character 'a' after object key
2019/11/20 06:28:04.885557 json .go:32: ERR Error decoding JSON: invalid character 'a' looking for beginning of value
Also some logs are being overriden by another making it an invalid json, hence filebeat doesn't push these logs to elastic search.
sample logs : {"primary_id": "3f72d0fd-7f05-4f2a-9e5c-a5a090407cbf__0", "ch": "android", "ad": ["APA91bGRgCheCfghPyC_4BZuI-cXx3W3-xOd5HeX-93ZM1VfzYxslQPvoxR1FPGJ4zZF6wl7c01NL7l4BKKL2tCX5Gq5LlZ6QD7Rq3V2IM5OQekLjmDSrcFlU6egcvADhZxpqUBLfMBk", "APA91bHbKPa6Ae0ff0-YttkeFUi33tRjK0MyUlWdWeZQR5C8oWE4Ie1N6QIvHlmfmxv5hjYRXhYrCD49E-sc1blpwjH5s2-PlMaMsD0nXNM1OMFEMRCS6l6ROO7RZnYMAMzpORQ0Ah1d", "APA91bE8vcxuzQ8_ItHaw5qaZYyiDo62PoImh0kygmeuNWjgYwb6KQ3Zrin54ZK7sloOB3QgIyd1Y5Cbi-tWdBaphenAyM4TwQ89VD14CSYHDZlAFV1z3IdCgGw4tBMGTJC-xtNX-bgw", "APA91bHJXjBkEzV9A5hfv6h-yv-hUm9ixOEhr-lZp7mgWothmJAIzeiNXyX7lF0Xsk8H46MF77oWroE9GErFwtg2927DSaCHvdgVcFkLqy3sc1drbnnraVnx7QA44q5qCCci7mZ6KNzk", "APA91bE7UESyiIAF_58lgpd9mEETxu9q7XPlXGFpWopbwojlGCzQBbvWLAvH2VV_dIA36dIj9aoT_SfvLvpulmXpmivGRD_Eeq6kCJIe69NShfBt-0YQ9jyxUOzv18B3Zzcy8_nfqauu", "APA91bETfPnCygDyxMKDAFJhvytRDb9xltvT1ASnycRP3guAAYy6f3_-6SJ0snGkJrwvBLXesAmkYl6gL4tk26ue5urZppGErpLXUw6pqaus-ikhoAvs_uq4GuztoqXcljl1rcMy52M3", "APA91bFAoETYeU_0hoS75UV5KvEorTXMP1UnrREDsibXqxsRnp8Ahixx30A1FznKMReFpNgKPGyblHYh8UwtkLLB6ocNVbUGfv4GaP55gJqCieDc8DYBPJLS_MUfdP0QfSiHI_lkXCWu", "APA91bHHwYsRm_CjrFohHltvHvoZABgiGAWiqIUPQqE8QLNizaOXFP4CPV5KyvTINJfd6VORDqD4xQIJgU9c8BWiyNgjkrwUiDEkpc1w1h5vxn3OYPSLdh67CIee7OzPF1a9ekNVZAPy"], "md": "2019-11-13 14:14:24", "cb": "", "app": "mmt", "message_identifiers": {}, "nid": "3f72d0fd-7f05-4f2a-9e5c-a5a090407cbf", "cd": "2019-11-13 14:14:24", "tn": "mmt_template", "tag": [{"value": "aakash.rockstar2009@gmail.com", "key": "link"}, {"value": "917895406243", "key": "link"}, {"value": "APA91bGRgCheCfghPyC_4BZuI-cXx3W3-xOd5HeX-93ZM1VfzYxslQPvoxR1FPGJ4zZF6wl7c01NL7l4BKKL2tCX5Gq5LlZ6QD7Rq3V2IM5OQekLjmDSrcFlU6egcvADhZxpqUBLfMBk", "key": "pntoken"}, {"value": "APA91bHbKPa6Ae0ff0-YttkeFUi33tRjK0MyUlWdWeZQR5C8oWE4Ie1N6QIvHlmfmxv5hjYRXhYrCD49E-sc1blpwjH5s2-PlMaMsD0nXNM1OMFEMRCS6l6ROO7RZnYMAMzpORQ0Ah1d", "key": "pntoken"}, {"value": "APA91bE8vcxuzQ8_ItHaw5qaZYyiDo62PoImh0kygmeuNWjgYwb6KQ3Zrin54ZK7sloOB3QgIyd1Y5Cbi-tWdBaphenAyM4TwQ89VD14CSYHDZlAFV1z3IdCgGw4tBMGTJC-xtNX-bgw", "key": "pntoken"}, {"value": "APA91bHJXjBkEzV9A5hfv6h-yv-hUm9ixOEhr-lZp7mgWothmJAIzeiNXyX7lF0Xsk8H46MF77oWroE9GErFwtg2927DSaCHvdgVcFkLqy3sc1drbnnraVnx7QA44q5qCCci7mZ6KNzk", "key": "pntoken"}, {"value": "APA91bE7UESyiIAF_58lgpd9mEETxu9q7XPlXGFpWopbwojlGCzQBbvWLAvH2VV_dIA36dIj9aoT_SfvLvpulmXpmivGRD_Eeq6kCJIe69NShfBt-0YQ9jyxUOzv18B3Zzcy8_nfqauu", "key": "pntoken"}, {"value": "APA91bETfPnCygDyxMKDAFJhvytRDb9xltvT1ASnycRP3guAAYy6f3_-6SJ0snGkJrwvBLXesAmkYl6gL4tk26ue5urZppGErpLXUw6pqaus-ikhoAvs_uq4GuztoqXcljl1rcMy52M3", "key": "pntoken"}, {"value": "APA91bFAoETYeU_0hoS75UV5KvEorTXMP1UnrREDsibXqxsRnp8Ahixx30A1FznKMReFpNgKPGyblHYh8UwtkLLB6ocNVbUGfv4GaP55gJqCieDc8DYBPJLS_MUfdP0QfSiHI_lkXCWu", "key": "pntoken"}, {"value": "APA91bHHwYsRm_CjrFohHltvHvoZABgiGAWiqIUPQqE8QLNizaOXFP4CPV5KyvTINJfd6VORDqD4xQIJgU9c8BWiyNgjkrwUiDEkpc1w1h5vxn3OYPSLdh67CIee7OzPF1a9ekNVZAPy", "key": "pntoken"}], "fd": null, "time": "2019-11-13T08:44:24.605837", "dt": {"primary_id": "3f72d0fd-7f05-4f2a-9e5c-a5a090407cbf__0", "token": ["APA91bGRgCheCfghPyC_4BZuI-cXx3W3-xOd5HeX-93ZM1VfzYxslQPvoxR1FPGJ4zZF6wl7c01NL7l4BKKL2tCX5Gq5LlZ6QD7Rq3V2IM5OQekLjmDSrcFlU6egcvADhZxpqUBLfMBk", "APA91bHbKPa6Ae0ff0-YttkeFUi33tRjK0MyUlWdWeZQR5C8oWE4Ie1N6QIvHlmfmxv5hjYRXhYrCD49E-sc1blpwjH5s2-PlMaMsD0nXNM1OMFEMRCS6l6ROO7RZnYMAMzpORQ0Ah1d", "APA91bE8vcxuzQ8_ItHaw5qaZYyiDo62PoImh0kygmeuNWjgYwb6KQ3Zrin54ZK7sloOB3QgIyd1Y5Cbi-tWdBaphenAyM4TwQ89VD14CSYHDZlAFV1z3IdCgGw4tBMGTJC-xtNX-bgw", "APA91bHJXjBkEzV9A5hfv6h-yv-hUm9ixOEhr-lZp7mgWothmJAIzeiNXyX7lF0Xsk8H46MF77oWroE9GErFwtg2927DSaCHvdgVcFkLqy3sc1drbnnraVnx7QA44q5qCCci7mZ6KNzk", "APA91bE7UESyiIAF_58lgpd9mEETxu9q7XPlXGFpWopbwojlGCzQBbvWLAvH2VV_dIA36dIj9aoT_SfvLvpulmXpmivGRD_Eeq6kCJIe69NShfBt-0YQ9jyxUOzv18B3Zzcy8_nfqauu", "APA91bETfPnCygDyxMKDAFJhvytRDb9xltvT1ASnycRP3guAAYy6f3_-6SJ0snGkJrwvBLXesAmkYl6gL4tk26ue5urZppGErpLXUw6pqaus-ikhoAvs_uq4GuztoqXcljl1rcMy52M3", "APA91bFAoETYeU_0hoS75UV5KvEorTXMP1UnrREDsibXqxsRnp8Ahixx30A1FznKMReFpNgKPGyblHYh8UwtkLLB6ocNVbUGfv4GaP55gJqCieDc8DYBPJLS_MUfdP0QfSiHI_lkXCWu", "APA91bHHwYsRm_CjrFohHltvHvoZABgiGAWiqIUPQqE8QLNizaOXFP4C {"md": "2019-11-13 14:14:24", "ch": "sms", "nm": "DownloadSms", "sent_ack_received_at": "2019-11-13 14:14:24", "app": "pigeon", "nid": "b206b4cb-aab6-4858-8dba-13b4bfebc5ee", "st": 9, "primary_id": "b206b4cb-aab6-4858-8dba-13b4bfebc5ee__0", "time": "2019-11-13T08:44:24.605968", "message": "SENT_ACK_RECIEVED_STATUS"}
PV5KyvTINJfd6VORDqD4xQIJgU9c8BWiyNgjkrwUiDEkpc1w1h5vxn3OYPSLdh67CIee7OzPF1a9ekNVZAPy"], "message": {"mmt_message": {"priority": 1, "silentPush": {"silent": "false"}, "message": {"sound": "default", "subtext": "Book Now !!", "title": "Exclusive collection of Meals & Deals just for you", "webTitle": "Exclusive collection of Meals & Deals just for you", "text": "Choose from an exclusive collection of Buffet, Spa, Theme Parks, Cabs, Gift Cards and many more. Book memorable Meals & Deals on the MakeMyTrip app today!", "appu": "0", "validity": "YYYY-MM-DD-HH-MM", "ver": "3.8.2", "ntc": "16", "badge": "0"}, "options": {"category": "1", "lob": "Makemytrip Icon", "campaignType": "Fare Alert", "campaign": "acme", "density": "resolution", "webpageURL": "", "slideViewImages": "", "imageURL": "https://images-acme.mmtcdn.com/prod-acme-image/system/product_media/c/33984/media6CNB5P2SQXTL3.jpg", "action1": "", "appVersionCode": "5.6.4", "visibility": "0", "alert": "-1", "priority": "0", "appUpdateType": "0", "deepLinkURL": "mmyt://acme/?page=acmeLanding&categoryName=fd", "expandable": "0", "webpageURLBrowser": "", "action2": ""}, "launchAction": {"lob": "0", "screen": "0", "data": {}}}}, "channel": "android"}, "st": 1, "message": "QUEUED_STATUS", "nm": "acme"}

I have marked the merged log in bold. Is the error related to merging of these logs? please help
Thanks

kvch · November 20, 2019, 2:43pm

Could you please share debug logs of Filebeat while it is processing the messages which return an error? Also, please share your configuration formatted using </>.

sur1029 · December 9, 2019, 12:36pm

Hi,
Above error is very rare and only happens 5-6 times in a day. I have already enabled debug logs but couldn’t replicate same error.
I have already attached a sample entry for which it happened. Formatting the logs again.
Hi,
We are pushing json logs from application and have configured filebeat to push same logs to es.
I am getting below error in logs:

2019/11/20 06:12:59.971200  **json**  .go:32: ERR Error decoding JSON:  **json**  : cannot unmarshal string into Go value of type map[string]interface {}
2019/11/20 06:14:02.714919  **json**  .go:32: ERR Error decoding JSON: invalid character 'p' after object key
2019/11/20 06:14:02.714982  **json**  .go:32: ERR Error decoding JSON: invalid character 'p' looking for beginning of value
2019/11/20 06:22:16.153381  **json**  .go:32: ERR Error decoding JSON: invalid character 'p' after array element
2019/11/20 06:22:16.153478  **json**  .go:32: ERR Error decoding JSON: invalid character 'D' looking for beginning of value
2019/11/20 06:28:04.885510  **json**  .go:32: ERR Error decoding JSON: invalid character 'a' after object key
2019/11/20 06:28:04.885557  **json**  .go:32: ERR Error decoding JSON: invalid character 'a' looking for beginning of value
Also some logs are being overriden by another making it an invalid json, hence filebeat doesn't push these logs to elastic search.
sample logs : {"primary_id": "3f72d0fd-7f05-4f2a-9e5c-a5a090407cbf__0", "ch": "android", "ad": ["APA91bGRgCheCfghPyC_4BZuI-cXx3W3-xOd5HeX-93ZM1VfzYxslQPvoxR1FPGJ4zZF6wl7c01NL7l4BKKL2tCX5Gq5LlZ6QD7Rq3V2IM5OQekLjmDSrcFlU6egcvADhZxpqUBLfMBk", "APA91bHbKPa6Ae0ff0-YttkeFUi33tRjK0MyUlWdWeZQR5C8oWE4Ie1N6QIvHlmfmxv5hjYRXhYrCD49E-sc1blpwjH5s2-PlMaMsD0nXNM1OMFEMRCS6l6ROO7RZnYMAMzpORQ0Ah1d", "APA91bE8vcxuzQ8_ItHaw5qaZYyiDo62PoImh0kygmeuNWjgYwb6KQ3Zrin54ZK7sloOB3QgIyd1Y5Cbi-tWdBaphenAyM4TwQ89VD14CSYHDZlAFV1z3IdCgGw4tBMGTJC-xtNX-bgw", "APA91bHJXjBkEzV9A5hfv6h-yv-hUm9ixOEhr-lZp7mgWothmJAIzeiNXyX7lF0Xsk8H46MF77oWroE9GErFwtg2927DSaCHvdgVcFkLqy3sc1drbnnraVnx7QA44q5qCCci7mZ6KNzk", "APA91bE7UESyiIAF_58lgpd9mEETxu9q7XPlXGFpWopbwojlGCzQBbvWLAvH2VV_dIA36dIj9aoT_SfvLvpulmXpmivGRD_Eeq6kCJIe69NShfBt-0YQ9jyxUOzv18B3Zzcy8_nfqauu", "APA91bETfPnCygDyxMKDAFJhvytRDb9xltvT1ASnycRP3guAAYy6f3_-6SJ0snGkJrwvBLXesAmkYl6gL4tk26ue5urZppGErpLXUw6pqaus-ikhoAvs_uq4GuztoqXcljl1rcMy52M3", "APA91bFAoETYeU_0hoS75UV5KvEorTXMP1UnrREDsibXqxsRnp8Ahixx30A1FznKMReFpNgKPGyblHYh8UwtkLLB6ocNVbUGfv4GaP55gJqCieDc8DYBPJLS_MUfdP0QfSiHI_lkXCWu", "APA91bHHwYsRm_CjrFohHltvHvoZABgiGAWiqIUPQqE8QLNizaOXFP4CPV5KyvTINJfd6VORDqD4xQIJgU9c8BWiyNgjkrwUiDEkpc1w1h5vxn3OYPSLdh67CIee7OzPF1a9ekNVZAPy"], "md": "2019-11-13 14:14:24", "cb": "", "app": "mmt", "message_identifiers": {}, "nid": "3f72d0fd-7f05-4f2a-9e5c-a5a090407cbf", "cd": "2019-11-13 14:14:24", "tn": "mmt_template", "tag": [{"value": "[aakash.rockstar2009@gmail.com](mailto:aakash.rockstar2009@gmail.com)", "key": "link"}, {"value": "917895406243", "key": "link"}, {"value": "APA91bGRgCheCfghPyC_4BZuI-cXx3W3-xOd5HeX-93ZM1VfzYxslQPvoxR1FPGJ4zZF6wl7c01NL7l4BKKL2tCX5Gq5LlZ6QD7Rq3V2IM5OQekLjmDSrcFlU6egcvADhZxpqUBLfMBk", "key": "pntoken"}, {"value": "APA91bHbKPa6Ae0ff0-YttkeFUi33tRjK0MyUlWdWeZQR5C8oWE4Ie1N6QIvHlmfmxv5hjYRXhYrCD49E-sc1blpwjH5s2-PlMaMsD0nXNM1OMFEMRCS6l6ROO7RZnYMAMzpORQ0Ah1d", "key": "pntoken"}, {"value": "APA91bE8vcxuzQ8_ItHaw5qaZYyiDo62PoImh0kygmeuNWjgYwb6KQ3Zrin54ZK7sloOB3QgIyd1Y5Cbi-tWdBaphenAyM4TwQ89VD14CSYHDZlAFV1z3IdCgGw4tBMGTJC-xtNX-bgw", "key": "pntoken"}, {"value": "APA91bHJXjBkEzV9A5hfv6h-yv-hUm9ixOEhr-lZp7mgWothmJAIzeiNXyX7lF0Xsk8H46MF77oWroE9GErFwtg2927DSaCHvdgVcFkLqy3sc1drbnnraVnx7QA44q5qCCci7mZ6KNzk", "key": "pntoken"}, {"value": "APA91bE7UESyiIAF_58lgpd9mEETxu9q7XPlXGFpWopbwojlGCzQBbvWLAvH2VV_dIA36dIj9aoT_SfvLvpulmXpmivGRD_Eeq6kCJIe69NShfBt-0YQ9jyxUOzv18B3Zzcy8_nfqauu", "key": "pntoken"}, {"value": "APA91bETfPnCygDyxMKDAFJhvytRDb9xltvT1ASnycRP3guAAYy6f3_-6SJ0snGkJrwvBLXesAmkYl6gL4tk26ue5urZppGErpLXUw6pqaus-ikhoAvs_uq4GuztoqXcljl1rcMy52M3", "key": "pntoken"}, {"value": "APA91bFAoETYeU_0hoS75UV5KvEorTXMP1UnrREDsibXqxsRnp8Ahixx30A1FznKMReFpNgKPGyblHYh8UwtkLLB6ocNVbUGfv4GaP55gJqCieDc8DYBPJLS_MUfdP0QfSiHI_lkXCWu", "key": "pntoken"}, {"value": "APA91bHHwYsRm_CjrFohHltvHvoZABgiGAWiqIUPQqE8QLNizaOXFP4CPV5KyvTINJfd6VORDqD4xQIJgU9c8BWiyNgjkrwUiDEkpc1w1h5vxn3OYPSLdh67CIee7OzPF1a9ekNVZAPy", "key": "pntoken"}], "fd": null, "time": "2019-11-13T08:44:24.605837", "dt": {"primary_id": "3f72d0fd-7f05-4f2a-9e5c-a5a090407cbf__0", "token": ["APA91bGRgCheCfghPyC_4BZuI-cXx3W3-xOd5HeX-93ZM1VfzYxslQPvoxR1FPGJ4zZF6wl7c01NL7l4BKKL2tCX5Gq5LlZ6QD7Rq3V2IM5OQekLjmDSrcFlU6egcvADhZxpqUBLfMBk", "APA91bHbKPa6Ae0ff0-YttkeFUi33tRjK0MyUlWdWeZQR5C8oWE4Ie1N6QIvHlmfmxv5hjYRXhYrCD49E-sc1blpwjH5s2-PlMaMsD0nXNM1OMFEMRCS6l6ROO7RZnYMAMzpORQ0Ah1d", "APA91bE8vcxuzQ8_ItHaw5qaZYyiDo62PoImh0kygmeuNWjgYwb6KQ3Zrin54ZK7sloOB3QgIyd1Y5Cbi-tWdBaphenAyM4TwQ89VD14CSYHDZlAFV1z3IdCgGw4tBMGTJC-xtNX-bgw", "APA91bHJXjBkEzV9A5hfv6h-yv-hUm9ixOEhr-lZp7mgWothmJAIzeiNXyX7lF0Xsk8H46MF77oWroE9GErFwtg2927DSaCHvdgVcFkLqy3sc1drbnnraVnx7QA44q5qCCci7mZ6KNzk", "APA91bE7UESyiIAF_58lgpd9mEETxu9q7XPlXGFpWopbwojlGCzQBbvWLAvH2VV_dIA36dIj9aoT_SfvLvpulmXpmivGRD_Eeq6kCJIe69NShfBt-0YQ9jyxUOzv18B3Zzcy8_nfqauu", "APA91bETfPnCygDyxMKDAFJhvytRDb9xltvT1ASnycRP3guAAYy6f3_-6SJ0snGkJrwvBLXesAmkYl6gL4tk26ue5urZppGErpLXUw6pqaus-ikhoAvs_uq4GuztoqXcljl1rcMy52M3", "APA91bFAoETYeU_0hoS75UV5KvEorTXMP1UnrREDsibXqxsRnp8Ahixx30A1FznKMReFpNgKPGyblHYh8UwtkLLB6ocNVbUGfv4GaP55gJqCieDc8DYBPJLS_MUfdP0QfSiHI_lkXCWu", "APA91bHHwYsRm_CjrFohHltvHvoZABgiGAWiqIUPQqE8QLNizaOXFP4C  ***{"md": "2019-11-13 14:14:24", "ch": "sms", "nm": "DownloadSms", "sent_ack_received_at": "2019-11-13 14:14:24", "app": "pigeon", "nid": "b206b4cb-aab6-4858-8dba-13b4bfebc5ee", "st": 9, "primary_id": "b206b4cb-aab6-4858-8dba-13b4bfebc5ee__0", "time": "2019-11-13T08:44:24.605968", "message": "SENT_ACK_RECIEVED_STATUS"}***
PV5KyvTINJfd6VORDqD4xQIJgU9c8BWiyNgjkrwUiDEkpc1w1h5vxn3OYPSLdh67CIee7OzPF1a9ekNVZAPy"], "message": {"mmt_message": {"priority": 1, "silentPush": {"silent": "false"}, "message": {"sound": "default", "subtext": "Book Now !!", "title": "Exclusive collection of Meals & Deals just for you", "webTitle": "Exclusive collection of Meals & Deals just for you", "text": "Choose from an exclusive collection of Buffet, Spa, Theme Parks, Cabs, Gift Cards and many more. Book memorable Meals & Deals on the MakeMyTrip app today!", "appu": "0", "validity": "YYYY-MM-DD-HH-MM", "ver": "3.8.2", "ntc": "16", "badge": "0"}, "options": {"category": "1", "lob": "Makemytrip Icon", "campaignType": "Fare Alert", "campaign": "acme", "density": "resolution", "webpageURL": "", "slideViewImages": "", "imageURL": "https://images-acme.mmtcdn.com/prod-acme-image/system/product_media/c/33984/media6CNB5P2SQXTL3.jpg", "action1": "", "appVersionCode": "5.6.4", "visibility": "0", "alert": "-1", "priority": "0", "appUpdateType": "0", "deepLinkURL": "mmyt://acme/?page=acmeLanding&categoryName=fd", "expandable": "0", "webpageURLBrowser": "", "action2": ""}, "launchAction": {"lob": "0", "screen": "0", "data": {}}}}, "channel": "android"}, "st": 1, "message": "QUEUED_STATUS", "nm": "acme"}

I have marked the merged log in bold. Is the error related to merging of these logs? please help
Thanks

system · January 6, 2020, 12:36pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.