# Filebeat stops sending json logs midway. Version: 6.0.0-rc1 **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [October 31, 2017, 12:37pm UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885 "2017-10-31T12:37:54Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![rohit-smpx](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@rohit-smpx](https://discuss.elastic.co/u/rohit-smpx)\ **Post date:** [October 31, 2017, 12:37pm UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885/1 "2017-10-31T12:37:54Z")
I have some test logs in json format. The number of entries are 900+ in the file I'm testing. After a fresh installation, Filebeat starts sending the logs but after some time (a few minute) starts giving this err: > ERR Failed to publish events: temporary bulk send failure Enabling debug logs, shows this : > DBG Bulk item insert failed (i=13, status=500): {"type":"string\_index\_out\_of\_bounds\_exception","reason":"String index out of range: 0"} The final count of documents in the index created in elasticsearch is 631, less than the 900+ in log file (Untitled-2.json in the logs attached). It has sometimes stopped as early as 100s. The debug logs and the filebeat configuration are attached in this gist: [https://gist.github.com/rohit-smpx/79669da791f8ec76f93043ae9fb505d4](https://gist.github.com/rohit-smpx/79669da791f8ec76f93043ae9fb505d4) ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [November 1, 2017, 12:25pm UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885/2 "2017-11-01T12:25:54Z")
Thanks for reporting. I wonder if the error happens when processing event from the syslog module or due to the json prospector. Either way, it might be a bug. Can you test with only the syslog and only the json prospector being enabled? If one or the other fails can you try to reduce the log files, until we find a few events causing the issue? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [November 1, 2017, 12:27pm UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885/3 "2017-11-01T12:27:04Z")
Also check the Elasticsearch logs. The full exception including a stack-trace should be available in the logs. ---
**Author:** ![rohit-smpx](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@rohit-smpx](https://discuss.elastic.co/u/rohit-smpx)\ **Post date:** [November 1, 2017, 12:56pm UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885/4 "2017-11-01T12:56:00Z")
I updated the gist to include the elasticsearch logs. I didn't see any ERR in there. I did check with the json prospector disabled, and there were no errors. I'll reduce the events and try again. ---
**Author:** ![rohit-smpx](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@rohit-smpx](https://discuss.elastic.co/u/rohit-smpx)\ **Post date:** [November 1, 2017, 1:34pm UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885/5 "2017-11-01T13:34:08Z")
> Disabling syslog fixed it for now. I'll add some debug logs tomorrow. ---
**Author:** ![rohit-smpx](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@rohit-smpx](https://discuss.elastic.co/u/rohit-smpx)\ **Post date:** [November 2, 2017, 7:07am UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885/6 "2017-11-02T07:07:09Z")
This is fixed. In the filebeat conf I had set up the index name as > index: "filebeat.%{[table]}.%{+yyyy.MM}" Where 'table' was a field in the json logs. But the syslog logs didn't have those, so I think the error was because of that. I changed it to this: ```auto indices: - index: "filebeat.%{[table]}.%{+yyyy.MM}" when.regexp: table: ".*" index: "filebeat.syslog.%{+yyyy.MM.dd}" ``` Now, the default index is syslog and if the log has a table field it will go to it's respective index. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [November 2, 2017, 11:58am UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885/7 "2017-11-02T11:58:59Z")
Oh, seems like the index was not correctly set due to this. You can also use 'defaults' like this: ```auto index: 'filebeat.%{[table]:syslog}.%{+yyyy.MM}' ``` ---
**Author:** ![rohit-smpx](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@rohit-smpx](https://discuss.elastic.co/u/rohit-smpx)\ **Post date:** [November 2, 2017, 12:58pm UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885/8 "2017-11-02T12:58:55Z")
Thanks, will do 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 30, 2017, 12:59pm UTC](https://discuss.elastic.co/t/filebeat-stops-sending-json-logs-midway-version-6-0-0-rc1/105885/9 "2017-11-30T12:59:33Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.