# Max spans has been reached **URL:** \ **Category:** APM\ **Tags:** docker, java\ **Created:** [February 16, 2022, 10:41am UTC](https://discuss.elastic.co/t/max-spans-has-been-reached/297357 "2022-02-16T10:41:48Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![scroodj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scroodj/32/55132_2.png) [@scroodj](https://discuss.elastic.co/u/scroodj)\ **Post date:** [February 16, 2022, 10:41am UTC](https://discuss.elastic.co/t/max-spans-has-been-reached/297357/1 "2022-02-16T10:41:49Z")
**Kibana version** : 7.17.0 **Elasticsearch version** : 7.17.0 **APM Server version** : 7.17.0 **APM Agent language and version** : Java 8, agent version 1.28.4 **Description of the problem** : Nomatter that value (up to 2147483647) i put into " **transaction\_max\_spans**" properties i get WARN in the app output: > {"@timestamp":"2022-02-16T09:53:54.309Z", "log.level": "WARN", "message":"Max spans (2147483647) for transaction 'someclassnamehere#start' 00-e52f912e1a1185766deeae7ce9f8b76a-88bdcb02f7e163ed-01 (5a1c0881) has been reached. For this transaction and possibly others, further spans will be dropped. See config param 'transaction\_max\_spans'.", "ecs.version": "1.2.0","event.dataset":"someservicenamehere.apm","process.thread.name":"scheduling-1","log.logger":"co.elastic.apm.agent.impl.transaction.Span"} Does it mean that my transaction produce a very enormous spans (which i doubt) that can`t be recorded? Is there a way to record(mb partially) a transaction and show it in Kibana APM or at least to debug the issue and find a root cause? ---
**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\ **Post date:** [February 16, 2022, 1:22pm UTC](https://discuss.elastic.co/t/max-spans-has-been-reached/297357/2 "2022-02-16T13:22:56Z")
These are the two main causes for such issues: 1. There is a transaction leak, meaning - there are transactions that somewhere have reference that do not get cleared, so spans keep getting created for them. If those are transactions that are never be ended, they won't even show in Kibana. If you create and manage transactions manually (i.e. through the public API or using the OpenTracing bridge), it is most probably there. If not and there is indeed a leak - it could be an agent bug we need to look into. 2. The transactions are ended and cleared properly, but there is indeed a very large number of spans created for each. If you set the [`trace_methods`](https://www.elastic.co/guide/en/apm/agent/java/current/config-core.html#config-trace-methods) config - look in there and narrow it. If you only rely only on the inherent agent instrumentations, we may want to see why that is. The best way to get to the root cause is through a `debug` log (see [logging configurations](https://www.elastic.co/guide/en/apm/agent/java/current/config-logging.html)). If this is caused only by agent-inherent instrumentations, we would like to hear all about that - including what exact instrumentation is related, which technology/library and version, and of course your debug log that includes from startup and until you get one or two of these log events. I hope this helps ---
**Author:** ![scroodj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scroodj/32/55132_2.png) [@scroodj](https://discuss.elastic.co/u/scroodj)\ **Post date:** [February 16, 2022, 5:11pm UTC](https://discuss.elastic.co/t/max-spans-has-been-reached/297357/3 "2022-02-16T17:11:59Z")
Hello Eyal, Thank you for the fast feedback. We don`t use the public API or the OpenTracing bridge. SpringBoot 2.3.0 Problematic transaction: DeliveryNoteGoodsDEConsumer#start First appearence of "...Max spans (500) for transaction..." is about 45 min after service startup debug log output: [https://raw.githubusercontent.com/scroodj/apm/main/transcation\_spans\_debug.txt](https://raw.githubusercontent.com/scroodj/apm/main/transcation_spans_debug.txt) ---
**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\ **Post date:** [February 17, 2022, 9:27am UTC](https://discuss.elastic.co/t/max-spans-has-been-reached/297357/4 "2022-02-17T09:27:00Z")
OK, then it is a real issue of creating too many spans and a never ending transaction that traces a scheduled method. What is your `DeliveryNoteGoodsDEConsumer#start` annotated with? Is it `org.springframework.scheduling.annotation.Schedules` or something else? Normally, our instrumentation of such methods assumes that they start and end, letting the framework that runs them to reschedule them every time they finish. Is it possible that this method runs an endless loop? ---
**Author:** ![scroodj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scroodj/32/55132_2.png) [@scroodj](https://discuss.elastic.co/u/scroodj)\ **Post date:** [February 17, 2022, 12:55pm UTC](https://discuss.elastic.co/t/max-spans-has-been-reached/297357/5 "2022-02-17T12:55:48Z")
_Is it org.springframework.scheduling.annotation.Schedules or something else?_ - Yes, It`s org.springframework.scheduling.annotation.Scheduled _Is it possible that this method runs an endless loop?_ - Yes, It runs for ever. ---
**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\ **Post date:** [February 17, 2022, 1:33pm UTC](https://discuss.elastic.co/t/max-spans-has-been-reached/297357/6 "2022-02-17T13:33:40Z")
> [@scroodj](#): > > _Is it possible that this method runs an endless loop?_ > > - Yes, It runs for ever. In that case, the agent behaves as expected. I'd say running an endless loop within a `Scheduled` task defeats the usage of scheduling, which is to let something else measure the timings and do the sleeps between method invocations. If your `Scheduled` method would just run its task and return, you would get meaningful transactions. With your current implementation, these transactions will only be meaningless, they will never end and never be reported. You can disable this instrumentation by setting [`disable_instrumentations`](https://www.elastic.co/guide/en/apm/agent/java/current/config-core.html#config-disable-instrumentations)`=scheduled` and use the public API to monitor some other method that encapsulates the recurring task and does return after every task execution. I hope this helps. ---
**Author:** ![scroodj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scroodj/32/55132_2.png) [@scroodj](https://discuss.elastic.co/u/scroodj)\ **Post date:** [February 18, 2022, 9:23am UTC](https://discuss.elastic.co/t/max-spans-has-been-reached/297357/7 "2022-02-18T09:23:17Z")
Thank you for shedding a light. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 11, 2022, 5:23am UTC](https://discuss.elastic.co/t/max-spans-has-been-reached/297357/8 "2022-03-11T05:23:19Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.