# 'Shard Allocation' dashboard in Marvel 1.2 showing SearchParseExceptions

**URL:** <https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969>\
**Category:** Elasticsearch\
**Created:** [June 6, 2014, 10:51pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969 "2014-06-06T22:51:44Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![philsttr](https://avatars.discourse-cdn.com/v4/letter/p/91b2a8/32.png) [@philsttr](https://discuss.elastic.co/u/philsttr)\
**Post date:** [June 6, 2014, 10:51pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/1 "2014-06-06T22:51:44Z")

</div>

I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.

When I go to the new "Shard Allocation" dashboard in Marvel, I see the  
following error:

SearchPhaseExecutionException[Failed to execute phase [query\_fetch], all  
shards failed; shardFailures  
{[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
Parse Failure [Failed to parse source  
[{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]

I see the following in my elasticsearch.log

[2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx] All  
shards failed for phase: [query\_fetch]  
[2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
[.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
org.elasticsearch.search.SearchParseException: [.marvel-2014.06.06][0]:  
from[-1],size[1]: Parse Failure [Failed to parse source  
[{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
at  
org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
at  
org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
at  
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
at  
org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
at  
org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
at  
org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
at  
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:745)  
Caused by: org.elasticsearch.search.SearchParseException:  
[.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
for [@timestamp] in order to sort on]  
at  
org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
at  
org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
at  
org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
at  
org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
... 9 more

I have confirmed that marvel is upgraded on every node in the cluster.

I tried deleting the .marvel\* indexes. Didn't help  
Shutdown the cluster, and restarted it. Didn't help.

Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1aeb0c36-9cd5-4f2b-a8c7-9b32471b4716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1aeb0c36-9cd5-4f2b-a8c7-9b32471b4716%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Boaz\_Leskes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/boaz_leskes/32/723_2.png) [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)\
**Post date:** [June 7, 2014, 6:50pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/2 "2014-06-07T18:50:16Z")

</div>

Hi,

Is there anything in the log indicating data shipping problems from the  
agent? Check the log of the current master node.

Cheers,  
Boaz

On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:

> I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> 
> When I go to the new "Shard Allocation" dashboard in Marvel, I see the  
> following error:
> 
> SearchPhaseExecutionException[Failed to execute phase [query\_fetch], all  
> shards failed; shardFailures  
> {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> Parse Failure [Failed to parse source  
> [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> 
> I see the following in my elasticsearch.log
> 
> [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx] All  
> shards failed for phase: [query\_fetch]  
> [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> org.elasticsearch.search.SearchParseException: [.marvel-2014.06.06][0]:  
> from[-1],size[1]: Parse Failure [Failed to parse source  
> [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> at  
> org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> at  
> org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> at  
> org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> at  
> org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> at  
> org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> at  
> org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> at  
> org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> at  
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> at  
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> at java.lang.Thread.run(Thread.java:745)  
> Caused by: org.elasticsearch.search.SearchParseException:  
> [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> for [@timestamp] in order to sort on]  
> at  
> org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> at  
> org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> at  
> org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> at  
> org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> ... 9 more
> 
> I have confirmed that marvel is upgraded on every node in the cluster.
> 
> I tried deleting the .marvel\* indexes. Didn't help  
> Shutdown the cluster, and restarted it. Didn't help.
> 
> Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4e26069b-c8e3-4963-bec3-2d524945e2e1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4e26069b-c8e3-4963-bec3-2d524945e2e1%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![philsttr](https://avatars.discourse-cdn.com/v4/letter/p/91b2a8/32.png) [@philsttr](https://discuss.elastic.co/u/philsttr)\
**Post date:** [June 9, 2014, 4:35pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/3 "2014-06-09T16:35:30Z")

</div>

Hi Boaz,

The only other exception I see in the logs is this:

[2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx] exporter  
[es\_exporter] has thrown an exception:  
java.lang.IllegalStateException: array not available  
at  
org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
at  
org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
at  
org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
at  
org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
at  
org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
at  
org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
at java.lang.Thread.run(Thread.java:745)

On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:

> Hi,
> 
> Is there anything in the log indicating data shipping problems from the  
> agent? Check the log of the current master node.
> 
> Cheers,  
> Boaz
> 
> On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> 
> > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > 
> > When I go to the new "Shard Allocation" dashboard in Marvel, I see the  
> > following error:
> > 
> > SearchPhaseExecutionException[Failed to execute phase [query\_fetch], all  
> > shards failed; shardFailures  
> > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > Parse Failure [Failed to parse source  
> > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > 
> > I see the following in my elasticsearch.log
> > 
> > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx] All  
> > shards failed for phase: [query\_fetch]  
> > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > org.elasticsearch.search.SearchParseException: [.marvel-2014.06.06][0]:  
> > from[-1],size[1]: Parse Failure [Failed to parse source  
> > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > at  
> > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > at  
> > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > at  
> > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > at  
> > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > at  
> > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > at  
> > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > at  
> > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > at  
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > at  
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > at java.lang.Thread.run(Thread.java:745)  
> > Caused by: org.elasticsearch.search.SearchParseException:  
> > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > for [@timestamp] in order to sort on]  
> > at  
> > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > at  
> > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > at  
> > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > at  
> > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > ... 9 more
> > 
> > I have confirmed that marvel is upgraded on every node in the cluster.
> > 
> > I tried deleting the .marvel\* indexes. Didn't help  
> > Shutdown the cluster, and restarted it. Didn't help.
> > 
> > Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/820ec77c-0d23-42f7-b966-737e6304085f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/820ec77c-0d23-42f7-b966-737e6304085f%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![philsttr](https://avatars.discourse-cdn.com/v4/letter/p/91b2a8/32.png) [@philsttr](https://discuss.elastic.co/u/philsttr)\
**Post date:** [June 9, 2014, 4:39pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/4 "2014-06-09T16:39:45Z")

</div>

Looks like this has already been found and resolved. I'll upgrade to  
Marvel 1.2.1.

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:

> Hi Boaz,
> 
> The only other exception I see in the logs is this:
> 
> [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> exporter [es\_exporter] has thrown an exception:  
> java.lang.IllegalStateException: array not available  
> at  
> org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> at  
> org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> at  
> org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> at  
> org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> at  
> org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> at  
> org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> at java.lang.Thread.run(Thread.java:745)
> 
> On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> 
> > Hi,
> > 
> > Is there anything in the log indicating data shipping problems from the  
> > agent? Check the log of the current master node.
> > 
> > Cheers,  
> > Boaz
> > 
> > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > 
> > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > 
> > > When I go to the new "Shard Allocation" dashboard in Marvel, I see the  
> > > following error:
> > > 
> > > SearchPhaseExecutionException[Failed to execute phase [query\_fetch], all  
> > > shards failed; shardFailures  
> > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > Parse Failure [Failed to parse source  
> > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > 
> > > I see the following in my elasticsearch.log
> > > 
> > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx] All  
> > > shards failed for phase: [query\_fetch]  
> > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > org.elasticsearch.search.SearchParseException: [.marvel-2014.06.06][0]:  
> > > from[-1],size[1]: Parse Failure [Failed to parse source  
> > > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > at  
> > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > at  
> > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > at  
> > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > at  
> > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > at  
> > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > at  
> > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > at  
> > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > at  
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > at  
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > at java.lang.Thread.run(Thread.java:745)  
> > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > for [@timestamp] in order to sort on]  
> > > at  
> > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > at  
> > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > at  
> > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > at  
> > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > ... 9 more
> > > 
> > > I have confirmed that marvel is upgraded on every node in the cluster.
> > > 
> > > I tried deleting the .marvel\* indexes. Didn't help  
> > > Shutdown the cluster, and restarted it. Didn't help.
> > > 
> > > Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/2393de32-1eef-4fe3-868d-a1fb72f948be%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2393de32-1eef-4fe3-868d-a1fb72f948be%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Boaz\_Leskes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/boaz_leskes/32/723_2.png) [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)\
**Post date:** [June 9, 2014, 5:01pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/5 "2014-06-09T17:01:56Z")

</div>

Yeah, I suspected that. Let me know how it goes...

On Mon, Jun 9, 2014 at 6:39 PM, null [philsttr@gmail.com](mailto:philsttr@gmail.com) wrote:

> Looks like this has already been found and resolved. I'll upgrade to  
> Marvel 1.2.1.  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/marvel-1-2-1-released/)  
> On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> 
> > Hi Boaz,
> > 
> > The only other exception I see in the logs is this:
> > 
> > [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> > exporter [es\_exporter] has thrown an exception:  
> > java.lang.IllegalStateException: array not available  
> > at  
> > org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> > at  
> > org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> > at  
> > org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> > at  
> > org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> > at  
> > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> > at  
> > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> > at java.lang.Thread.run(Thread.java:745)
> > 
> > On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> > 
> > > Hi,
> > > 
> > > Is there anything in the log indicating data shipping problems from the  
> > > agent? Check the log of the current master node.
> > > 
> > > Cheers,  
> > > Boaz
> > > 
> > > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > 
> > > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > > 
> > > > When I go to the new "Shard Allocation" dashboard in Marvel, I see the  
> > > > following error:
> > > > 
> > > > SearchPhaseExecutionException[Failed to execute phase [query\_fetch], all  
> > > > shards failed; shardFailures  
> > > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > Parse Failure [Failed to parse source  
> > > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > > 
> > > > I see the following in my elasticsearch.log
> > > > 
> > > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx] All  
> > > > shards failed for phase: [query\_fetch]  
> > > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > > org.elasticsearch.search.SearchParseException: [.marvel-2014.06.06][0]:  
> > > > from[-1],size[1]: Parse Failure [Failed to parse source  
> > > > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > at  
> > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > > at  
> > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > at  
> > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > at  
> > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > at  
> > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > at  
> > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > at java.lang.Thread.run(Thread.java:745)  
> > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > for [@timestamp] in order to sort on]  
> > > > at  
> > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > at  
> > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > at  
> > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > > ... 9 more
> > > > 
> > > > I have confirmed that marvel is upgraded on every node in the cluster.
> > > > 
> > > > I tried deleting the .marvel\* indexes. Didn't help  
> > > > Shutdown the cluster, and restarted it. Didn't help.
> > > > 
> > > > Anybody have ideas?
> 
> --  
> You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/YET1SifrKy4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/YET1SifrKy4/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/2393de32-1eef-4fe3-868d-a1fb72f948be%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2393de32-1eef-4fe3-868d-a1fb72f948be%40googlegroups.com).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1402333315298.de2a2bbe%40Nodemailer](https://groups.google.com/d/msgid/elasticsearch/1402333315298.de2a2bbe%40Nodemailer).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![return\_null](https://avatars.discourse-cdn.com/v4/letter/r/4bbf92/32.png) [@return\_null](https://discuss.elastic.co/u/return_null)\
**Post date:** [September 17, 2014, 4:52pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/6 "2014-09-17T16:52:14Z")

</div>

I am on Marvel 1.2.1 and had the same error (see below). The time on one of  
my nodes was off a fair bit. Fixed that and the problem went away.

[2014-09-17 11:55:14,615][DEBUG][action.search.type] [Pip the Troll]  
All shards failed for phase: [query\_fetch]  
[2014-09-17 12:16:00,532][DEBUG][action.search.type] [Pip the Troll]  
[.marvel-2014.09.17][0], node[2d8iMl4tRHSS8JFPJMN01A], [P], s[STARTED]:  
Failed to execute [org.elasticsearch.action.search.SearchRequest@769694e9]  
org.elasticsearch.search.SearchParseException: [.marvel-2014.09.17][0]:  
from[-1],size[1]: Parse Failure [Failed to parse source  
[{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
at  
org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)  
at  
org.elasticsearch.search.SearchService.createContext(SearchService.java:515)  
at  
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)  
at  
org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:328)  
at  
org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
at  
org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
at  
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:745)  
Caused by: org.elasticsearch.search.SearchParseException:  
[.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [No mapping found  
for [@timestamp] in order to sort on]  
at  
org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
at  
org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
at  
org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
at  
org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)  
... 9 more

On Monday, June 9, 2014 12:39:45 PM UTC-4, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:

> Looks like this has already been found and resolved. I'll upgrade to  
> Marvel 1.2.1.
> 
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/marvel-1-2-1-released/)
> 
> On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> 
> > Hi Boaz,
> > 
> > The only other exception I see in the logs is this:
> > 
> > [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> > exporter [es\_exporter] has thrown an exception:  
> > java.lang.IllegalStateException: array not available  
> > at  
> > org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> > at  
> > org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> > at  
> > org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> > at  
> > org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> > at  
> > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> > at  
> > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> > at java.lang.Thread.run(Thread.java:745)
> > 
> > On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> > 
> > > Hi,
> > > 
> > > Is there anything in the log indicating data shipping problems from the  
> > > agent? Check the log of the current master node.
> > > 
> > > Cheers,  
> > > Boaz
> > > 
> > > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > 
> > > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > > 
> > > > When I go to the new "Shard Allocation" dashboard in Marvel, I see the  
> > > > following error:
> > > > 
> > > > SearchPhaseExecutionException[Failed to execute phase [query\_fetch],  
> > > > all shards failed; shardFailures  
> > > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > Parse Failure [Failed to parse source  
> > > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > > 
> > > > I see the following in my elasticsearch.log
> > > > 
> > > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx] All  
> > > > shards failed for phase: [query\_fetch]  
> > > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > > org.elasticsearch.search.SearchParseException: [.marvel-2014.06.06][0]:  
> > > > from[-1],size[1]: Parse Failure [Failed to parse source  
> > > > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > at  
> > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > > at  
> > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > at  
> > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > at  
> > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > at  
> > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > at  
> > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > at java.lang.Thread.run(Thread.java:745)  
> > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > for [@timestamp] in order to sort on]  
> > > > at  
> > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > at  
> > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > at  
> > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > > ... 9 more
> > > > 
> > > > I have confirmed that marvel is upgraded on every node in the cluster.
> > > > 
> > > > I tried deleting the .marvel\* indexes. Didn't help  
> > > > Shutdown the cluster, and restarted it. Didn't help.
> > > > 
> > > > Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/33caa09e-3db5-4efc-94e5-b258c0ecca96%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/33caa09e-3db5-4efc-94e5-b258c0ecca96%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![rahst12](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rahst12/32/1123_2.png) [@rahst12](https://discuss.elastic.co/u/rahst12)\
**Post date:** [November 7, 2014, 4:54pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/7 "2014-11-07T16:54:09Z")

</div>

I see the same error Were you able to resolve it?

Thanks

On Wednesday, September 17, 2014 12:52:14 PM UTC-4, Sean Kipling wrote:

> I am on Marvel 1.2.1 and had the same error (see below). The time on one  
> of my nodes was off a fair bit. Fixed that and the problem went away.
> 
> [2014-09-17 11:55:14,615][DEBUG][action.search.type] [Pip the  
> Troll] All shards failed for phase: [query\_fetch]  
> [2014-09-17 12:16:00,532][DEBUG][action.search.type] [Pip the  
> Troll] [.marvel-2014.09.17][0], node[2d8iMl4tRHSS8JFPJMN01A], [P],  
> s[STARTED]: Failed to execute  
> [org.elasticsearch.action.search.SearchRequest@769694e9]  
> org.elasticsearch.search.SearchParseException: [.marvel-2014.09.17][0]:  
> from[-1],size[1]: Parse Failure [Failed to parse source  
> [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> at  
> org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)  
> at  
> org.elasticsearch.search.SearchService.createContext(SearchService.java:515)  
> at  
> org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)  
> at  
> org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:328)  
> at  
> org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> at  
> org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> at  
> org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> at  
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> at  
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> at java.lang.Thread.run(Thread.java:745)  
> Caused by: org.elasticsearch.search.SearchParseException:  
> [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [No mapping found  
> for [@timestamp] in order to sort on]  
> at  
> org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> at  
> org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> at  
> org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> at  
> org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)  
> ... 9 more
> 
> On Monday, June 9, 2014 12:39:45 PM UTC-4, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> 
> > Looks like this has already been found and resolved. I'll upgrade to  
> > Marvel 1.2.1.
> > 
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/marvel-1-2-1-released/)
> > 
> > On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > 
> > > Hi Boaz,
> > > 
> > > The only other exception I see in the logs is this:
> > > 
> > > [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> > > exporter [es\_exporter] has thrown an exception:  
> > > java.lang.IllegalStateException: array not available  
> > > at  
> > > org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> > > at  
> > > org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> > > at  
> > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> > > at  
> > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> > > at  
> > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> > > at  
> > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> > > at java.lang.Thread.run(Thread.java:745)
> > > 
> > > On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> > > 
> > > > Hi,
> > > > 
> > > > Is there anything in the log indicating data shipping problems from the  
> > > > agent? Check the log of the current master node.
> > > > 
> > > > Cheers,  
> > > > Boaz
> > > > 
> > > > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > 
> > > > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > > > 
> > > > > When I go to the new "Shard Allocation" dashboard in Marvel, I see the  
> > > > > following error:
> > > > > 
> > > > > SearchPhaseExecutionException[Failed to execute phase [query\_fetch],  
> > > > > all shards failed; shardFailures  
> > > > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > Parse Failure [Failed to parse source  
> > > > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > > > 
> > > > > I see the following in my elasticsearch.log
> > > > > 
> > > > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx] All  
> > > > > shards failed for phase: [query\_fetch]  
> > > > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > > > org.elasticsearch.search.SearchParseException:  
> > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > > > at  
> > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > at  
> > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > at  
> > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > at  
> > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > at  
> > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > for [@timestamp] in order to sort on]  
> > > > > at  
> > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > at  
> > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > at  
> > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > > > ... 9 more
> > > > > 
> > > > > I have confirmed that marvel is upgraded on every node in the cluster.
> > > > > 
> > > > > I tried deleting the .marvel\* indexes. Didn't help  
> > > > > Shutdown the cluster, and restarted it. Didn't help.
> > > > > 
> > > > > Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f83d9fe9-e4d4-4e66-8dba-5386e50d3512%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f83d9fe9-e4d4-4e66-8dba-5386e50d3512%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Max\_Charas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/max_charas/32/889_2.png) [@Max\_Charas](https://discuss.elastic.co/u/Max_Charas)\
**Post date:** [November 25, 2014, 9:29am UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/8 "2014-11-25T09:29:45Z")

</div>

Same problem here.

Log says:

Parse Failure [Failed to parse source  
[{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
Verified that the index(es) have the timestamp field.

Running Marvel 1.2.1 and Elasticsearch 1.3.2.

Any help would be appreciated as we cant get our Marvel up and running.

Den fredagen den 7:e november 2014 kl. 17:54:09 UTC+1 skrev Ryan  
Hendrickson:

> I see the same error Were you able to resolve it?
> 
> Thanks
> 
> On Wednesday, September 17, 2014 12:52:14 PM UTC-4, Sean Kipling wrote:
> 
> > I am on Marvel 1.2.1 and had the same error (see below). The time on one  
> > of my nodes was off a fair bit. Fixed that and the problem went away.
> > 
> > [2014-09-17 11:55:14,615][DEBUG][action.search.type] [Pip the  
> > Troll] All shards failed for phase: [query\_fetch]  
> > [2014-09-17 12:16:00,532][DEBUG][action.search.type] [Pip the  
> > Troll] [.marvel-2014.09.17][0], node[2d8iMl4tRHSS8JFPJMN01A], [P],  
> > s[STARTED]: Failed to execute  
> > [org.elasticsearch.action.search.SearchRequest@769694e9]  
> > org.elasticsearch.search.SearchParseException: [.marvel-2014.09.17][0]:  
> > from[-1],size[1]: Parse Failure [Failed to parse source  
> > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > at  
> > org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)  
> > at  
> > org.elasticsearch.search.SearchService.createContext(SearchService.java:515)  
> > at  
> > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)  
> > at  
> > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:328)  
> > at  
> > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > at  
> > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > at  
> > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > at  
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > at  
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > at java.lang.Thread.run(Thread.java:745)  
> > Caused by: org.elasticsearch.search.SearchParseException:  
> > [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > for [@timestamp] in order to sort on]  
> > at  
> > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > at  
> > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > at  
> > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > at  
> > org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)  
> > ... 9 more
> > 
> > On Monday, June 9, 2014 12:39:45 PM UTC-4, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > 
> > > Looks like this has already been found and resolved. I'll upgrade to  
> > > Marvel 1.2.1.
> > > 
> > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/marvel-1-2-1-released/)
> > > 
> > > On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > 
> > > > Hi Boaz,
> > > > 
> > > > The only other exception I see in the logs is this:
> > > > 
> > > > [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> > > > exporter [es\_exporter] has thrown an exception:  
> > > > java.lang.IllegalStateException: array not available  
> > > > at  
> > > > org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> > > > at  
> > > > org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> > > > at  
> > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> > > > at  
> > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> > > > at  
> > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> > > > at  
> > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> > > > at java.lang.Thread.run(Thread.java:745)
> > > > 
> > > > On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> > > > 
> > > > > Hi,
> > > > > 
> > > > > Is there anything in the log indicating data shipping problems from  
> > > > > the agent? Check the log of the current master node.
> > > > > 
> > > > > Cheers,  
> > > > > Boaz
> > > > > 
> > > > > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > > 
> > > > > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > > > > 
> > > > > > When I go to the new "Shard Allocation" dashboard in Marvel, I see  
> > > > > > the following error:
> > > > > > 
> > > > > > SearchPhaseExecutionException[Failed to execute phase [query\_fetch],  
> > > > > > all shards failed; shardFailures  
> > > > > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > > > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > Parse Failure [Failed to parse source  
> > > > > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > > > > 
> > > > > > I see the following in my elasticsearch.log
> > > > > > 
> > > > > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx]  
> > > > > > All shards failed for phase: [query\_fetch]  
> > > > > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > > > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > > > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > > > > org.elasticsearch.search.SearchParseException:  
> > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > > > > at  
> > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > > at  
> > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > > at  
> > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > > at  
> > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > > at  
> > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > > for [@timestamp] in order to sort on]  
> > > > > > at  
> > > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > > at  
> > > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > > at  
> > > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > > > > ... 9 more
> > > > > > 
> > > > > > I have confirmed that marvel is upgraded on every node in the cluster.
> > > > > > 
> > > > > > I tried deleting the .marvel\* indexes. Didn't help  
> > > > > > Shutdown the cluster, and restarted it. Didn't help.
> > > > > > 
> > > > > > Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ddb93440-ea7f-4a73-a632-eb4615ca458a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ddb93440-ea7f-4a73-a632-eb4615ca458a%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Teo\_Ruiz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/teo_ruiz/32/1073_2.png) [@Teo\_Ruiz](https://discuss.elastic.co/u/Teo_Ruiz)\
**Post date:** [November 26, 2014, 4:24pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/9 "2014-11-26T16:24:57Z")

</div>

Hi guys,

We're seeing exactly the same error. This is the error we see:  
org.elasticsearch.search.SearchParseException: [.marvel-2014.11.26][0]:  
from[-1],size[1]: Parse Failure [Failed to parse source [{"size":1,"sort":{"  
@timestamp":{"order":"desc"}}}]]

Full trace  
here: [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-es_error_log)  
Mappings  
here: [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-marvel_index_mapping-json)

ES version on monitoring and production clusters: 1.3.4  
Marvel version on monitoring and production clusters: 1.2.1

I'm not sure how to proceed, since we removed and reinstalled marvel  
already. The problem started showing when we upgraded the production  
cluster from 0.90.x to 1.3.4.

I'd appreciate some light shedding on this 🙂

On Tuesday, November 25, 2014 10:29:45 AM UTC+1, Max Charas wrote:

> Same problem here.
> 
> Log says:
> 
> Parse Failure [Failed to parse source  
> [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> Verified that the index(es) have the timestamp field.
> 
> Running Marvel 1.2.1 and Elasticsearch 1.3.2.
> 
> Any help would be appreciated as we cant get our Marvel up and running.
> 
> Den fredagen den 7:e november 2014 kl. 17:54:09 UTC+1 skrev Ryan  
> Hendrickson:
> 
> > I see the same error Were you able to resolve it?
> > 
> > Thanks
> > 
> > On Wednesday, September 17, 2014 12:52:14 PM UTC-4, Sean Kipling wrote:
> > 
> > > I am on Marvel 1.2.1 and had the same error (see below). The time on one  
> > > of my nodes was off a fair bit. Fixed that and the problem went away.
> > > 
> > > [2014-09-17 11:55:14,615][DEBUG][action.search.type] [Pip the  
> > > Troll] All shards failed for phase: [query\_fetch]  
> > > [2014-09-17 12:16:00,532][DEBUG][action.search.type] [Pip the  
> > > Troll] [.marvel-2014.09.17][0], node[2d8iMl4tRHSS8JFPJMN01A], [P],  
> > > s[STARTED]: Failed to execute  
> > > [org.elasticsearch.action.search.SearchRequest@769694e9]  
> > > org.elasticsearch.search.SearchParseException: [.marvel-2014.09.17][0]:  
> > > from[-1],size[1]: Parse Failure [Failed to parse source  
> > > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > at  
> > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)  
> > > at  
> > > org.elasticsearch.search.SearchService.createContext(SearchService.java:515)  
> > > at  
> > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)  
> > > at  
> > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:328)  
> > > at  
> > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > at  
> > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > at  
> > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > at  
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > at  
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > at java.lang.Thread.run(Thread.java:745)  
> > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > for [@timestamp] in order to sort on]  
> > > at  
> > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > at  
> > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > at  
> > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > at  
> > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)  
> > > ... 9 more
> > > 
> > > On Monday, June 9, 2014 12:39:45 PM UTC-4, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > 
> > > > Looks like this has already been found and resolved. I'll upgrade to  
> > > > Marvel 1.2.1.
> > > > 
> > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/marvel-1-2-1-released/)
> > > > 
> > > > On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > 
> > > > > Hi Boaz,
> > > > > 
> > > > > The only other exception I see in the logs is this:
> > > > > 
> > > > > [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> > > > > exporter [es\_exporter] has thrown an exception:  
> > > > > java.lang.IllegalStateException: array not available  
> > > > > at  
> > > > > org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> > > > > at  
> > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> > > > > at  
> > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> > > > > at  
> > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> > > > > at  
> > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> > > > > at  
> > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > 
> > > > > On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> > > > > 
> > > > > > Hi,
> > > > > > 
> > > > > > Is there anything in the log indicating data shipping problems from  
> > > > > > the agent? Check the log of the current master node.
> > > > > > 
> > > > > > Cheers,  
> > > > > > Boaz
> > > > > > 
> > > > > > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > > > 
> > > > > > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > > > > > 
> > > > > > > When I go to the new "Shard Allocation" dashboard in Marvel, I see  
> > > > > > > the following error:
> > > > > > > 
> > > > > > > SearchPhaseExecutionException[Failed to execute phase [query\_fetch],  
> > > > > > > all shards failed; shardFailures  
> > > > > > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > > > > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > Parse Failure [Failed to parse source  
> > > > > > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > > > > > 
> > > > > > > I see the following in my elasticsearch.log
> > > > > > > 
> > > > > > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx]  
> > > > > > > All shards failed for phase: [query\_fetch]  
> > > > > > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > > > > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > > > > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > > > > > org.elasticsearch.search.SearchParseException:  
> > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > > > at  
> > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > > > at  
> > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > > > for [@timestamp] in order to sort on]  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > > > at  
> > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > > > > > ... 9 more
> > > > > > > 
> > > > > > > I have confirmed that marvel is upgraded on every node in the  
> > > > > > > cluster.
> > > > > > > 
> > > > > > > I tried deleting the .marvel\* indexes. Didn't help  
> > > > > > > Shutdown the cluster, and restarted it. Didn't help.
> > > > > > > 
> > > > > > > Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e897ccba-e5ac-49c6-baa3-6f43208bf095%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e897ccba-e5ac-49c6-baa3-6f43208bf095%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Boaz\_Leskes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/boaz_leskes/32/723_2.png) [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)\
**Post date:** [November 30, 2014, 10:45pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/10 "2014-11-30T22:45:15Z")

</div>

Hi Teo, Max,

From the query, I can see that it that cluster\_state is not shipped. The  
marvel index template adds the cluster\_type with some non-default fields  
but all fields (like @timestamp) are added when the first document is  
shipped.

Can you validate there are no documents of that type?

Further a couple of clarifying questions:

1. Did you install the marvel agent on all cluster nodes? i.e., also master  
nodes? The later is the ones that ship that data..
2. Did you upgrade from a previous version of marvel or is it a fresh  
install?

Cheers,  
Boaz

On Wednesday, November 26, 2014 5:24:58 PM UTC+1, Teo Ruiz wrote:

> Hi guys,
> 
> We're seeing exactly the same error. This is the error we see:  
> org.elasticsearch.search.SearchParseException: [.marvel-2014.11.26][0]:  
> from[-1],size[1]: Parse Failure [Failed to parse source [{"size":1,"sort":  
> {"@timestamp":{"order":"desc"}}}]]
> 
> Full trace here:  
> [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-es_error_log)  
> Mappings here:  
> [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-marvel_index_mapping-json)
> 
> ES version on monitoring and production clusters: 1.3.4  
> Marvel version on monitoring and production clusters: 1.2.1
> 
> I'm not sure how to proceed, since we removed and reinstalled marvel  
> already. The problem started showing when we upgraded the production  
> cluster from 0.90.x to 1.3.4.
> 
> I'd appreciate some light shedding on this 🙂
> 
> On Tuesday, November 25, 2014 10:29:45 AM UTC+1, Max Charas wrote:
> 
> > Same problem here.
> > 
> > Log says:
> > 
> > Parse Failure [Failed to parse source  
> > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > Verified that the index(es) have the timestamp field.
> > 
> > Running Marvel 1.2.1 and Elasticsearch 1.3.2.
> > 
> > Any help would be appreciated as we cant get our Marvel up and running.
> > 
> > Den fredagen den 7:e november 2014 kl. 17:54:09 UTC+1 skrev Ryan  
> > Hendrickson:
> > 
> > > I see the same error Were you able to resolve it?
> > > 
> > > Thanks
> > > 
> > > On Wednesday, September 17, 2014 12:52:14 PM UTC-4, Sean Kipling wrote:
> > > 
> > > > I am on Marvel 1.2.1 and had the same error (see below). The time on  
> > > > one of my nodes was off a fair bit. Fixed that and the problem went away.
> > > > 
> > > > [2014-09-17 11:55:14,615][DEBUG][action.search.type] [Pip the  
> > > > Troll] All shards failed for phase: [query\_fetch]  
> > > > [2014-09-17 12:16:00,532][DEBUG][action.search.type] [Pip the  
> > > > Troll] [.marvel-2014.09.17][0], node[2d8iMl4tRHSS8JFPJMN01A], [P],  
> > > > s[STARTED]: Failed to execute  
> > > > [org.elasticsearch.action.search.SearchRequest@769694e9]  
> > > > org.elasticsearch.search.SearchParseException: [.marvel-2014.09.17][0]:  
> > > > from[-1],size[1]: Parse Failure [Failed to parse source  
> > > > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > at  
> > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:515)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:328)  
> > > > at  
> > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > at  
> > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > at  
> > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > at  
> > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > at  
> > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > at java.lang.Thread.run(Thread.java:745)  
> > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > for [@timestamp] in order to sort on]  
> > > > at  
> > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > at  
> > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > at  
> > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > at  
> > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)  
> > > > ... 9 more
> > > > 
> > > > On Monday, June 9, 2014 12:39:45 PM UTC-4, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > 
> > > > > Looks like this has already been found and resolved. I'll upgrade to  
> > > > > Marvel 1.2.1.
> > > > > 
> > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/marvel-1-2-1-released/)
> > > > > 
> > > > > On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > > 
> > > > > > Hi Boaz,
> > > > > > 
> > > > > > The only other exception I see in the logs is this:
> > > > > > 
> > > > > > [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> > > > > > exporter [es\_exporter] has thrown an exception:  
> > > > > > java.lang.IllegalStateException: array not available  
> > > > > > at  
> > > > > > org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> > > > > > at  
> > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> > > > > > at  
> > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> > > > > > at  
> > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> > > > > > at  
> > > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> > > > > > at  
> > > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> > > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > > 
> > > > > > On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> > > > > > 
> > > > > > > Hi,
> > > > > > > 
> > > > > > > Is there anything in the log indicating data shipping problems from  
> > > > > > > the agent? Check the log of the current master node.
> > > > > > > 
> > > > > > > Cheers,  
> > > > > > > Boaz
> > > > > > > 
> > > > > > > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com)  
> > > > > > > wrote:
> > > > > > > 
> > > > > > > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > > > > > > 
> > > > > > > > When I go to the new "Shard Allocation" dashboard in Marvel, I see  
> > > > > > > > the following error:
> > > > > > > > 
> > > > > > > > SearchPhaseExecutionException[Failed to execute phase  
> > > > > > > > [query\_fetch], all shards failed; shardFailures  
> > > > > > > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > > > > > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > > Parse Failure [Failed to parse source  
> > > > > > > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > > > > > > 
> > > > > > > > I see the following in my elasticsearch.log
> > > > > > > > 
> > > > > > > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx]  
> > > > > > > > All shards failed for phase: [query\_fetch]  
> > > > > > > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > > > > > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > > > > > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > > > > > > org.elasticsearch.search.SearchParseException:  
> > > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > > > > at  
> > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > > > > at  
> > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > > > > for [@timestamp] in order to sort on]  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > > > > > > ... 9 more
> > > > > > > > 
> > > > > > > > I have confirmed that marvel is upgraded on every node in the  
> > > > > > > > cluster.
> > > > > > > > 
> > > > > > > > I tried deleting the .marvel\* indexes. Didn't help  
> > > > > > > > Shutdown the cluster, and restarted it. Didn't help.
> > > > > > > > 
> > > > > > > > Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d0519c6d-0cba-4f55-9521-0ce6c945f284%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d0519c6d-0cba-4f55-9521-0ce6c945f284%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Max\_Charas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/max_charas/32/889_2.png) [@Max\_Charas](https://discuss.elastic.co/u/Max_Charas)\
**Post date:** [December 1, 2014, 7:17pm UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/11 "2014-12-01T19:17:23Z")

</div>

Hello Boaz,  
Thanks for helping!

I read through your post a few times and realized that it was probably the  
master node that wasn't sending any date as you indicated. A restart of the  
node fixed the problem.

I have no idea why it wasn't shipping any data to start with, most likely  
we hadn't restarted it since we installed the plugin but I'm not completely  
sure.

In any case its working now.

Many thanks,  
Max

Den söndagen den 30:e november 2014 kl. 23:45:15 UTC+1 skrev Boaz Leskes:

> Hi Teo, Max,
> 
> From the query, I can see that it that cluster\_state is not shipped. The  
> marvel index template adds the cluster\_type with some non-default fields  
> but all fields (like @timestamp) are added when the first document is  
> shipped.
> 
> Can you validate there are no documents of that type?
> 
> Further a couple of clarifying questions:
> 
> 1. Did you install the marvel agent on all cluster nodes? i.e., also  
> master nodes? The later is the ones that ship that data..
> 2. Did you upgrade from a previous version of marvel or is it a fresh  
> install?
> 
> Cheers,  
> Boaz
> 
> On Wednesday, November 26, 2014 5:24:58 PM UTC+1, Teo Ruiz wrote:
> 
> > Hi guys,
> > 
> > We're seeing exactly the same error. This is the error we see:  
> > org.elasticsearch.search.SearchParseException: [.marvel-2014.11.26][0]:  
> > from[-1],size[1]: Parse Failure [Failed to parse source [{"size":1,"sort"  
> > :{"@timestamp":{"order":"desc"}}}]]
> > 
> > Full trace here:  
> > [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-es_error_log)  
> > Mappings here:  
> > [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-marvel_index_mapping-json)
> > 
> > ES version on monitoring and production clusters: 1.3.4  
> > Marvel version on monitoring and production clusters: 1.2.1
> > 
> > I'm not sure how to proceed, since we removed and reinstalled marvel  
> > already. The problem started showing when we upgraded the production  
> > cluster from 0.90.x to 1.3.4.
> > 
> > I'd appreciate some light shedding on this 🙂
> > 
> > On Tuesday, November 25, 2014 10:29:45 AM UTC+1, Max Charas wrote:
> > 
> > > Same problem here.
> > > 
> > > Log says:
> > > 
> > > Parse Failure [Failed to parse source  
> > > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > Verified that the index(es) have the timestamp field.
> > > 
> > > Running Marvel 1.2.1 and Elasticsearch 1.3.2.
> > > 
> > > Any help would be appreciated as we cant get our Marvel up and running.
> > > 
> > > Den fredagen den 7:e november 2014 kl. 17:54:09 UTC+1 skrev Ryan  
> > > Hendrickson:
> > > 
> > > > I see the same error Were you able to resolve it?
> > > > 
> > > > Thanks
> > > > 
> > > > On Wednesday, September 17, 2014 12:52:14 PM UTC-4, Sean Kipling wrote:
> > > > 
> > > > > I am on Marvel 1.2.1 and had the same error (see below). The time on  
> > > > > one of my nodes was off a fair bit. Fixed that and the problem went away.
> > > > > 
> > > > > [2014-09-17 11:55:14,615][DEBUG][action.search.type] [Pip the  
> > > > > Troll] All shards failed for phase: [query\_fetch]  
> > > > > [2014-09-17 12:16:00,532][DEBUG][action.search.type] [Pip the  
> > > > > Troll] [.marvel-2014.09.17][0], node[2d8iMl4tRHSS8JFPJMN01A], [P],  
> > > > > s[STARTED]: Failed to execute  
> > > > > [org.elasticsearch.action.search.SearchRequest@769694e9]  
> > > > > org.elasticsearch.search.SearchParseException:  
> > > > > [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:515)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:328)  
> > > > > at  
> > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > at  
> > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > at  
> > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > at  
> > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > at  
> > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > for [@timestamp] in order to sort on]  
> > > > > at  
> > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > at  
> > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > at  
> > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)  
> > > > > ... 9 more
> > > > > 
> > > > > On Monday, June 9, 2014 12:39:45 PM UTC-4, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > > 
> > > > > > Looks like this has already been found and resolved. I'll upgrade to  
> > > > > > Marvel 1.2.1.
> > > > > > 
> > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/marvel-1-2-1-released/)
> > > > > > 
> > > > > > On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > > > 
> > > > > > > Hi Boaz,
> > > > > > > 
> > > > > > > The only other exception I see in the logs is this:
> > > > > > > 
> > > > > > > [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> > > > > > > exporter [es\_exporter] has thrown an exception:  
> > > > > > > java.lang.IllegalStateException: array not available  
> > > > > > > at  
> > > > > > > org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> > > > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > > > 
> > > > > > > On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> > > > > > > 
> > > > > > > > Hi,
> > > > > > > > 
> > > > > > > > Is there anything in the log indicating data shipping problems from  
> > > > > > > > the agent? Check the log of the current master node.
> > > > > > > > 
> > > > > > > > Cheers,  
> > > > > > > > Boaz
> > > > > > > > 
> > > > > > > > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com)  
> > > > > > > > wrote:
> > > > > > > > 
> > > > > > > > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > > > > > > > 
> > > > > > > > > When I go to the new "Shard Allocation" dashboard in Marvel, I see  
> > > > > > > > > the following error:
> > > > > > > > > 
> > > > > > > > > SearchPhaseExecutionException[Failed to execute phase  
> > > > > > > > > [query\_fetch], all shards failed; shardFailures  
> > > > > > > > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > > > > > > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > > > Parse Failure [Failed to parse source  
> > > > > > > > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > > > > > > > 
> > > > > > > > > I see the following in my elasticsearch.log
> > > > > > > > > 
> > > > > > > > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx]  
> > > > > > > > > All shards failed for phase: [query\_fetch]  
> > > > > > > > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > > > > > > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > > > > > > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > > > > > > > org.elasticsearch.search.SearchParseException:  
> > > > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > > > > > at  
> > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > > > > > at  
> > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > > > > > for [@timestamp] in order to sort on]  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > > > > > > > ... 9 more
> > > > > > > > > 
> > > > > > > > > I have confirmed that marvel is upgraded on every node in the  
> > > > > > > > > cluster.
> > > > > > > > > 
> > > > > > > > > I tried deleting the .marvel\* indexes. Didn't help  
> > > > > > > > > Shutdown the cluster, and restarted it. Didn't help.
> > > > > > > > > 
> > > > > > > > > Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/aa6036bc-1979-4b0c-a7f2-e35c4a96d80f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/aa6036bc-1979-4b0c-a7f2-e35c4a96d80f%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Teo\_Ruiz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/teo_ruiz/32/1073_2.png) [@Teo\_Ruiz](https://discuss.elastic.co/u/Teo_Ruiz)\
**Post date:** [December 2, 2014, 10:07am UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/12 "2014-12-02T10:07:09Z")

</div>

Hi Boaz,

I finally restarted the master node and it worked, indeed. It's anyway a  
quite confusing error message / situation 🙂

Cheers,

On Sunday, November 30, 2014 11:45:15 PM UTC+1, Boaz Leskes wrote:

> Hi Teo, Max,
> 
> From the query, I can see that it that cluster\_state is not shipped. The  
> marvel index template adds the cluster\_type with some non-default fields  
> but all fields (like @timestamp) are added when the first document is  
> shipped.
> 
> Can you validate there are no documents of that type?
> 
> Further a couple of clarifying questions:
> 
> 1. Did you install the marvel agent on all cluster nodes? i.e., also  
> master nodes? The later is the ones that ship that data..
> 2. Did you upgrade from a previous version of marvel or is it a fresh  
> install?
> 
> Cheers,  
> Boaz
> 
> On Wednesday, November 26, 2014 5:24:58 PM UTC+1, Teo Ruiz wrote:
> 
> > Hi guys,
> > 
> > We're seeing exactly the same error. This is the error we see:  
> > org.elasticsearch.search.SearchParseException: [.marvel-2014.11.26][0]:  
> > from[-1],size[1]: Parse Failure [Failed to parse source [{"size":1,"sort"  
> > :{"@timestamp":{"order":"desc"}}}]]
> > 
> > Full trace here:  
> > [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-es_error_log)  
> > Mappings here:  
> > [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-marvel_index_mapping-json)
> > 
> > ES version on monitoring and production clusters: 1.3.4  
> > Marvel version on monitoring and production clusters: 1.2.1
> > 
> > I'm not sure how to proceed, since we removed and reinstalled marvel  
> > already. The problem started showing when we upgraded the production  
> > cluster from 0.90.x to 1.3.4.
> > 
> > I'd appreciate some light shedding on this 🙂
> > 
> > On Tuesday, November 25, 2014 10:29:45 AM UTC+1, Max Charas wrote:
> > 
> > > Same problem here.
> > > 
> > > Log says:
> > > 
> > > Parse Failure [Failed to parse source  
> > > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > Verified that the index(es) have the timestamp field.
> > > 
> > > Running Marvel 1.2.1 and Elasticsearch 1.3.2.
> > > 
> > > Any help would be appreciated as we cant get our Marvel up and running.
> > > 
> > > Den fredagen den 7:e november 2014 kl. 17:54:09 UTC+1 skrev Ryan  
> > > Hendrickson:
> > > 
> > > > I see the same error Were you able to resolve it?
> > > > 
> > > > Thanks
> > > > 
> > > > On Wednesday, September 17, 2014 12:52:14 PM UTC-4, Sean Kipling wrote:
> > > > 
> > > > > I am on Marvel 1.2.1 and had the same error (see below). The time on  
> > > > > one of my nodes was off a fair bit. Fixed that and the problem went away.
> > > > > 
> > > > > [2014-09-17 11:55:14,615][DEBUG][action.search.type] [Pip the  
> > > > > Troll] All shards failed for phase: [query\_fetch]  
> > > > > [2014-09-17 12:16:00,532][DEBUG][action.search.type] [Pip the  
> > > > > Troll] [.marvel-2014.09.17][0], node[2d8iMl4tRHSS8JFPJMN01A], [P],  
> > > > > s[STARTED]: Failed to execute  
> > > > > [org.elasticsearch.action.search.SearchRequest@769694e9]  
> > > > > org.elasticsearch.search.SearchParseException:  
> > > > > [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:515)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:328)  
> > > > > at  
> > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > at  
> > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > at  
> > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > at  
> > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > at  
> > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > for [@timestamp] in order to sort on]  
> > > > > at  
> > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > at  
> > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > at  
> > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > at  
> > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)  
> > > > > ... 9 more
> > > > > 
> > > > > On Monday, June 9, 2014 12:39:45 PM UTC-4, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > > 
> > > > > > Looks like this has already been found and resolved. I'll upgrade to  
> > > > > > Marvel 1.2.1.
> > > > > > 
> > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/marvel-1-2-1-released/)
> > > > > > 
> > > > > > On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > > > 
> > > > > > > Hi Boaz,
> > > > > > > 
> > > > > > > The only other exception I see in the logs is this:
> > > > > > > 
> > > > > > > [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> > > > > > > exporter [es\_exporter] has thrown an exception:  
> > > > > > > java.lang.IllegalStateException: array not available  
> > > > > > > at  
> > > > > > > org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> > > > > > > at  
> > > > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> > > > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > > > 
> > > > > > > On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> > > > > > > 
> > > > > > > > Hi,
> > > > > > > > 
> > > > > > > > Is there anything in the log indicating data shipping problems from  
> > > > > > > > the agent? Check the log of the current master node.
> > > > > > > > 
> > > > > > > > Cheers,  
> > > > > > > > Boaz
> > > > > > > > 
> > > > > > > > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com)  
> > > > > > > > wrote:
> > > > > > > > 
> > > > > > > > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > > > > > > > 
> > > > > > > > > When I go to the new "Shard Allocation" dashboard in Marvel, I see  
> > > > > > > > > the following error:
> > > > > > > > > 
> > > > > > > > > SearchPhaseExecutionException[Failed to execute phase  
> > > > > > > > > [query\_fetch], all shards failed; shardFailures  
> > > > > > > > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > > > > > > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > > > Parse Failure [Failed to parse source  
> > > > > > > > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > > > > > > > 
> > > > > > > > > I see the following in my elasticsearch.log
> > > > > > > > > 
> > > > > > > > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx]  
> > > > > > > > > All shards failed for phase: [query\_fetch]  
> > > > > > > > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > > > > > > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > > > > > > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > > > > > > > org.elasticsearch.search.SearchParseException:  
> > > > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > > > > > at  
> > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > > > > > at  
> > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > > > > > for [@timestamp] in order to sort on]  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > > > > > at  
> > > > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > > > > > > > ... 9 more
> > > > > > > > > 
> > > > > > > > > I have confirmed that marvel is upgraded on every node in the  
> > > > > > > > > cluster.
> > > > > > > > > 
> > > > > > > > > I tried deleting the .marvel\* indexes. Didn't help  
> > > > > > > > > Shutdown the cluster, and restarted it. Didn't help.
> > > > > > > > > 
> > > > > > > > > Anybody have ideas?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/14c54640-a555-4b6e-be4a-5a52d1fb2f7f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/14c54640-a555-4b6e-be4a-5a52d1fb2f7f%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Boaz\_Leskes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/boaz_leskes/32/723_2.png) [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)\
**Post date:** [December 2, 2014, 10:46am UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/13 "2014-12-02T10:46:49Z")

</div>

Agreed. Happy things work now.

On Tue, Dec 2, 2014 at 11:07 AM, Teo Ruiz [teoruiz@gmail.com](mailto:teoruiz@gmail.com) wrote:

> Hi Boaz,  
> I finally restarted the master node and it worked, indeed. It's anyway a  
> quite confusing error message / situation 🙂  
> Cheers,  
> On Sunday, November 30, 2014 11:45:15 PM UTC+1, Boaz Leskes wrote:
> 
> > Hi Teo, Max,
> > 
> > From the query, I can see that it that cluster\_state is not shipped. The  
> > marvel index template adds the cluster\_type with some non-default fields  
> > but all fields (like @timestamp) are added when the first document is  
> > shipped.
> > 
> > Can you validate there are no documents of that type?
> > 
> > Further a couple of clarifying questions:
> > 
> > 1. Did you install the marvel agent on all cluster nodes? i.e., also  
> > master nodes? The later is the ones that ship that data..
> > 2. Did you upgrade from a previous version of marvel or is it a fresh  
> > install?
> > 
> > Cheers,  
> > Boaz
> > 
> > On Wednesday, November 26, 2014 5:24:58 PM UTC+1, Teo Ruiz wrote:
> > 
> > > Hi guys,
> > > 
> > > We're seeing exactly the same error. This is the error we see:  
> > > org.elasticsearch.search.SearchParseException: [.marvel-2014.11.26][0]:  
> > > from[-1],size[1]: Parse Failure [Failed to parse source [{"size":1,"sort"  
> > > :{"@timestamp":{"order":"desc"}}}]]
> > > 
> > > Full trace here:  
> > > [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-es_error_log)  
> > > Mappings here:  
> > > [marvel failure with mapping · GitHub](https://gist.github.com/teoruiz/9cdb4e9cd7dc5e564346#file-marvel_index_mapping-json)
> > > 
> > > ES version on monitoring and production clusters: 1.3.4  
> > > Marvel version on monitoring and production clusters: 1.2.1
> > > 
> > > I'm not sure how to proceed, since we removed and reinstalled marvel  
> > > already. The problem started showing when we upgraded the production  
> > > cluster from 0.90.x to 1.3.4.
> > > 
> > > I'd appreciate some light shedding on this 🙂
> > > 
> > > On Tuesday, November 25, 2014 10:29:45 AM UTC+1, Max Charas wrote:
> > > 
> > > > Same problem here.
> > > > 
> > > > Log says:
> > > > 
> > > > Parse Failure [Failed to parse source  
> > > > [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > Verified that the index(es) have the timestamp field.
> > > > 
> > > > Running Marvel 1.2.1 and Elasticsearch 1.3.2.
> > > > 
> > > > Any help would be appreciated as we cant get our Marvel up and running.
> > > > 
> > > > Den fredagen den 7:e november 2014 kl. 17:54:09 UTC+1 skrev Ryan  
> > > > Hendrickson:
> > > > 
> > > > > I see the same error Were you able to resolve it?
> > > > > 
> > > > > Thanks
> > > > > 
> > > > > On Wednesday, September 17, 2014 12:52:14 PM UTC-4, Sean Kipling wrote:
> > > > > 
> > > > > > I am on Marvel 1.2.1 and had the same error (see below). The time on  
> > > > > > one of my nodes was off a fair bit. Fixed that and the problem went away.
> > > > > > 
> > > > > > [2014-09-17 11:55:14,615][DEBUG][action.search.type] [Pip the  
> > > > > > Troll] All shards failed for phase: [query\_fetch]  
> > > > > > [2014-09-17 12:16:00,532][DEBUG][action.search.type] [Pip the  
> > > > > > Troll] [.marvel-2014.09.17][0], node[2d8iMl4tRHSS8JFPJMN01A], [P],  
> > > > > > s[STARTED]: Failed to execute  
> > > > > > [org.elasticsearch.action.search.SearchRequest@769694e9]  
> > > > > > org.elasticsearch.search.SearchParseException:  
> > > > > > [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:664)  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:515)  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:487)  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:328)  
> > > > > > at  
> > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > > at  
> > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > > at  
> > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > > at  
> > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > > at  
> > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > > [.marvel-2014.09.17][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > > for [@timestamp] in order to sort on]  
> > > > > > at  
> > > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > > at  
> > > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > > at  
> > > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > > at  
> > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:648)  
> > > > > > ... 9 more
> > > > > > 
> > > > > > On Monday, June 9, 2014 12:39:45 PM UTC-4, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > > > 
> > > > > > > Looks like this has already been found and resolved. I'll upgrade to  
> > > > > > > Marvel 1.2.1.
> > > > > > > 
> > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/marvel-1-2-1-released/)
> > > > > > > 
> > > > > > > On Monday, June 9, 2014 9:35:30 AM UTC-7, [phil...@gmail.com](mailto:phil...@gmail.com) wrote:
> > > > > > > 
> > > > > > > > Hi Boaz,
> > > > > > > > 
> > > > > > > > The only other exception I see in the logs is this:
> > > > > > > > 
> > > > > > > > [2014-06-09 15:38:11,340][ERROR][marvel.agent] [xxxx]  
> > > > > > > > exporter [es\_exporter] has thrown an exception:  
> > > > > > > > java.lang.IllegalStateException: array not available  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.common.bytes.PagedBytesReference.array(PagedBytesReference.java:289)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.addXContentRendererToConnection(ESExporter.java:209)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent(ESExporter.java:252)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.marvel.agent.exporter.ESExporter.exportEvents(ESExporter.java:161)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.exportEvents(AgentService.java:305)  
> > > > > > > > at  
> > > > > > > > org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(AgentService.java:240)  
> > > > > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > > > > 
> > > > > > > > On Saturday, June 7, 2014 11:50:16 AM UTC-7, Boaz Leskes wrote:
> > > > > > > > 
> > > > > > > > > Hi,
> > > > > > > > > 
> > > > > > > > > Is there anything in the log indicating data shipping problems from  
> > > > > > > > > the agent? Check the log of the current master node.
> > > > > > > > > 
> > > > > > > > > Cheers,  
> > > > > > > > > Boaz
> > > > > > > > > 
> > > > > > > > > On Saturday, June 7, 2014 12:51:44 AM UTC+2, [phil...@gmail.com](mailto:phil...@gmail.com)  
> > > > > > > > > wrote:
> > > > > > > > > 
> > > > > > > > > > I'm running elasticsearch 1.2.1, and just upgraded to Marvel 1.2.
> > > > > > > > > > 
> > > > > > > > > > When I go to the new "Shard Allocation" dashboard in Marvel, I see  
> > > > > > > > > > the following error:
> > > > > > > > > > 
> > > > > > > > > > SearchPhaseExecutionException[Failed to execute phase  
> > > > > > > > > > [query\_fetch], all shards failed; shardFailures  
> > > > > > > > > > {[5FNjdIADRE2rRuOkDZdieA][.marvel-2014.06.04][0]:  
> > > > > > > > > > RemoteTransportException[[xxxx][inet[/x.x.x.x:9300]][search/phase/query+fetch]];  
> > > > > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > > > > Parse Failure [Failed to parse source  
> > > > > > > > > > [{"size":300,"from":0,"fields":["@timestamp","message","status"],"sort":{"@timestamp":{"order":"desc"}},"query":{"filtered":{"filter":{"range":{"@timestamp":{"from":"2014-05-30T22:33:17.689Z","to":"2014-06-06T22:33:17.689Z"}}}}}}]]];  
> > > > > > > > > > nested: SearchParseException[[.marvel-2014.06.04][0]: from[0],size[300]:  
> > > > > > > > > > Parse Failure [No mapping found for [@timestamp] in order to sort on]]; }]
> > > > > > > > > > 
> > > > > > > > > > I see the following in my elasticsearch.log
> > > > > > > > > > 
> > > > > > > > > > [2014-06-06 22:48:07,919][DEBUG][action.search.type] [xxxx]  
> > > > > > > > > > All shards failed for phase: [query\_fetch]  
> > > > > > > > > > [2014-06-06 22:48:17,937][DEBUG][action.search.type] [xxxx]  
> > > > > > > > > > [.marvel-2014.06.06][0], node[5FNjdIADRE2rRuOkDZdieA], [R], s[STARTED]:  
> > > > > > > > > > Failed to execute [org.elasticsearch.action.search.SearchRequest@6af079e8]  
> > > > > > > > > > org.elasticsearch.search.SearchParseException:  
> > > > > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [Failed to parse  
> > > > > > > > > > source [{"size":1,"sort":{"@timestamp":{"order":"desc"}}}]]  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.SearchService.createContext(SearchService.java:511)  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:324)  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:308)  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$11.call(SearchServiceTransportAction.java:305)  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)  
> > > > > > > > > > at  
> > > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> > > > > > > > > > at  
> > > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> > > > > > > > > > at java.lang.Thread.run(Thread.java:745)  
> > > > > > > > > > Caused by: org.elasticsearch.search.SearchParseException:  
> > > > > > > > > > [.marvel-2014.06.06][0]: from[-1],size[1]: Parse Failure [No mapping found  
> > > > > > > > > > for [@timestamp] in order to sort on]  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.sort.SortParseElement.addSortField(SortParseElement.java:198)  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.sort.SortParseElement.addCompoundSortField(SortParseElement.java:172)  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.sort.SortParseElement.parse(SortParseElement.java:90)  
> > > > > > > > > > at  
> > > > > > > > > > org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)  
> > > > > > > > > > ... 9 more
> > > > > > > > > > 
> > > > > > > > > > I have confirmed that marvel is upgraded on every node in the  
> > > > > > > > > > cluster.
> > > > > > > > > > 
> > > > > > > > > > I tried deleting the .marvel\* indexes. Didn't help  
> > > > > > > > > > Shutdown the cluster, and restarted it. Didn't help.
> > > > > > > > > > 
> > > > > > > > > > Anybody have ideas?
> 
> --  
> You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/YET1SifrKy4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/YET1SifrKy4/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/14c54640-a555-4b6e-be4a-5a52d1fb2f7f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/14c54640-a555-4b6e-be4a-5a52d1fb2f7f%40googlegroups.com).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1417517208059.f25f0b45%40Nodemailer](https://groups.google.com/d/msgid/elasticsearch/1417517208059.f25f0b45%40Nodemailer).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:46am UTC](https://discuss.elastic.co/t/shard-allocation-dashboard-in-marvel-1-2-showing-searchparseexceptions/17969/14 "2017-07-06T00:46:39Z")

</div>