TokenStream contract violation: close() call missing

An internal error happens when I do a "suggest" query. I get "TokenStream
contract violation: close() call missing"

This only happens when I add pre_filter to the suggest body.

    {
        'text': 'my term',
        'suggest' : {
            'phrase' : {
                'field' : 'title',
                'direct_generator' : [
                    {
                        'field' : 'title.suggest',
                        'suggest_mode': 'always',
                        'pre_filter': 'autocomplete' // <-- this 

triggers the problem
}
]
}
}
}

However, I need pre_filter to achieve the desired outcome.

According the this thread
http://elasticsearch-users.115913.n3.nabble.com/TokenStream-contract-violation-error-after-update-to-0-90-10-td4047479.html the
problem may be caused by old jar files that are not aware of a stricter
process flow lucene expects to be followed.

The thread suggests looking for old jars related to lucene - but the only
ones I find are in /usr/share/elasticsearch/lib/, and they are all version
4.10.2 - which is after when the thread says the fix was applied.

This problems affects me on elastic search version 1.3 and 1.4. I have
tried reinstalling elasticsearch and the problem still happens.

Has anyone else had this problem?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a22f3427-f6ac-4965-a27a-4879d3e6c851%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hmm, not good. What does your "autocomplete" analyzer look like? Can you
post the full stack trace?

Mike McCandless

http://blog.mikemccandless.com

On Wed, Nov 5, 2014 at 7:05 PM, Richard Tier rikatee@gmail.com wrote:

An internal error happens when I do a "suggest" query. I get "TokenStream
contract violation: close() call missing"

This only happens when I add pre_filter to the suggest body.

    {
        'text': 'my term',
        'suggest' : {
            'phrase' : {
                'field' : 'title',
                'direct_generator' : [
                    {
                        'field' : 'title.suggest',
                        'suggest_mode': 'always',
                        'pre_filter': 'autocomplete' // <-- this

triggers the problem
}
]
}
}
}

However, I need pre_filter to achieve the desired outcome.

According the this thread
http://elasticsearch-users.115913.n3.nabble.com/TokenStream-contract-violation-error-after-update-to-0-90-10-td4047479.html the
problem may be caused by old jar files that are not aware of a stricter
process flow lucene expects to be followed.

The thread suggests looking for old jars related to lucene - but the only
ones I find are in /usr/share/elasticsearch/lib/, and they are all version
4.10.2 - which is after when the thread says the fix was applied.

This problems affects me on elastic search version 1.3 and 1.4. I have
tried reinstalling elasticsearch and the problem still happens.

Has anyone else had this problem?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a22f3427-f6ac-4965-a27a-4879d3e6c851%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a22f3427-f6ac-4965-a27a-4879d3e6c851%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAD7smRc81rhAY5i%3D2CXgF-_mw7kp-OW4DHSapjU59e9bND5Wpg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks for reply.

The autocomplete analyzer:

{
    'analysis': {
      'analyzer': {
        'autocomplete': {
           'type': 'custom',
           'tokenizer': 'standard',
           'filter': [
              'filter_pair_shingle',
              'lowercase',
              'asciifolding',
           ]
        }
      },
      'filter': {
          'filter_pair_shingle':{
              'type': 'shingle'
          }
    }

stacktrace is not very informative really:

{u'_shards': {u'successful': 0, u'failed': 5, u'total': 5, u'failures':
[{u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][0] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 0}, {u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][1] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 1}, {u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][2] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 2}, {u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][3] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 3}, {u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][4] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 4}]}}

On Thursday, November 6, 2014 11:29:22 AM UTC, Michael McCandless wrote:

Hmm, not good. What does your "autocomplete" analyzer look like? Can you
post the full stack trace?

Mike McCandless

http://blog.mikemccandless.com

On Wed, Nov 5, 2014 at 7:05 PM, Richard Tier <rik...@gmail.com
<javascript:>> wrote:

An internal error happens when I do a "suggest" query. I get "TokenStream
contract violation: close() call missing"

This only happens when I add pre_filter to the suggest body.

    {
        'text': 'my term',
        'suggest' : {
            'phrase' : {
                'field' : 'title',
                'direct_generator' : [
                    {
                        'field' : 'title.suggest',
                        'suggest_mode': 'always',
                        'pre_filter': 'autocomplete' // <-- this 

triggers the problem
}
]
}
}
}

However, I need pre_filter to achieve the desired outcome.

According the this thread
http://elasticsearch-users.115913.n3.nabble.com/TokenStream-contract-violation-error-after-update-to-0-90-10-td4047479.html the
problem may be caused by old jar files that are not aware of a stricter
process flow lucene expects to be followed.

The thread suggests looking for old jars related to lucene - but the only
ones I find are in /usr/share/elasticsearch/lib/, and they are all version
4.10.2 - which is after when the thread says the fix was applied.

This problems affects me on elastic search version 1.3 and 1.4. I have
tried reinstalling elasticsearch and the problem still happens.

Has anyone else had this problem?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a22f3427-f6ac-4965-a27a-4879d3e6c851%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a22f3427-f6ac-4965-a27a-4879d3e6c851%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c8e027ec-1126-4ad2-9620-c2a3f01f2a54%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

On Thu, Nov 6, 2014 at 3:05 PM, Richard Tier rikatee@gmail.com wrote:

Thanks for reply.

The autocomplete analyzer:

{
    'analysis': {
      'analyzer': {
        'autocomplete': {
           'type': 'custom',
           'tokenizer': 'standard',
           'filter': [
              'filter_pair_shingle',
              'lowercase',
              'asciifolding',
           ]
        }
      },
      'filter': {
          'filter_pair_shingle':{
              'type': 'shingle'
          }
    }

stacktrace is not very informative really:

Hmm indeed this is not very helpful, but I strongly suspect this is a bug
inside Elasticsearch in how it interacts with Lucene's Analyzer/TokenStream
API. Could you open an issue on github? Thanks.

{u'_shards': {u'successful': 0, u'failed': 5, u'total': 5, u'failures':
[{u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][0] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 0}, {u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][1] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 1}, {u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][2] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 2}, {u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][3] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 3}, {u'index': u'test_market', u'reason':
u'BroadcastShardOperationFailedException[[test_market][4] ]; nested:
ElasticsearchException[failed to execute suggest]; nested:
IllegalStateException[TokenStream contract violation: close() call
missing]; ', u'shard': 4}]}}

Mike McCandless

http://blog.mikemccandless.com

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAD7smRf0V0rRUU9_KWTcK07K8kFfGtC9wdib2JWzP4qBDwS1Gw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.