Match query against Stored & Unindexed Field (Not storing source)

Aaron_Troy · May 7, 2014, 4:18pm

Hi all,

I'm trying to build a side-set of data for some people by querying the
'sourceName' field of an index. This field (as you can see from the
mapping) is stored and not indexed. I'm also NOT storing the source:

mappings: {
- doc: {
  - _source: {
    - enabled: false
      }
  - properties: {
    - sourceTypeName: {
      - index: no
      - type: string
        }
    - pubDate: {
      - store: true
      - format: dd MMM yyyy HH:mm:ss
      - type: date
        }
    - text: {
      - store: true
      - analyzer: standard
      - term_vector: with_positions_offsets
      - type: string
        }
    - coverageId: {
      - index: no
      - type: string
        }
    - coverageName: {
      - index: not_analyzed
      - store: true
      - type: string
        }
    - sourceTypeId: {
      - index: no
      - type: string
        }
    - locationId: {
      - index: no
      - type: string
        }
    - nativeName: {
      - index: no
      - type: string
        }
    - routingId: {
      - type: string
        }
    - locationName: {
      - index: not_analyzed
      - store: true
      - type: string
        }
    - url: {
      - index: no
      - store: true
      - type: string
        }
    - articleId: {
      - index: not_analyzed
      - store: true
      - type: string
        }
    - capDate: {
      - index: no
      - type: string
        }
    - languageName: {
      - index: no
      - type: string
        }
    - robotName: {
      - index: not_analyzed
      - store: true
      - type: string
        }
    - title: {
      - store: true
      - analyzer: standard
      - term_vector: with_positions_offsets
      - type: string
        }
    - languageCode: {
      - index: no
      - type: string
        }
    - scopeType: {
      - index: no
      - type: string
        }
    - sourceName: {
      - index: no
      - store: true
      - type: string
        }
    - encoding: {
      - index: no
      - type: string
        }
    - trTitle: {
      - store: true
      - term_vector: with_positions_offsets
      - type: string
        }
    - scopeId: {
      - index: no
      - type: string
        }
        }
        }
        }

I've tried running both a match query and a query string against the field
with data that I know exists and I continuously get 0 results:

Data that exists in ES index:

fields: {
- sourceName: [
  - The Daily Telegraph
    ]
    }

My match query that returns NO results:

{
"query" : {
"match" : {
"sourceName" : "The Daily Telegraph"
}
}
}

My query string that returns NO results:

{
"query" : {
"query_string" : {
"default_field" : "sourceName",
"query" : "The Daily Telegraph"
}
}
}

My questions are:

Can I even query against the field if it is stored, not indexed, and
there's no source?
If I can, why am I getting no results if I know the data exists?

Thanks in advance for any assistance!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jprante · May 7, 2014, 4:34pm

Settings a field to "index: no" means you do not want to search on the
field.

A field with "store: yes" means the original content of the field values is
stored in the Lucene data structure. It does not enable search per se.

Jörg

On Wed, May 7, 2014 at 6:18 PM, Aaron Troy atoy3731@gmail.com wrote:

Hi all,

I'm trying to build a side-set of data for some people by querying the
'sourceName' field of an index. This field (as you can see from the
mapping) is stored and not indexed. I'm also NOT storing the source:

mappings: {

doc: {

_source: {

enabled: false
}

properties: {

sourceTypeName: {

index: no

type: string
}

pubDate: {

store: true

format: dd MMM yyyy HH:mm:ss

type: date
}

text: {

store: true

analyzer: standard

term_vector: with_positions_offsets

type: string
}

coverageId: {

index: no

type: string
}

coverageName: {

index: not_analyzed

store: true

type: string
}

sourceTypeId: {

index: no

type: string
}

locationId: {

index: no

type: string
}

nativeName: {

index: no

type: string
}

routingId: {

type: string
}

locationName: {

index: not_analyzed

store: true

type: string
}

url: {

index: no

store: true

type: string
}

articleId: {

index: not_analyzed

store: true

type: string
}

capDate: {

index: no

type: string
}

languageName: {

index: no

type: string
}

robotName: {

index: not_analyzed

store: true

type: string
}

title: {

store: true

analyzer: standard

term_vector: with_positions_offsets

type: string
}

languageCode: {

index: no

type: string
}

scopeType: {

index: no

type: string
}

sourceName: {

index: no

store: true

type: string
}

encoding: {

index: no

type: string
}

trTitle: {

store: true

term_vector: with_positions_offsets

type: string
}

scopeId: {

index: no

type: string
}
}
}
}

I've tried running both a match query and a query string against the field
with data that I know exists and I continuously get 0 results:

Data that exists in ES index:

fields: {

sourceName: [

The Daily Telegraph
]
}

My match query that returns NO results:

{
"query" : {
"match" : {
"sourceName" : "The Daily Telegraph"
}
}
}

My query string that returns NO results:

{
"query" : {
"query_string" : {
"default_field" : "sourceName",
"query" : "The Daily Telegraph"
}
}
}

My questions are:

Can I even query against the field if it is stored, not indexed, and
there's no source?

If I can, why am I getting no results if I know the data exists?

Thanks in advance for any assistance!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG-YWbnz2nz0F5ASvcTUutuQ0yt1UJWXDHN0WzFhsVZKQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Aaron_Troy · May 7, 2014, 4:37pm

Drat, that's what I was afraid of. So realistically, my only option is to
retrieve the entire dataset and programmatically filter results?

On Wednesday, May 7, 2014 12:34:25 PM UTC-4, Jörg Prante wrote:

Settings a field to "index: no" means you do not want to search on the
field.

A field with "store: yes" means the original content of the field values
is stored in the Lucene data structure. It does not enable search per se.

Jörg

On Wed, May 7, 2014 at 6:18 PM, Aaron Troy <atoy...@gmail.com<javascript:>

wrote:
Hi all,

I'm trying to build a side-set of data for some people by querying the
'sourceName' field of an index. This field (as you can see from the
mapping) is stored and not indexed. I'm also NOT storing the source:

mappings: {

doc: {

_source: {

enabled: false
}

properties: {

sourceTypeName: {

index: no

type: string
}

pubDate: {

store: true

format: dd MMM yyyy HH:mm:ss

type: date
}

text: {

store: true

analyzer: standard

term_vector: with_positions_offsets

type: string
}

coverageId: {

index: no

type: string
}

coverageName: {

index: not_analyzed

store: true

type: string
}

sourceTypeId: {

index: no

type: string
}

locationId: {

index: no

type: string
}

nativeName: {

index: no

type: string
}

routingId: {

type: string
}

locationName: {

index: not_analyzed

store: true

type: string
}

url: {

index: no

store: true

type: string
}

articleId: {

index: not_analyzed

store: true

type: string
}

capDate: {

index: no

type: string
}

languageName: {

index: no

type: string
}

robotName: {

index: not_analyzed

store: true

type: string
}

title: {

store: true

analyzer: standard

term_vector: with_positions_offsets

type: string
}

languageCode: {

index: no

type: string
}

scopeType: {

index: no

type: string
}

sourceName: {

index: no

store: true

type: string
}

encoding: {

index: no

type: string
}

trTitle: {

store: true

term_vector: with_positions_offsets

type: string
}

scopeId: {

index: no

type: string
}
}
}
}

I've tried running both a match query and a query string against the
field with data that I know exists and I continuously get 0 results:

Data that exists in ES index:

fields: {

sourceName: [

The Daily Telegraph
]
}

My match query that returns NO results:

{
"query" : {
"match" : {
"sourceName" : "The Daily Telegraph"
}
}
}

My query string that returns NO results:

{
"query" : {
"query_string" : {
        "default_field" : "sourceName",

        "query" : "The Daily Telegraph"

    }
}
}

My questions are:

Can I even query against the field if it is stored, not indexed, and
there's no source?

If I can, why am I getting no results if I know the data exists?

Thanks in advance for any assistance!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0b195441-72d8-4c2b-9f47-9a54a9016d75%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

nik9000 · May 7, 2014, 4:51pm

You can write a script filter to do it I believe but it won't be quick.

Nik

On Wed, May 7, 2014 at 12:37 PM, Aaron Troy atoy3731@gmail.com wrote:

Drat, that's what I was afraid of. So realistically, my only option is to
retrieve the entire dataset and programmatically filter results?

On Wednesday, May 7, 2014 12:34:25 PM UTC-4, Jörg Prante wrote:
Settings a field to "index: no" means you do not want to search on the
field.

A field with "store: yes" means the original content of the field values
is stored in the Lucene data structure. It does not enable search per se.

Jörg

On Wed, May 7, 2014 at 6:18 PM, Aaron Troy atoy...@gmail.com wrote:
Hi all,

I'm trying to build a side-set of data for some people by querying the
'sourceName' field of an index. This field (as you can see from the
mapping) is stored and not indexed. I'm also NOT storing the source:

mappings: {

doc: {

_source: {

enabled: false
}

properties: {

sourceTypeName: {

index: no

type: string
}

pubDate: {

store: true

format: dd MMM yyyy HH:mm:ss

type: date
}

text: {

store: true

analyzer: standard

term_vector: with_positions_offsets

type: string
}

coverageId: {

index: no

type: string
}

coverageName: {

index: not_analyzed

store: true

type: string
}

sourceTypeId: {

index: no

type: string
}

locationId: {

index: no

type: string
}

nativeName: {

index: no

type: string
}

routingId: {

type: string
}

locationName: {

index: not_analyzed

store: true

type: string
}

url: {

index: no

store: true

type: string
}

articleId: {

index: not_analyzed

store: true

type: string
}

capDate: {

index: no

type: string
}

languageName: {

index: no

type: string
}

robotName: {

index: not_analyzed

store: true

type: string
}

title: {

store: true

analyzer: standard

term_vector: with_positions_offsets

type: string
}

languageCode: {

index: no

type: string
}

scopeType: {

index: no

type: string
}

sourceName: {

index: no

store: true

type: string
}

encoding: {

index: no

type: string
}

trTitle: {

store: true

term_vector: with_positions_offsets

type: string
}

scopeId: {

index: no

type: string
}
}
}
}

I've tried running both a match query and a query string against the
field with data that I know exists and I continuously get 0 results:

Data that exists in ES index:

fields: {

sourceName: [

The Daily Telegraph
]
}

My match query that returns NO results:

{
"query" : {
"match" : {
"sourceName" : "The Daily Telegraph"
}
}
}

My query string that returns NO results:

{
"query" : {
"query_string" : {
        "default_field" : "sourceName",


        "query" : "The Daily Telegraph"


    }
}
}

My questions are:

Can I even query against the field if it is stored, not indexed, and
there's no source?

If I can, why am I getting no results if I know the data exists?

Thanks in advance for any assistance!

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%
40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0b195441-72d8-4c2b-9f47-9a54a9016d75%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/0b195441-72d8-4c2b-9f47-9a54a9016d75%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2rryyiYtv%2B7Q5AtLaNtULhMO9TbLAR4Mop4-N%2B6jH%3DWA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.