Match query against Stored & Unindexed Field (Not storing source)


(Aaron Troy) #1

Hi all,

I'm trying to build a side-set of data for some people by querying the
'sourceName' field of an index. This field (as you can see from the
mapping) is stored and not indexed. I'm also NOT storing the source:

  • mappings: {
    • doc: {
      • _source: {
        • enabled: false
          }
      • properties: {
        • sourceTypeName: {
          • index: no
          • type: string
            }
        • pubDate: {
          • store: true
          • format: dd MMM yyyy HH:mm:ss
          • type: date
            }
        • text: {
          • store: true
          • analyzer: standard
          • term_vector: with_positions_offsets
          • type: string
            }
        • coverageId: {
          • index: no
          • type: string
            }
        • coverageName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • sourceTypeId: {
          • index: no
          • type: string
            }
        • locationId: {
          • index: no
          • type: string
            }
        • nativeName: {
          • index: no
          • type: string
            }
        • routingId: {
          • type: string
            }
        • locationName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • url: {
          • index: no
          • store: true
          • type: string
            }
        • articleId: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • capDate: {
          • index: no
          • type: string
            }
        • languageName: {
          • index: no
          • type: string
            }
        • robotName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • title: {
          • store: true
          • analyzer: standard
          • term_vector: with_positions_offsets
          • type: string
            }
        • languageCode: {
          • index: no
          • type: string
            }
        • scopeType: {
          • index: no
          • type: string
            }
        • sourceName: {
          • index: no
          • store: true
          • type: string
            }
        • encoding: {
          • index: no
          • type: string
            }
        • trTitle: {
          • store: true
          • term_vector: with_positions_offsets
          • type: string
            }
        • scopeId: {
          • index: no
          • type: string
            }
            }
            }
            }

I've tried running both a match query and a query string against the field
with data that I know exists and I continuously get 0 results:

Data that exists in ES index:

  • fields: {
    • sourceName: [
      • The Daily Telegraph
        ]
        }

My match query that returns NO results:

{
"query" : {
"match" : {
"sourceName" : "The Daily Telegraph"
}
}
}

My query string that returns NO results:

{
"query" : {
"query_string" : {
"default_field" : "sourceName",
"query" : "The Daily Telegraph"
}
}
}

My questions are:

  1. Can I even query against the field if it is stored, not indexed, and
    there's no source?
  2. If I can, why am I getting no results if I know the data exists?

Thanks in advance for any assistance!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

Settings a field to "index: no" means you do not want to search on the
field.

A field with "store: yes" means the original content of the field values is
stored in the Lucene data structure. It does not enable search per se.

Jörg

On Wed, May 7, 2014 at 6:18 PM, Aaron Troy atoy3731@gmail.com wrote:

Hi all,

I'm trying to build a side-set of data for some people by querying the
'sourceName' field of an index. This field (as you can see from the
mapping) is stored and not indexed. I'm also NOT storing the source:

  • mappings: {
    • doc: {
      • _source: {
        • enabled: false
          }
      • properties: {
        • sourceTypeName: {
          • index: no
          • type: string
            }
        • pubDate: {
          • store: true
          • format: dd MMM yyyy HH:mm:ss
          • type: date
            }
        • text: {
          • store: true
          • analyzer: standard
          • term_vector: with_positions_offsets
          • type: string
            }
        • coverageId: {
          • index: no
          • type: string
            }
        • coverageName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • sourceTypeId: {
          • index: no
          • type: string
            }
        • locationId: {
          • index: no
          • type: string
            }
        • nativeName: {
          • index: no
          • type: string
            }
        • routingId: {
          • type: string
            }
        • locationName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • url: {
          • index: no
          • store: true
          • type: string
            }
        • articleId: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • capDate: {
          • index: no
          • type: string
            }
        • languageName: {
          • index: no
          • type: string
            }
        • robotName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • title: {
          • store: true
          • analyzer: standard
          • term_vector: with_positions_offsets
          • type: string
            }
        • languageCode: {
          • index: no
          • type: string
            }
        • scopeType: {
          • index: no
          • type: string
            }
        • sourceName: {
          • index: no
          • store: true
          • type: string
            }
        • encoding: {
          • index: no
          • type: string
            }
        • trTitle: {
          • store: true
          • term_vector: with_positions_offsets
          • type: string
            }
        • scopeId: {
          • index: no
          • type: string
            }
            }
            }
            }

I've tried running both a match query and a query string against the field
with data that I know exists and I continuously get 0 results:

Data that exists in ES index:

  • fields: {
    • sourceName: [
      • The Daily Telegraph
        ]
        }

My match query that returns NO results:

{
"query" : {
"match" : {
"sourceName" : "The Daily Telegraph"
}
}
}

My query string that returns NO results:

{
"query" : {
"query_string" : {
"default_field" : "sourceName",
"query" : "The Daily Telegraph"
}
}
}

My questions are:

  1. Can I even query against the field if it is stored, not indexed, and
    there's no source?
  2. If I can, why am I getting no results if I know the data exists?

Thanks in advance for any assistance!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG-YWbnz2nz0F5ASvcTUutuQ0yt1UJWXDHN0WzFhsVZKQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Aaron Troy) #3

Drat, that's what I was afraid of. So realistically, my only option is to
retrieve the entire dataset and programmatically filter results?

On Wednesday, May 7, 2014 12:34:25 PM UTC-4, Jörg Prante wrote:

Settings a field to "index: no" means you do not want to search on the
field.

A field with "store: yes" means the original content of the field values
is stored in the Lucene data structure. It does not enable search per se.

Jörg

On Wed, May 7, 2014 at 6:18 PM, Aaron Troy <atoy...@gmail.com<javascript:>

wrote:

Hi all,

I'm trying to build a side-set of data for some people by querying the
'sourceName' field of an index. This field (as you can see from the
mapping) is stored and not indexed. I'm also NOT storing the source:

  • mappings: {
    • doc: {
      • _source: {
        • enabled: false
          }
      • properties: {
        • sourceTypeName: {
          • index: no
          • type: string
            }
        • pubDate: {
          • store: true
          • format: dd MMM yyyy HH:mm:ss
          • type: date
            }
        • text: {
          • store: true
          • analyzer: standard
          • term_vector: with_positions_offsets
          • type: string
            }
        • coverageId: {
          • index: no
          • type: string
            }
        • coverageName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • sourceTypeId: {
          • index: no
          • type: string
            }
        • locationId: {
          • index: no
          • type: string
            }
        • nativeName: {
          • index: no
          • type: string
            }
        • routingId: {
          • type: string
            }
        • locationName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • url: {
          • index: no
          • store: true
          • type: string
            }
        • articleId: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • capDate: {
          • index: no
          • type: string
            }
        • languageName: {
          • index: no
          • type: string
            }
        • robotName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • title: {
          • store: true
          • analyzer: standard
          • term_vector: with_positions_offsets
          • type: string
            }
        • languageCode: {
          • index: no
          • type: string
            }
        • scopeType: {
          • index: no
          • type: string
            }
        • sourceName: {
          • index: no
          • store: true
          • type: string
            }
        • encoding: {
          • index: no
          • type: string
            }
        • trTitle: {
          • store: true
          • term_vector: with_positions_offsets
          • type: string
            }
        • scopeId: {
          • index: no
          • type: string
            }
            }
            }
            }

I've tried running both a match query and a query string against the
field with data that I know exists and I continuously get 0 results:

Data that exists in ES index:

  • fields: {
    • sourceName: [
      • The Daily Telegraph
        ]
        }

My match query that returns NO results:

{
"query" : {
"match" : {
"sourceName" : "The Daily Telegraph"
}
}
}

My query string that returns NO results:

{
"query" : {
"query_string" : {

        "default_field" : "sourceName",

        "query" : "The Daily Telegraph"

    }
}

}

My questions are:

  1. Can I even query against the field if it is stored, not indexed, and
    there's no source?
  2. If I can, why am I getting no results if I know the data exists?

Thanks in advance for any assistance!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0b195441-72d8-4c2b-9f47-9a54a9016d75%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Nik Everett) #4

You can write a script filter to do it I believe but it won't be quick.

Nik

On Wed, May 7, 2014 at 12:37 PM, Aaron Troy atoy3731@gmail.com wrote:

Drat, that's what I was afraid of. So realistically, my only option is to
retrieve the entire dataset and programmatically filter results?

On Wednesday, May 7, 2014 12:34:25 PM UTC-4, Jörg Prante wrote:

Settings a field to "index: no" means you do not want to search on the
field.

A field with "store: yes" means the original content of the field values
is stored in the Lucene data structure. It does not enable search per se.

Jörg

On Wed, May 7, 2014 at 6:18 PM, Aaron Troy atoy...@gmail.com wrote:

Hi all,

I'm trying to build a side-set of data for some people by querying the
'sourceName' field of an index. This field (as you can see from the
mapping) is stored and not indexed. I'm also NOT storing the source:

  • mappings: {
    • doc: {
      • _source: {
        • enabled: false
          }
      • properties: {
        • sourceTypeName: {
          • index: no
          • type: string
            }
        • pubDate: {
          • store: true
          • format: dd MMM yyyy HH:mm:ss
          • type: date
            }
        • text: {
          • store: true
          • analyzer: standard
          • term_vector: with_positions_offsets
          • type: string
            }
        • coverageId: {
          • index: no
          • type: string
            }
        • coverageName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • sourceTypeId: {
          • index: no
          • type: string
            }
        • locationId: {
          • index: no
          • type: string
            }
        • nativeName: {
          • index: no
          • type: string
            }
        • routingId: {
          • type: string
            }
        • locationName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • url: {
          • index: no
          • store: true
          • type: string
            }
        • articleId: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • capDate: {
          • index: no
          • type: string
            }
        • languageName: {
          • index: no
          • type: string
            }
        • robotName: {
          • index: not_analyzed
          • store: true
          • type: string
            }
        • title: {
          • store: true
          • analyzer: standard
          • term_vector: with_positions_offsets
          • type: string
            }
        • languageCode: {
          • index: no
          • type: string
            }
        • scopeType: {
          • index: no
          • type: string
            }
        • sourceName: {
          • index: no
          • store: true
          • type: string
            }
        • encoding: {
          • index: no
          • type: string
            }
        • trTitle: {
          • store: true
          • term_vector: with_positions_offsets
          • type: string
            }
        • scopeId: {
          • index: no
          • type: string
            }
            }
            }
            }

I've tried running both a match query and a query string against the
field with data that I know exists and I continuously get 0 results:

Data that exists in ES index:

  • fields: {
    • sourceName: [
      • The Daily Telegraph
        ]
        }

My match query that returns NO results:

{
"query" : {
"match" : {
"sourceName" : "The Daily Telegraph"
}
}
}

My query string that returns NO results:

{
"query" : {
"query_string" : {

        "default_field" : "sourceName",


        "query" : "The Daily Telegraph"


    }
}

}

My questions are:

  1. Can I even query against the field if it is stored, not indexed, and
    there's no source?
  2. If I can, why am I getting no results if I know the data exists?

Thanks in advance for any assistance!

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/9eacf608-cf1e-46de-a82f-3be1de057a38%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0b195441-72d8-4c2b-9f47-9a54a9016d75%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/0b195441-72d8-4c2b-9f47-9a54a9016d75%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2rryyiYtv%2B7Q5AtLaNtULhMO9TbLAR4Mop4-N%2B6jH%3DWA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #5