Search terms and match_all not working

Elastic Stack Elasticsearch
1

I am making a logging system in elasticsearch. I have different types of
logs with different hosts. Because I am using different type's for the
different type of logs its easy. When i try to get logs of just one host
with term or terms it wont return anything, I think it has something to do
with my mapping (default), but when I try to use match_all, it returns
things but its the same returns all hosts, so if i put a hostname for one
node it will just return the other nodes regardless. I am trying to use
filters as much as I can to make sure things are in cache.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Can you gist an example of your docs, your queries and mapping, if you set one?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 2 févr. 2013 à 22:31, Wojons Tech wojonstech@gmail.com a écrit :

I am making a logging system in elasticsearch. I have different types of logs with different hosts. Because I am using different type's for the different type of logs its easy. When i try to get logs of just one host with term or terms it wont return anything, I think it has something to do with my mapping (default), but when I try to use match_all, it returns things but its the same returns all hosts, so if i put a hostname for one node it will just return the other nodes regardless. I am trying to use filters as much as I can to make sure things are in cache.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

i think gist is broken but here is a link to a google doc

On Sat, Feb 2, 2013 at 7:23 PM, David Pilato david@pilato.fr wrote:

Can you gist an example of your docs, your queries and mapping, if you set
one?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 2 févr. 2013 à 22:31, Wojons Tech wojonstech@gmail.com a écrit :

I am making a logging system in elasticsearch. I have different types of
logs with different hosts. Because I am using different type's for the
different type of logs its easy. When i try to get logs of just one host
with term or terms it wont return anything, I think it has something to do
with my mapping (default), but when I try to use match_all, it returns
things but its the same returns all hosts, so if i put a hostname for one
node it will just return the other nodes regardless. I am trying to use
filters as much as I can to make sure things are in cache.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Search with the lowercase value of "YdmAojXGSKGOqMBjvVRAXA"

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 3 févr. 2013 à 07:35, Wojons Tech wojonstech@gmail.com a écrit :

i think gist is broken but here is a link to a google doc

On Sat, Feb 2, 2013 at 7:23 PM, David Pilato david@pilato.fr wrote:

Can you gist an example of your docs, your queries and mapping, if you set one?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 2 févr. 2013 à 22:31, Wojons Tech wojonstech@gmail.com a écrit :

I am making a logging system in elasticsearch. I have different types of logs with different hosts. Because I am using different type's for the different type of logs its easy. When i try to get logs of just one host with term or terms it wont return anything, I think it has something to do with my mapping (default), but when I try to use match_all, it returns things but its the same returns all hosts, so if i put a hostname for one node it will just return the other nodes regardless. I am trying to use filters as much as I can to make sure things are in cache.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

That seems to work, that is kinda of a bummer because that node id is the
one generated by elasticsearch, so thats the _id of another document. so
there is always a chance of two documents have the same characters but
different case. anyway to fix this?

On Sat, Feb 2, 2013 at 11:16 PM, David Pilato david@pilato.fr wrote:

Search with the lowercase value of "YdmAojXGSKGOqMBjvVRAXA"

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 3 févr. 2013 à 07:35, Wojons Tech wojonstech@gmail.com a écrit :

i think gist is broken but here is a link to a google doc

es bug - Google Docs

On Sat, Feb 2, 2013 at 7:23 PM, David Pilato david@pilato.fr wrote:

Can you gist an example of your docs, your queries and mapping, if you
set one?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 2 févr. 2013 à 22:31, Wojons Tech wojonstech@gmail.com a écrit :

I am making a logging system in elasticsearch. I have different types of
logs with different hosts. Because I am using different type's for the
different type of logs its easy. When i try to get logs of just one host
with term or terms it wont return anything, I think it has something to do
with my mapping (default), but when I try to use match_all, it returns
things but its the same returns all hosts, so if i put a hostname for one
node it will just return the other nodes regardless. I am trying to use
filters as much as I can to make sure things are in cache.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

You can change the mapping and set not_analyzed for this kind of field.

That way, ES will index it without breaking in into tokens, lowercase it and ignore common english words (standard analyzer).

Or, you can search with matchQuery (analyzed) instead of termQuery (not analyzed) but for your use case, I recommend the first method.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 3 févr. 2013 à 08:19, Wojons Tech wojonstech@gmail.com a écrit :

That seems to work, that is kinda of a bummer because that node id is the one generated by elasticsearch, so thats the _id of another document. so there is always a chance of two documents have the same characters but different case. anyway to fix this?

On Sat, Feb 2, 2013 at 11:16 PM, David Pilato david@pilato.fr wrote:

Search with the lowercase value of "YdmAojXGSKGOqMBjvVRAXA"

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 3 févr. 2013 à 07:35, Wojons Tech wojonstech@gmail.com a écrit :

i think gist is broken but here is a link to a google doc

es bug - Google Docs

On Sat, Feb 2, 2013 at 7:23 PM, David Pilato david@pilato.fr wrote:

Can you gist an example of your docs, your queries and mapping, if you set one?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 2 févr. 2013 à 22:31, Wojons Tech wojonstech@gmail.com a écrit :

I am making a logging system in elasticsearch. I have different types of logs with different hosts. Because I am using different type's for the different type of logs its easy. When i try to get logs of just one host with term or terms it wont return anything, I think it has something to do with my mapping (default), but when I try to use match_all, it returns things but its the same returns all hosts, so if i put a hostname for one node it will just return the other nodes regardless. I am trying to use filters as much as I can to make sure things are in cache.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

How do i change the mapping and are there other things i should be changing
the mapping for, I really have not been able to understand what mapping is
in es can you explain it to me.

On Sat, Feb 2, 2013 at 11:33 PM, David Pilato david@pilato.fr wrote:

You can change the mapping and set not_analyzed for this kind of field.

That way, ES will index it without breaking in into tokens, lowercase it
and ignore common english words (standard analyzer).

Or, you can search with matchQuery (analyzed) instead of termQuery (not
analyzed) but for your use case, I recommend the first method.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 3 févr. 2013 à 08:19, Wojons Tech wojonstech@gmail.com a écrit :

That seems to work, that is kinda of a bummer because that node id is the
one generated by elasticsearch, so thats the _id of another document. so
there is always a chance of two documents have the same characters but
different case. anyway to fix this?

On Sat, Feb 2, 2013 at 11:16 PM, David Pilato david@pilato.fr wrote:

Search with the lowercase value of "YdmAojXGSKGOqMBjvVRAXA"

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 3 févr. 2013 à 07:35, Wojons Tech wojonstech@gmail.com a écrit :

i think gist is broken but here is a link to a google doc

es bug - Google Docs

On Sat, Feb 2, 2013 at 7:23 PM, David Pilato david@pilato.fr wrote:

Can you gist an example of your docs, your queries and mapping, if you
set one?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 2 févr. 2013 à 22:31, Wojons Tech wojonstech@gmail.com a écrit :

I am making a logging system in elasticsearch. I have different types of
logs with different hosts. Because I am using different type's for the
different type of logs its easy. When i try to get logs of just one host
with term or terms it wont return anything, I think it has something to do
with my mapping (default), but when I try to use match_all, it returns
things but its the same returns all hosts, so if i put a hostname for one
node it will just return the other nodes regardless. I am trying to use
filters as much as I can to make sure things are in cache.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

Do something like:
$ curl -XPUT 'http://localhost:9200/twitter/tweet/_mapping' -d '
{
"tweet" : {
"properties" : {
"message" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
'
You have to define it before indexing the first document.

HTH

Le 3 févr. 2013 à 08:36, Wojons Tech wojonstech@gmail.com a écrit :

How do i change the mapping and are there other things i should be changing the mapping for, I really have not been able to understand what mapping is in es can you explain it to me.

On Sat, Feb 2, 2013 at 11:33 PM, David Pilato david@pilato.fr wrote:
You can change the mapping and set not_analyzed for this kind of field.

That way, ES will index it without breaking in into tokens, lowercase it and ignore common english words (standard analyzer).

Or, you can search with matchQuery (analyzed) instead of termQuery (not analyzed) but for your use case, I recommend the first method.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 3 févr. 2013 à 08:19, Wojons Tech wojonstech@gmail.com a écrit :

That seems to work, that is kinda of a bummer because that node id is the one generated by elasticsearch, so thats the _id of another document. so there is always a chance of two documents have the same characters but different case. anyway to fix this?

On Sat, Feb 2, 2013 at 11:16 PM, David Pilato david@pilato.fr wrote:
Search with the lowercase value of "YdmAojXGSKGOqMBjvVRAXA"

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 3 févr. 2013 à 07:35, Wojons Tech wojonstech@gmail.com a écrit :

i think gist is broken but here is a link to a google doc

https://docs.google.com/document/d/12b5NG-ZB_Pm1P-v4qrG4qzRBsJvKNRMctGbjoSY9Dfo/edit?usp=sharing

On Sat, Feb 2, 2013 at 7:23 PM, David Pilato david@pilato.fr wrote:
Can you gist an example of your docs, your queries and mapping, if you set one?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 2 févr. 2013 à 22:31, Wojons Tech wojonstech@gmail.com a écrit :

I am making a logging system in elasticsearch. I have different types of logs with different hosts. Because I am using different type's for the different type of logs its easy. When i try to get logs of just one host with term or terms it wont return anything, I think it has something to do with my mapping (default), but when I try to use match_all, it returns things but its the same returns all hosts, so if i put a hostname for one node it will just return the other nodes regardless. I am trying to use filters as much as I can to make sure things are in cache.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

The field mapping defines what Lucene analyzers and fields attributes
should be assigned to a field in your source for a document you index.

I you want to disable mapping completey and index your data analyzed by
keyword, you can create an index with

curl -XPUT localhost:9200/test -d '{
"index" : {
"analysis" : {
"analyzer" : {
"default" : {
"type" : "keyword"
}
}
}
}
}'

Best regards,

Jörg

Am 03.02.13 08:36, schrieb Wojons Tech:

How do i change the mapping and are there other things i should be
changing the mapping for, I really have not been able to understand
what mapping is in es can you explain it to me.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.