Hide some system fields

11152 · May 29, 2014, 11:11am

Hi all!
I use elasticsearch in a high-load project where an urgent need to save
traffic.
I have a some queries like this:

curl -XGET 'http://localhost:9200/testindex/testmapping/_search?pretty&scroll=5m' -d '{"from":0, "size":1000}'

{
"_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7MjszMjp6TmdjNmxkM1NtV1NOeTl5X3dab1FnOzMxOnpOZ2M2bGQzU21XU055OXlfd1pvUWc7MDs=",
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"hits" : {
"total" : 15457332,
"max_score" : 1.0,
"hits" : [ {
"_index" : "testindex",
"_type" : "testmapping",
"_id" : "mo7vQrWUTquBRowjq2AVkw",
"_score" : 1.0, "_source" : {"reffer_id":"","date":"2013-05-31T00:00:00","source":5,"user_id":"2fdfdf0fbbce603cf24c0eee7dabf28c"}
}, ....]
}
}

Can I exclude some system fields (like _shards.*, hits._index, hits._type,
hits._id, hits._score)? I found how exclude source fields, but not system.
Also I need to get _timestamp field in _source rows. It generated from
'date' field:

'_timestamp' => array(
'enabled' => true,
'path' => 'date',
'format' => "YYYY-MM-dd'T'HH:mm:ss"
)

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cbc201ec-42b1-47a6-9186-2d240ae669c3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Florentin_Zorca · May 29, 2014, 12:29pm

Hi,

try using fields to specify only the fields you are interested in:

Your query should then be something like this:

{"fields":["date"], "from":0, "size":1000}

Kind Regards,

Florentin

Am Donnerstag, 29. Mai 2014 13:11:07 UTC+2 schrieb Сергей Шилов:

Hi all!
I use elasticsearch in a high-load project where an urgent need to save
traffic.
I have a some queries like this:

curl -XGET 'http://localhost:9200/testindex/testmapping/_search?pretty&scroll=5m' -d '{"from":0, "size":1000}'

{
"_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7MjszMjp6TmdjNmxkM1NtV1NOeTl5X3dab1FnOzMxOnpOZ2M2bGQzU21XU055OXlfd1pvUWc7MDs=",
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"hits" : {
"total" : 15457332,
"max_score" : 1.0,
"hits" : [ {
"_index" : "testindex",
"_type" : "testmapping",
"_id" : "mo7vQrWUTquBRowjq2AVkw",
"_score" : 1.0, "_source" : {"reffer_id":"","date":"2013-05-31T00:00:00","source":5,"user_id":"2fdfdf0fbbce603cf24c0eee7dabf28c"}
}, ....]
}
}

Can I exclude some system fields (like _shards.*, hits._index, hits._type,
hits._id, hits._score)? I found how exclude source fields, but not system.
Also I need to get _timestamp field in _source rows. It generated from
'date' field:

'_timestamp' => array(
'enabled' => true,
'path' => 'date',
'format' => "YYYY-MM-dd'T'HH:mm:ss"
)

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/16245f90-64fc-4a39-adf4-71663e91d950%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ivan · May 29, 2014, 9:25pm

I do not think you can exclude the search metadata fields.

--
Ivan

On Thu, May 29, 2014 at 5:29 AM, Florentin Zorca google@zorca.de wrote:

Hi,

try using fields to specify only the fields you are interested in:

Elasticsearch Platform — Find real-time answers at scale | Elastic

Your query should then be something like this:

{"fields":["date"], "from":0, "size":1000}

Kind Regards,

Florentin

Am Donnerstag, 29. Mai 2014 13:11:07 UTC+2 schrieb Сергей Шилов:

Hi all!
I use elasticsearch in a high-load project where an urgent need to save
traffic.
I have a some queries like this:

curl -XGET 'http://localhost:9200/testindex/testmapping/_search?pretty&scroll=5m' -d '{"from":0, "size":1000}'

{
"_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7MjszMjp6TmdjNmxkM1NtV1NOeTl5X3dab1FnOzMxOnpOZ2M2bGQzU21XU055OXlfd1pvUWc7MDs=",
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"hits" : {
"total" : 15457332,
"max_score" : 1.0,
"hits" : [ {
"_index" : "testindex",
"_type" : "testmapping",
"_id" : "mo7vQrWUTquBRowjq2AVkw",
"_score" : 1.0, "_source" : {"reffer_id":"","date":"2013-05-31T00:00:00","source":5,"user_id":"2fdfdf0fbbce603cf24c0eee7dabf28c"}
}, ....]
}
}

Can I exclude some system fields (like _shards.*, hits._index,
hits._type, hits._id, hits._score)? I found how exclude source fields, but
not system.
Also I need to get _timestamp field in _source rows. It generated from
'date' field:

'_timestamp' => array(
'enabled' => true,
'path' => 'date',
'format' => "YYYY-MM-dd'T'HH:mm:ss"
)

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/16245f90-64fc-4a39-adf4-71663e91d950%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/16245f90-64fc-4a39-adf4-71663e91d950%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA0NEzd30i9vrpLv1D5nZY%2BfX9%2BLEKPUesznXa%3D2LL7EQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

nik9000 · May 29, 2014, 9:29pm

Source filtering is similar but normally returns nicer results.

Sent from my iPhone

On May 29, 2014, at 8:29 AM, Florentin Zorca google@zorca.de wrote:

Hi,

try using fields to specify only the fields you are interested in:

Elasticsearch Platform — Find real-time answers at scale | Elastic

Your query should then be something like this:
{"fields":["date"], "from":0, "size":1000}
Kind Regards,
Florentin

Am Donnerstag, 29. Mai 2014 13:11:07 UTC+2 schrieb Сергей Шилов:

Hi all!
I use elasticsearch in a high-load project where an urgent need to save traffic.
I have a some queries like this:
curl -XGET 'http://localhost:9200/testindex/testmapping/_search?pretty&scroll=5m' -d '{"from":0, "size":1000}'

{
"_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7MjszMjp6TmdjNmxkM1NtV1NOeTl5X3dab1FnOzMxOnpOZ2M2bGQzU21XU055OXlfd1pvUWc7MDs=",
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"hits" : {
"total" : 15457332,
"max_score" : 1.0,
"hits" : [ {
"_index" : "testindex",
"_type" : "testmapping",
"_id" : "mo7vQrWUTquBRowjq2AVkw",
"_score" : 1.0, "_source" : {"reffer_id":"","date":"2013-05-31T00:00:00","source":5,"user_id":"2fdfdf0fbbce603cf24c0eee7dabf28c"}
}, ....]
}
}

Can I exclude some system fields (like _shards.*, hits._index, hits._type, hits._id, hits._score)? I found how exclude source fields, but not system.
Also I need to get _timestamp field in _source rows. It generated from 'date' field:
'_timestamp' => array(
'enabled' => true,
'path' => 'date',
'format' => "YYYY-MM-dd'T'HH:mm:ss"
)

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/16245f90-64fc-4a39-adf4-71663e91d950%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/B13EA515-40ED-4F09-97BA-799B23B98DF8%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Gail_Long · May 29, 2014, 10:34pm

You can use jq to selectively pull JSON values, arrays, and objects from
elasticsearch returns. It lets you filter out everything except the things
you want and formats it decently as well. The syntax takes a bit of
getting used to but once you get it the product makes working with JSON
output much nicer.

http://stedolan.github.io/jq/

On Thursday, May 29, 2014 5:11:07 AM UTC-6, Сергей Шилов wrote:

Hi all!
I use elasticsearch in a high-load project where an urgent need to save
traffic.
I have a some queries like this:

curl -XGET 'http://localhost:9200/testindex/testmapping/_search?pretty&scroll=5m' -d '{"from":0, "size":1000}'

{
"_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7MjszMjp6TmdjNmxkM1NtV1NOeTl5X3dab1FnOzMxOnpOZ2M2bGQzU21XU055OXlfd1pvUWc7MDs=",
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"hits" : {
"total" : 15457332,
"max_score" : 1.0,
"hits" : [ {
"_index" : "testindex",
"_type" : "testmapping",
"_id" : "mo7vQrWUTquBRowjq2AVkw",
"_score" : 1.0, "_source" : {"reffer_id":"","date":"2013-05-31T00:00:00","source":5,"user_id":"2fdfdf0fbbce603cf24c0eee7dabf28c"}
}, ....]
}
}

Can I exclude some system fields (like _shards.*, hits._index, hits._type,
hits._id, hits._score)? I found how exclude source fields, but not system.
Also I need to get _timestamp field in _source rows. It generated from
'date' field:

'_timestamp' => array(
'enabled' => true,
'path' => 'date',
'format' => "YYYY-MM-dd'T'HH:mm:ss"
)

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/73a11577-11c9-4931-a0cd-a55bf36b8739%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

11152 · May 30, 2014, 1:32pm

Thanks a lot!
i'll try this.
Also i need _timestamp field into output. I tried several ways, but to no
avail.

пятница, 30 мая 2014 г., 2:34:47 UTC+4 пользователь Gail Long написал:

You can use jq to selectively pull JSON values, arrays, and objects from
elasticsearch returns. It lets you filter out everything except the things
you want and formats it decently as well. The syntax takes a bit of
getting used to but once you get it the product makes working with JSON
output much nicer.

Redirecting to jqlang.github.io

On Thursday, May 29, 2014 5:11:07 AM UTC-6, Сергей Шилов wrote:

Hi all!
I use elasticsearch in a high-load project where an urgent need to save
traffic.
I have a some queries like this:

curl -XGET 'http://localhost:9200/testindex/testmapping/_search?pretty&scroll=5m' -d '{"from":0, "size":1000}'

{
"_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7MjszMjp6TmdjNmxkM1NtV1NOeTl5X3dab1FnOzMxOnpOZ2M2bGQzU21XU055OXlfd1pvUWc7MDs=",
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"hits" : {
"total" : 15457332,
"max_score" : 1.0,
"hits" : [ {
"_index" : "testindex",
"_type" : "testmapping",
"_id" : "mo7vQrWUTquBRowjq2AVkw",
"_score" : 1.0, "_source" : {"reffer_id":"","date":"2013-05-31T00:00:00","source":5,"user_id":"2fdfdf0fbbce603cf24c0eee7dabf28c"}
}, ....]
}
}

Can I exclude some system fields (like _shards.*, hits._index,
hits._type, hits._id, hits._score)? I found how exclude source fields, but
not system.
Also I need to get _timestamp field in _source rows. It generated from
'date' field:

'_timestamp' => array(
'enabled' => true,
'path' => 'date',
'format' => "YYYY-MM-dd'T'HH:mm:ss"
)

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a7dd7cb7-6f72-46df-9f55-18eab0a152bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.