Selective Field Importing With River


(mblodnick) #1

I am currently using ElasticSearch to search through data stored in a mongo db. However I have come across an issue that I seem to be stuck on. I am using a river to build indices from my database, I want to exclude some data from being stored into the JSON document. I have some security sensitive information that I would prefer not to have on my server.

In fact, it would be more beneficial if I could just list the data fields that I want to be included and stored into the JSON document while still using a mongo db river-plugin. Please share any guidance you have on this problem.

Any advice is appreciated?


(David Pilato) #2

May be script filters is the way to go? See https://github.com/richardwilly98/elasticsearch-river-mongodb/wiki#script-filters

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 7 juin 2013 à 22:05, mblodnick mblodnick@gmail.com a écrit :

I am currently using ElasticSearch to search through data stored in a mongo
db. However I have come across an issue that I seem to be stuck on. I am
using a river to build indices from my database, I want to exclude some data
from being stored into the JSON document. I have some security sensitive
information that I would prefer not to have on my server.

In fact, it would be more beneficial if I could just list the data fields
that I want to be included and stored into the JSON document while still
using a mongo db river-plugin. Please share any guidance you have on this
problem.

Any advice is appreciated?

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Selective-Field-Importing-With-River-tp4036229.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Sahana) #3

I have been trying to combine a few conditions using logical operators in
the script filter but, it doesn't seem to work

{
"index": {
"name": "myindex",
"type": "mytype"
},
"_enabled": true,
"_name": "myindex",
"mongodb": {
"db": "mydb",
"servers": [
{
"port": 27017,
"host": "127.0.0.1"
}
],
"collection": "mycollection",
"script": "ctx.deleted = (ctx.document.trash == 1 || ctx.document.private == 1)"
},
"type": "mongodb"
}

I also tried -

"script": "if( ctx.document.trash == 1 || ctx.document.private == 1 ) { ctx.deleted = true; }"

but, it indexes even when one of the flags(trash/private) is 0. it doesn't
seem to be taking the OR into consideration at all.

Is there something wrong with the above?

Note - Also tried defining a mongodb filter on the oplogs -

"filter": "{"o.private": 0, "o.trash":0}". But, this resulted in a parser
exception.

On Friday, June 14, 2013 9:27:35 PM UTC+5:30, David Pilato wrote:

May be script filters is the way to go? See
https://github.com/richardwilly98/elasticsearch-river-mongodb/wiki#script-filters

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 7 juin 2013 à 22:05, mblodnick <mblo...@gmail.com <javascript:>> a
écrit :

I am currently using ElasticSearch to search through data stored in a mongo
db. However I have come across an issue that I seem to be stuck on. I am
using a river to build indices from my database, I want to exclude some
data
from being stored into the JSON document. I have some security sensitive
information that I would prefer not to have on my server.

In fact, it would be more beneficial if I could just list the data fields
that I want to be included and stored into the JSON document while still
using a mongo db river-plugin. Please share any guidance you have on this
problem.

Any advice is appreciated?

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Selective-Field-Importing-With-River-tp4036229.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #4