ES for storing index alone

Imagine I have some data stored in mongoDB collection.
Example:
Collection name: Users
record1: {
"first":"Cyril",
"age":33,
"id":1234
}
record2: {
"first":"Jeff",
"age":23,
"id":3423
}

My idea is to index this mongo data in the Elastic.
Such that Elastic only stores the IDs but not the full JSON.

So when I query in Elastic for getting users above the age of 30 I must get
only 1234 and not the full JSON.
I will use this ID to search in mongoDB.
The idea is to store only the index info in Elastic while teh full data
may reside on some other DB.

Is this possible ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fa48fe10-e9da-4011-a58b-6db1522eb128%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You can disable _source field in mapping but note that won't be possible anymore in the future.
Also, IMO as Elasticsearch gives you in one single Network trip, all the information you need to display results to your users, you don't really need to send other requests to Mongo.

David

Le 6 avr. 2015 à 10:35, Cyril Cherian cyril@mobomo.com a écrit :

Imagine I have some data stored in mongoDB collection.
Example:
Collection name: Users
record1: {
"first":"Cyril",
"age":33,
"id":1234
}
record2: {
"first":"Jeff",
"age":23,
"id":3423
}

My idea is to index this mongo data in the Elastic.
Such that Elastic only stores the IDs but not the full JSON.

So when I query in Elastic for getting users above the age of 30 I must get only 1234 and not the full JSON.
I will use this ID to search in mongoDB.
The idea is to store only the index info in Elastic while teh full data may reside on some other DB.

Is this possible ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fa48fe10-e9da-4011-a58b-6db1522eb128%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/463B6851-84AD-4661-811C-590F67829E4A%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Thanks David,

I still have some doubts in teh approach you sent.
I am using Elastic version 1.1.1

  1. I created a mapping called tests
  2. Created a mapping
    curl -XPUT 'http://localhost:9200/tests/_mapping/test' -d '
    {
    "test" : {
    "properties" : {
    * "_source" : {"enabled" : false},*
    "message" : {"type" : "string" },
    "projectId": {"type": "string" },
    "userName": {"type": "string"}
    }
    }
    }
    '
  3. Inserted some values inthe index
    Example:
    curl -XPUT 'http://localhost:9200/tests/test/1' -d '{
    "message" : "Cyril here",
    "projectId": "1234",
    "userName": "rahul"
    }'

curl -XPUT 'http://localhost:9200/tests/test/1' -d '{
"message" : "a message Cyril here",
"projectId": 134,
"userName": "cyril"
}'

  1. Now when i do a search
    like http://localhost:9200/tests/test/_search?q="message"

I get _source (which i am not expecting)...what i want to achieve is that
elastic should just index the data but not store the data.

"hits": [ http://localhost:9200/tests/test/_search?q="message"#

  { <http://localhost:9200/tests/test/_search?q=%22message%22#>

    "_index": "tests",
    "_type": "test",
    "_id": "1",
    "_score": 0.11506981,
    "_source": { <http://localhost:9200/tests/test/_search?q=%22message%22#>

      "message": "a message Cyril here",
      "projectId": 134,
      "userName": "cyril"
    }

  }

]

Let me know if I am missing on something.

On Monday, April 6, 2015 at 2:05:15 PM UTC+5:30, Cyril Cherian wrote:

Imagine I have some data stored in mongoDB collection.
Example:
Collection name: Users
record1: {
"first":"Cyril",
"age":33,
"id":1234
}
record2: {
"first":"Jeff",
"age":23,
"id":3423
}

My idea is to index this mongo data in the Elastic.
Such that Elastic only stores the IDs but not the full JSON.

So when I query in Elastic for getting users above the age of 30 I must
get only 1234 and not the full JSON.
I will use this ID to search in mongoDB.
The idea is to store only the index info in Elastic while teh full data
may reside on some other DB.

Is this possible ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7648f826-549e-4991-8532-9cd6ceb6148d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

It should be inside « test » not inside « properties ».
See _source field | Elasticsearch Guide [8.11] | Elastic http://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html
{
"tweet" : {
"_source" : {"enabled" : false}
}
}

--
David Pilato - Developer | Evangelist

@dadoonet https://twitter.com/dadoonet | @elasticsearchfr https://twitter.com/elasticsearchfr | @scrutmydocs https://twitter.com/scrutmydocs

Le 6 avr. 2015 à 11:50, Cyril Cherian cyril@mobomo.com a écrit :

Thanks David,

I still have some doubts in teh approach you sent.
I am using Elastic version 1.1.1

  1. I created a mapping called tests
  2. Created a mapping
    curl -XPUT 'http://localhost:9200/tests/_mapping/test' -d '
    {
    "test" : {
    "properties" : {
    "_source" : {"enabled" : false},
    "message" : {"type" : "string" },
    "projectId": {"type": "string" },
    "userName": {"type": "string"}
    }
    }
    }
    '
  3. Inserted some values inthe index
    Example:
    curl -XPUT 'http://localhost:9200/tests/test/1' -d '{
    "message" : "Cyril here",
    "projectId": "1234",
    "userName": "rahul"
    }'

curl -XPUT 'http://localhost:9200/tests/test/1' -d '{
"message" : "a message Cyril here",
"projectId": 134,
"userName": "cyril"
}'

  1. Now when i do a search like http://localhost:9200/tests/test/_search?q="message"

I get _source (which i am not expecting)...what i want to achieve is that elastic should just index the data but not store the data.
"hits": [ http://localhost:9200/tests/test/_search?q="message"#
{ http://localhost:9200/tests/test/_search?q="message"#
"_index": "tests",
"_type": "test",
"_id": "1",
"_score": 0.11506981,
"_source": { http://localhost:9200/tests/test/_search?q="message"#
"message": "a message Cyril here",
"projectId": 134,
"userName": "cyril"
}
}
]

Let me know if I am missing on something.

On Monday, April 6, 2015 at 2:05:15 PM UTC+5:30, Cyril Cherian wrote:
Imagine I have some data stored in mongoDB collection.
Example:
Collection name: Users
record1: {
"first":"Cyril",
"age":33,
"id":1234
}
record2: {
"first":"Jeff",
"age":23,
"id":3423
}

My idea is to index this mongo data in the Elastic.
Such that Elastic only stores the IDs but not the full JSON.

So when I query in Elastic for getting users above the age of 30 I must get only 1234 and not the full JSON.
I will use this ID to search in mongoDB.
The idea is to store only the index info in Elastic while teh full data may reside on some other DB.

Is this possible ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7648f826-549e-4991-8532-9cd6ceb6148d%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/7648f826-549e-4991-8532-9cd6ceb6148d%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/DBEC3652-BC00-4D9E-9F78-404847AD0E80%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Ok! mistake on my side
The mapping should have been..
curl -XPUT 'http://localhost:9200/tests/_mapping/test' -d '
{
"test" : {
* "_source" : {"enabled" : false},*
"properties" : {
"message" : {"type" : "string" },
"projectId": {"type": "string" },
"userName": {"type": "string"}
}
}
}
'

BTW can you please let me know how can elastic give me all the info in
mongo via single request...

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a6dad34c-b41d-420b-a64c-8ba002ac5bb1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

But once again, it might happen that this option won’t be available anymore in newer versions.
Look at discussion here: Mappings: Ensure that reindexing is always possible · Issue #8142 · elastic/elasticsearch · GitHub https://github.com/elastic/elasticsearch/issues/8142

--
David Pilato - Developer | Evangelist

@dadoonet https://twitter.com/dadoonet | @elasticsearchfr https://twitter.com/elasticsearchfr | @scrutmydocs https://twitter.com/scrutmydocs

Le 6 avr. 2015 à 12:19, David Pilato david@pilato.fr a écrit :

It should be inside « test » not inside « properties ».
See _source field | Elasticsearch Guide [8.11] | Elastic http://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html
{
"tweet" : {
"_source" : {"enabled" : false}
}
}

--
David Pilato - Developer | Evangelist
elastic.co http://elastic.co/
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr https://twitter.com/elasticsearchfr | @scrutmydocs https://twitter.com/scrutmydocs

Le 6 avr. 2015 à 11:50, Cyril Cherian <cyril@mobomo.com mailto:cyril@mobomo.com> a écrit :

Thanks David,

I still have some doubts in teh approach you sent.
I am using Elastic version 1.1.1

  1. I created a mapping called tests
  2. Created a mapping
    curl -XPUT 'http://localhost:9200/tests/_mapping/test' http://localhost:9200/tests/_mapping/test' -d '
    {
    "test" : {
    "properties" : {
    "_source" : {"enabled" : false},
    "message" : {"type" : "string" },
    "projectId": {"type": "string" },
    "userName": {"type": "string"}
    }
    }
    }
    '
  3. Inserted some values inthe index
    Example:
    curl -XPUT 'http://localhost:9200/tests/test/1' http://localhost:9200/tests/test/1' -d '{
    "message" : "Cyril here",
    "projectId": "1234",
    "userName": "rahul"
    }'

curl -XPUT 'http://localhost:9200/tests/test/1' http://localhost:9200/tests/test/1' -d '{
"message" : "a message Cyril here",
"projectId": 134,
"userName": "cyril"
}'

  1. Now when i do a search like http://localhost:9200/tests/test/_search?q="message" http://localhost:9200/tests/test/_search?q="message"

I get _source (which i am not expecting)...what i want to achieve is that elastic should just index the data but not store the data.
"hits": [ http://localhost:9200/tests/test/_search?q="message"#
{ http://localhost:9200/tests/test/_search?q="message"#
"_index": "tests",
"_type": "test",
"_id": "1",
"_score": 0.11506981,
"_source": { http://localhost:9200/tests/test/_search?q="message"#
"message": "a message Cyril here",
"projectId": 134,
"userName": "cyril"
}
}
]

Let me know if I am missing on something.

On Monday, April 6, 2015 at 2:05:15 PM UTC+5:30, Cyril Cherian wrote:
Imagine I have some data stored in mongoDB collection.
Example:
Collection name: Users
record1: {
"first":"Cyril",
"age":33,
"id":1234
}
record2: {
"first":"Jeff",
"age":23,
"id":3423
}

My idea is to index this mongo data in the Elastic.
Such that Elastic only stores the IDs but not the full JSON.

So when I query in Elastic for getting users above the age of 30 I must get only 1234 and not the full JSON.
I will use this ID to search in mongoDB.
The idea is to store only the index info in Elastic while teh full data may reside on some other DB.

Is this possible ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7648f826-549e-4991-8532-9cd6ceb6148d%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/7648f826-549e-4991-8532-9cd6ceb6148d%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/DBEC3652-BC00-4D9E-9F78-404847AD0E80%40pilato.fr https://groups.google.com/d/msgid/elasticsearch/DBEC3652-BC00-4D9E-9F78-404847AD0E80%40pilato.fr?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6C88277D-02D5-4FBD-A4BF-759ED485E793%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

If you store the _source, you by default get back your JSON content in _source field.
If you don't store _source, the you need to query Mongo.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 avr. 2015 à 12:21, Cyril Cherian cyril@mobomo.com a écrit :

Ok! mistake on my side
The mapping should have been..
curl -XPUT 'http://localhost:9200/tests/_mapping/test' -d '
{
"test" : {
"_source" : {"enabled" : false},
"properties" : {
"message" : {"type" : "string" },
"projectId": {"type": "string" },
"userName": {"type": "string"}
}
}
}
'

BTW can you please let me know how can elastic give me all the info in mongo via single request...

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a6dad34c-b41d-420b-a64c-8ba002ac5bb1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5E19331D-307E-496A-B661-9637E86A7E94%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Thanks David!

On Monday, April 6, 2015 at 2:05:15 PM UTC+5:30, Cyril Cherian wrote:

Imagine I have some data stored in mongoDB collection.
Example:
Collection name: Users
record1: {
"first":"Cyril",
"age":33,
"id":1234
}
record2: {
"first":"Jeff",
"age":23,
"id":3423
}

My idea is to index this mongo data in the Elastic.
Such that Elastic only stores the IDs but not the full JSON.

So when I query in Elastic for getting users above the age of 30 I must
get only 1234 and not the full JSON.
I will use this ID to search in mongoDB.
The idea is to store only the index info in Elastic while teh full data
may reside on some other DB.

Is this possible ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cfa57491-e0a0-4bd8-a5de-e75730da3e36%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.