In elastic search, when checking index health link, we could see that each index is having huge amount of data as per the size. But when running api <index_name>/_search, it is retrieving only only one alias data. We are not sure why we are not able to access other aliases data. Could you please help us on this?
What is the output from _cat/aliases?v?
Can someone provide me update on this?
I think what you mean to say is - It is returning data of only one aliasindex viz en_y_96 and not from remaining indices? Is that correct?
If you want data from all indices, then for your use case, you'll need to do GET en_*/_search.
Alternatively, you need to map all indices to an alias and not the other way round. Then you can do GET your_alias_name/_search.
From the alias screenshot you shared, it seems you've done the reverse whereby you've mapped a number of aliases to one index.
You can use the following snippet to map all indices to a single alias.
POST /_aliases
{
"actions" : [
{ "add" : {
"indices" : [
"en_*"], "alias" : "denmod_m" } }
]
}
and then get data from all indices using GET denmod_m/_search.
Hope this helps.
P.S: You should redact the URL of your ES Cluster.
Please note that there are no SLAs on these forums. If you'd like guaranteed response times, then please check out Subscriptions | Elastic Stack Products & Support | Elastic
4 Likes
When running the en_y_96/_alias, it is returning many aliases as like below screenshot.
But when we running en_y_96/_search, it is returning data of alias "denmod_y_108455" only and some how other data are not accessible.
Also we are using below code snippet for creating aliases.
Post
_aliases
{
"actions": [
{
"add": {
"index": "en_y_31",
"alias": "denmod_y_4280",
"filter": {
"term": {
"simulationId": 4280
}
}
}
}
]
}
@warkolm @sandeepkanabar Please let me know what should i do for accessing other data from the index en_y_96.
That might be because ES only returns top 10 records. To get more records, you can use the size and optionally from parameter.
@sandeepkanabar the problem is that data is available in elk server based on the size we can confirm that. But while fetching data through dot net application, it is returning null from ELK. Do you have any idea on that? My ELk version is 5.3.1.
Please don't post images of text as they are hard to read, may not display correctly for everyone, and are not searchable.
Instead, paste the text and format it with </> icon or pairs of triple backticks (```), and check the preview window to make sure it's properly formatted before posting it. This makes it more likely that your question will receive a useful answer.
1 Like
ok.. could you please help me on above issue?Could you explain what the problem is?
Hard to tell without seeing any information on that like code.
That said, I'm not a dotnet dev so I don't know if I can help but may be someone would be able to.
That's a way too old. It's not even the latest stable of the 5.x version. We don't support 5.x anymore.
Consider updating to 7.10.
1 Like
@DENMODSupport - In addition to David's answer, if the .NET application returns null, there could be a number of reasons.
- You need to check if the connection object is instantiated successfully or not.
- And when you get null, what does the stack-trace show?
- You can look up the appropriate documentation corresponding to your .NET client version.
I'm not a .NET guy but this should be your starting point and that question is better asked in a new post.
We are getting below error from elastic search logs.
[2020-12-04T13:25:20,571][WARN ][o.e.a.a.c.n.i.TransportNodesInfoAction] [elk-denmod-web] not accumulating exceptions, excluding exception from response
org.elasticsearch.action.FailedNodeException: Failed node [t7JmFe-iSn63CALYD-0lxw]
at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.onFailure(TransportNodesAction.java:246) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.access$200(TransportNodesAction.java:160) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleException(TransportNodesAction.java:218) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1032) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.transport.TransportService$Adapter.lambda$onNodeDisconnected$6(TransportService.java:859) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) [elasticsearch-5.3.1.jar:5.3.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: org.elasticsearch.transport.NodeDisconnectedException: [elk-denmod-1][168.124.25.140:9300][cluster:monitor/nodes/info[n]] disconnected
[2020-12-04T13:25:20,571][DEBUG][o.e.a.a.i.s.TransportIndicesStatsAction] [elk-denmod-web] failed to execute [indices:monitor/stats] on node [t7JmFe-iSn63CALYD-0lxw]
org.elasticsearch.transport.NodeDisconnectedException: [elk-denmod-1][168.124.25.140:9300][indices:monitor/stats[n]] disconnected
[2020-12-04T13:25:25,032][INFO ][o.e.c.s.ClusterService ] [elk-denmod-web] added {{elk-denmod-1}{t7JmFe-iSn63CALYD-0lxw}{Ah6ks-dJScCxug_-xO4zkA}{168.124.25.140}{168.124.25.140:9300},}, reason: zen-disco-receive(from master [master {elk-denmod-web-2}{GhAsUHj8TOSdqWwjdOuPJA}{hTnv0AekT6eVJYSJSBPiDg}{xspw50a102k.pharma.aventis.com}{168.124.170.195:9300} committed version [2274]])Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.
Or use markdown style like:
```
CODE
```
This is the icon to use if you are not using markdown format:
There's a live preview panel for exactly this reasons.
Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.
2 Likes
i formatted the code.. Could you please help me on that error which am getting in elastic search
Is that related to the original question? What are you doing to generate such a log? Or what is happening in the cluster when this log appears?
Hello @DENMODSupport ,
From the initial screenshot, it seems your indices are named (2 examples):
en_y_71
en_m_73
en_y_96
When you search against en_y_96, you are searching against a normal index.
The document count 6830118 in the _search result (hits.total) matches the screenshot you've shared (GET _cat/indices?v).
In the screenshot you've shared with all the aliases of en_y_96, we see you are trying to use a filtered alias to emulate some sort of isolation/security restriction based on SimulationId.
If you want to get the data of denmod_y_108455, you have to search against the alias, not the actual index, using POST dnmod_y_108455/_search.
If you search against en_y_96 it search against all documents.
Checking the stacktrace you've provided, it seems you might be hitting https://github.com/elastic/elasticsearch/pull/25016, but I am not 100% sure.
The node elk-denmod-web detects the node elk-denmod-1 disconnects/fails.
Can you check/share what are the logs on the other node around the time 2020-12-04T13:25:20,571?
Regarding the .NET issue and returning null, it might be because the node(s) is removed/disconnects from the cluster.
Would it be possible to share:
- What is the version of the Elasticsearch client you're using
- The settings of the client and the parameters to connect to Elasticsearch (excluding sensitive data)
- Potentially, a snippet of the code of the request you're sending.
The official Elasticsearch client documentation is at this page and as you can see in the compatibility matrix, they are no more officially supported as Elasticsearch 5.3.1 reached end of life on October 2018.
1 Like
The reference you have provided is having "AlreadyClosedException" but we are getting "Node Disconnected exception". Please find the above stack trace for your reference.
Please find error logs from other node at the same time.
[2020-12-04T13:25:20,520][INFO ][o.e.c.s.ClusterService ] [elk-denmod-6] removed {{elk-denmod-1}{t7JmFe-iSn63CALYD-0lxw}{Ah6ks-dJScCxug_-xO4zkA}{168.124.25.140}{168.124.25.140:9300},}, reason: zen-disco-receive(from master [master {elk-denmod-web-2}{GhAsUHj8TOSdqWwjdOuPJA}{hTnv0AekT6eVJYSJSBPiDg}{xspw50a102k.pharma.aventis.com}{168.124.170.195:9300} committed version [2273]])
[2020-12-04T13:25:24,998][INFO ][o.e.c.s.ClusterService ] [elk-denmod-6] added {{elk-denmod-1}{t7JmFe-iSn63CALYD-0lxw}{Ah6ks-dJScCxug_-xO4zkA}{168.124.25.140}{168.124.25.140:9300},}, reason: zen-disco-receive(from master [master {elk-denmod-web-2}{GhAsUHj8TOSdqWwjdOuPJA}{hTnv0AekT6eVJYSJSBPiDg}{xspw50a102k.pharma.aventis.com}{168.124.170.195:9300} committed version [2274]])
[2020-12-04T13:41:30,120][INFO ][o.e.d.z.ZenDiscovery ] [elk-denmod-6] master_left [{elk-denmod-web-2}{GhAsUHj8TOSdqWwjdOuPJA}{hTnv0AekT6eVJYSJSBPiDg}{xspw50a102k.pharma.aventis.com}{168.124.170.195:9300}], reason [transport disconnected]
[2020-12-04T13:41:30,135][WARN ][o.e.d.z.ZenDiscovery ] [elk-denmod-6] master left (reason = transport disconnected), current nodes: nodes:
{elk-denmod-4}{rS_6QTYGSiKTyXJnjvSdyw}{EZ4987G6TQymqCTJk_Mxmw}{168.124.170.244}{168.124.170.244:9300}
Our Elastic search client version is 5.3.1
Please find the settings and parameters to connect to elasticsearch below.
cluster.name: elk-denmod
cluster.routing.allocation.disk.watermark.low: 95%
node.name: elk-denmod-web
bootstrap.memory_lock: true
network.host: [ELK1.pharma.aventis.com,_local_]
discovery.zen.ping.unicast.hosts: [ELK1.pharma.aventis.com, ELK2.pharma.aventis.com, ELK3.pharma.aventis.com, ELK4.pharma.aventis.com, ELK5.pharma.aventis.com, ELK6.pharma.aventis.com, ELK7.pharma.aventis.com, ELK8.pharma.aventis.com]
node.master: true
node.data: false
node.ingest: false
xpack.security.enabled: false
xpack.monitoring.enabled: false
Note: Server names are replaced by ELK(1-8) numbering
Please find the document which contains snippet of the code.
I requested to share the logs of the node elk-denmod-1, not elk-denmod-6.
This is the configuration file of one of the nodes.
In particular, it is a master node and you should avoid connecting to it for search requests.
You should use a coordinating or target directly a data node if possible.
Can you run GET _cat/nodes?v and explain how the client connects to the cluster (what are the connection parameters you provide in your .NET application to connect to Elasticsearch and to which host you connect to)?