Java API for Shield is not Working

[Anusha]

Hi Team,

I installed shield 1.3 as am using elasticsearch 1.5.2,
created **user : es_admin **
role : admin
password : anusha

Tested the response in Sense plugin, working fine with user and password able to get the response.

But when
Am using Java API for to add user credentials as shown..

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "elasticsearch").put("shield.user", "admin:anusha").build();
String token = basicAuthHeaderValue("es_admin", new SecuredString("anusha".toCharArray()));

Client client = new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress("localhost", 9300))

// Code for Search Response using client

SearchResponse response = client.prepareSearch("ast").setTypes("ast_type").putHeader("Authorization", token).setSize(410000).execute() .actionGet();

When am executing above code getting exception as:

Exception in thread "main" org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: []
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:278)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:197)
....................................

Can anyone plz help me to solve this...

[jimczi (Jimferenczi)]

Maybe not related but why are you setting the size to 410,000?, seems like a very big number of hits to retrieve in one request. Check the logs of your es instance to see if your node did not ran out of memory (which could explain the exception you're getting).

[Anusha]

Hi Jim,

Thanks for your response,

If I disable the shield plugin (like I have added shield.enabled: false in elasticsearch.yml file), then the query working fine and able to get the response,

May I know the why it is behaving differently when I use shield plugin,
But am able to get the response in sense plugin..

The dependency that I have added for shield and license is :

    <!-- add the Shield jar as a dependency -->
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch-shield</artifactId>
        <version>1.0.0</version>
    </dependency>
    <!-- add the License jar as a dependency -->
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch-license-plugin</artifactId>
        <version>1.0.0</version>
        <scope>runtime</scope>
    </dependency>

[dadoonet (David Pilato)]

Why not using shield 1.3?

[Anusha]

Hello David,

Yes I changed to 1.3.3 (shield version), even though it is behaving in the same way

@Anusha

Exception in thread "main" org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available:

.........

Don't know, is there any thing wrong in the query??? If so, then why the query is working when I disabled the shield plugin..

[dadoonet (David Pilato)]

I'd check all the steps there: https://www.elastic.co/guide/en/shield/shield-1.3/_using_elasticsearch_java_clients_with_shield.html

I don't see BTW why you are adding the license plugin.

[Anusha]

I just commented out the license plugin dependency, my application is showing lot of errors.

Is there any problem on adding license plugin in dependencies??????

[Anusha]

And I even verified the versions of the shield and license plugin those are installed in ES using ,

GET /_nodes?plugin=true

in sense plugin, seen that shield : 1.3.3 and license :1.0.0 versions, so I have used those versions in my dependencies.

[dadoonet (David Pilato)]

If I read the doc correctly you don't need the license plugin when you use Java transport client.

[Anusha]

Yes David, I just removed the license plugin dependency(no errors I have seen in my code) , but even though am unable to get the response......

[Anusha]

Hi David,

This time am getting a different error, where I just changed here

ImmutableSettings.settingsBuilder().put("cluster.name", "elasticsearch").put("shield.user", "es_admin:anusha").build();

shield.user I have changed as es_admin:anusha

And for this am getting Exception as,

Exception in thread "main" org.elasticsearch.shield.authc.AuthenticationException: missing authentication token for action [indices:data/read/search] at org.elasticsearch.shield.authc.InternalAuthenticationService.authenticateWithRealms(InternalAuthenticationService.java:229) at org.elasticsearch.shield.authc.InternalAuthenticationService.authenticate(InternalAuthenticationService.java:136)......

[dadoonet (David Pilato)]

You really added that?

String token = basicAuthHeaderValue("es_admin", new SecuredString("anusha".toCharArray()));
client.prepareSearch().putHeader("Authorization", token).get();

[dadoonet (David Pilato)]

May be you could share:

• your config/shield/* files
• your config/elasticsearch.yml file
• your elasticsearch logs
• your java client code

[Anusha]

Hi David,

Here is my files:

elasticsearch.yml:

cluster.name: elasticsearch

node.name: "Franz Kafka1"

node.master: true

node.data: true

discovery.zen.ping.multicast.enabled: false

discovery.zen.ping.unicast.hosts: ["localhost"]

http.jsonp.enable: true
script.disable_dynamic: false
script.inline: on
script.indexed: on
http.cors.enabled: true
http.cors.allow-origin: http://localhost:5601

In shield folder:

logging.yml:

logger:
shield.audit.logfile: INFO, access_log

additivity:
shield.audit.logfile: false

appender:

access_log:
type: dailyRollingFile
file: {path.logs}/{cluster.name}-access.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}] %m%n"

role_mapping.yml:

roles.yml:

# All cluster rights

# All operations on all indices

admin:
cluster: all
indices:
'*': all

# monitoring cluster privileges

# All operations on all indices

power_user:
cluster: monitor
indices:
'*': all

# Read-only operations on indices

user:
indices:
'*': read

# Defines the required permissions for transport clients

transport_client:
cluster:
- cluster:monitor/nodes/info
#uncomment the following for sniffing
#- cluster:monitor/state

# The required role for kibana 3 users

kibana3:
cluster: cluster:monitor/nodes/info
indices:
'*': indices:data/read/search, indices:data/read/get, indices:admin/get
'kibana-int': indices:data/read/search, indices:data/read/get, indices:data/write/delete, indices:data/write/index, create_index

# The required permissions for kibana 4 users.

kibana4:
cluster:
- cluster:monitor/nodes/info
- cluster:monitor/health
indices:
'*':
- indices:admin/mappings/fields/get
- indices:admin/validate/query
- indices:data/read/search
- indices:data/read/msearch
- indices:admin/get
'.kibana':
- indices:admin/exists
- indices:admin/mapping/put
- indices:admin/mappings/fields/get
- indices:admin/refresh
- indices:admin/validate/query
- indices:data/read/get
- indices:data/read/mget
- indices:data/read/search
- indices:data/write/delete
- indices:data/write/index
- indices:data/write/update
- indices:admin/create

# The required permissions for the kibana 4 server

kibana4_server:
cluster:
- cluster:monitor/nodes/info
- cluster:monitor/health
indices:
'.kibana':
- indices:admin/exists
- indices:admin/mapping/put
- indices:admin/mappings/fields/get
- indices:admin/refresh
- indices:admin/validate/query
- indices:data/read/get
- indices:data/read/mget
- indices:data/read/search
- indices:data/write/delete
- indices:data/write/index
- indices:data/write/update

# The required role for logstash users

logstash:
cluster: indices:admin/template/get, indices:admin/template/put
indices:
'logstash-*': indices:data/write/bulk, indices:data/write/delete, indices:data/write/update, indices:data/read/search, indices:data/read/scroll, create_index

# Marvel role, allowing all operations

# on the marvel indices

marvel_user:
cluster: cluster:monitor/nodes/info, cluster:admin/plugin/license/get
indices:
'.marvel-*': all

# Marvel Agent users

marvel_agent:
cluster: indices:admin/template/get, indices:admin/template/put
indices:
'.marvel-*': indices:data/write/bulk, create_index

users.yml:

es_admin:$2a$10$RbWXUQSEmsR1JbhmP4xCFudyMc1SvT.KXjz3rvc2ZzxJ2XW8v9sgW

user_roles.yml:

admin:es_admin

Here is my Java Client Code:

public class Test{

public static final Client client = getTransportClient("localhost", 9300);
public static String token = null;

public static Client getTransportClient(String host, int port) {

    Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "elasticsearch").put("shield.user", "es_admin:anusha").build();
    token = basicAuthHeaderValue("es_admin", new SecuredString("anusha".toCharArray()));

    return new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress(host, port));
}

    public static void getResponse(String index, String indextype) throws InvalidFormatException, Exception {
    SearchHits hits = null;
    SearchHit hit = null;

    SearchResponse response = client.prepareSearch(index).setTypes(indextype).putHeader("Authorization", token).get();

    hits = response.getHits();
    Iterator<SearchHit> hitsIte = hits.iterator();
    while (hitsIte.hasNext()) {
        hit = hitsIte.next();
        // jsonArray.add(hit.getSource());
        System.out.println(hit.getSourceAsString());
    }
}

}

[Anusha]

Hi David,

I hope this information supports you to trace the root cause..

[Anusha]

Thanks David,

Got the response,

@Anusha

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "elasticsearch").put("shield.user", "es_admin:anusha").build();
token = basicAuthHeaderValue("es_admin", new SecuredString("anusha".toCharArray()));

return new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress(host, port));

Here token need to set after client....