when I restart the server, I am getting the error as
[2017-11-12T11:15:45,501][WARN ][o.e.c.InternalClusterInfoService] [DWzMZYO] Failed to execute IndicesStatsAction for ClusterInfoUpdateJob
org.elasticsearch.ElasticsearchSecurityException: error attempting to authenticate request
at org.elasticsearch.xpack.security.support.Exceptions.authenticationError(Exceptions.java:33) ~[?:?]
at org.elasticsearch.xpack.security.authc.DefaultAuthenticationFailureHandler.exceptionProcessingRequest(DefaultAuthenticationFailureHandler.java:64) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$AuditableTransportRequest.exceptionProcessingRequest(AuthenticationService.java:541) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$10(AuthenticationService.java:281) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:284) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$null$0(AuthenticationService.java:218) ~[?:?]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:59) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.xpack.security.authc.TokenService.getAndValidateToken(TokenService.java:218) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:214) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:246) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:257) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:210) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:159) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:122) ~[?:?]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:185) ~[?:?]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:145) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:168) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:408) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1256) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.stats(AbstractClient.java:1577) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.cluster.InternalClusterInfoService.updateIndicesStats(InternalClusterInfoService.java:270) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.cluster.InternalClusterInfoService.refresh(InternalClusterInfoService.java:321) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.cluster.InternalClusterInfoService.maybeRefresh(InternalClusterInfoService.java:277) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.cluster.InternalClusterInfoService.access$500(InternalClusterInfoService.java:67) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.cluster.InternalClusterInfoService$SubmitReschedulingClusterInfoUpdatedJob.lambda$run$0(InternalClusterInfoService.java:224) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) [elasticsearch-5.6.0.jar:5.6.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: java.lang.IllegalArgumentException: JWT Token can't be empty. JWT Token should be passed as authorization header.e.g Authorization Bearer <Token>
at com.jvemugunta.es.security.realm.JWTRealm.token(JWTRealm.java:119) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:273) ~[?:?]
Please make the effort to format your post to be as readable as possible - there's a live preview panel for exactly this reasons.
Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If you are interested in getting an answer, it's in your interest to make it as easy to read and understand as possible.
This exception is being thrown by your code, so I really don't understand what you are asking us.
Why does your code throw this exception? Is the Authorization header missing, or is it coming through as a standard Basic auth header?
If you can provide concrete details about what Elasticsearch is doing, we can help, but I can't just guess what's going on from an error message in your code.
Please also provide precise version numbers in your questions, our answers are almost always dependent on the version of ES you are running.
However, I can provide some pointers on your setup:
Don't throw an exception from your token method. That will cause fatal errors in the authentication module. Instead, call setFailedAuthenticationDetails to log the error and then return a null result.
You've enabled a file realm and an anonymous role, but then your code assumes that every incoming request will have a JWT token. That doesn't make sense - if you are allowing anonymous users and file based users, then you can't expect that every request will have a JWT header.
How does your data get into Elasticsearch? You seem to expect that every request comes via nginx and Kibana, but that's not going to be true for data ingest.
public static final String TYPE = "jwt-realm";
public static final String AUTHORIZATION_HEADER = "Authorization";
/**
* Constructor for the Realm. This constructor delegates to the super class
* to initialize the common aspects such as the logger.
*
* @param config
* the configuration specific to this realm
*/
public JWTRealm(RealmConfig config) {
super(TYPE, config);
}
/**
* This constructor should be used by extending classes so that they can
* specify their own specific type
*
* @param type
* the type of the realm
* @param config
* the configuration specific to this realm
*/
protected JWTRealm(String type, RealmConfig config) {
super(TYPE, config);
}
@Deprecated
@Override
public User authenticate(AuthenticationToken token) {
throw new UnsupportedOperationException("Deprecated");
}
/**
* Method that handles the actual authentication of the token. This method
* will only be called if the token is a supported token. The method
* validates the credentials of the user and if they match, a {@link User}
* will be returned as the argument to the {@code listener}'s
* {@link ActionListener#onResponse(Object)} method. Else {@code null} is
* returned.
*
* @param authenticationToken
* the token to authenticate
* @param listener
* return authentication result by calling
* {@link ActionListener#onResponse(Object)}
*/
@Override
public void authenticate(AuthenticationToken authenticationToken, ActionListener<User> listener) {
try {
JWTToken token = (JWTToken) authenticationToken;
final String jwtToken = (String) token.credentials();
UserInfo userInfo = new JWTValidator().parseAndValidateJWTToken(jwtToken);
if (userInfo != null) {
listener.onResponse(new User(userInfo.getSubject(), userInfo.getRoles()));
} else {
listener.onResponse(null);
}
} catch (Exception e) {
listener.onFailure(e);
}
}
@Override
public User lookupUser(String arg0) {
return null;
}
@Override
public boolean supports(AuthenticationToken token) {
return false;
}
@Override
public AuthenticationToken token(ThreadContext threadContext) {
String header = threadContext.getHeader(AUTHORIZATION_HEADER);
String token = null;
if (header != null && header.startsWith("Bearer")) {
token = header.substring("Bearer".length()).trim();
}
if (Strings.isEmpty(token)) {
throw new IllegalArgumentException(
"JWT Token can't be empty. JWT Token should be passed as authorization header.e.g Authorization Bearer <Token>");
}
return new JWTToken(token);
}
@Override
public boolean userLookupSupported() {
return false;
}
when I start the elasticsearch, I am getting the error at the immedietly after the startup as
Caused by: java.lang.IllegalArgumentException: JWT Token can't be empty. JWT Token should be passed as authorization header.e.g Authorization Bearer
** at com.bcbsfl.es.security.realm.JWTRealm.token(JWTRealm.java:119) ~[?:?]**
** at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:273) ~[?:?]**
** ... 28 more**
How does your data get into Elasticsearch? You seem to expect that every request comes via nginx and Kibana, but that's not going to be true for data ingest.
There are two ways to access Elasticsearch
The data ingest happens with Java client API using "elastic" user as Authorization Basic header and the data ingest is working
The second method is to search the data ingested as above. The flow would be as Browser -> NGINX -> Kibana -> Elastic Search
All the requests coming from the browser will reach Kibana as Authorization : Bearer eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJqdmVtdWd1bnRhIiwicm9sZXMiOiJzdXBlcnVzZXIiLCJleHAiOjE1MTA0OTQ1MDQsImlhdCI6MTUxMDQ1ODUwNH0.KwmtMgLBMwdL3mDmO0VgQ1k_-GF5HSig991h7jIVJ4bbJNoSlGAhd13YN60YhpgLGHG5jCTsR92W2txp4pqWpKnk1whaHMUn_et4u1XkgrlXnErtrihwVdyAYufR5FdksrItW9udFL82r4QnSdaQb4U2J7c0R6dgRg7S2paetetlYIYCZQ_uEwsOHDbRkM0Sxn4FDBy8mKwwSJsmrbLwZ2Ph2WaFfgqO1dCLW0hyMc-l82Ka1kZ0V9iDFLVRsaBXuSKPIVz_uxFYn9vFhcQ9KLGwoS_JbC7Ds2j8cfXGR1Rm11yueqh2ZCvhjLFCDVtyerFkMm6IDEzbiRlXR9KHVjuJ0J7QwJDZiRiNJG7dobhtNBFX5XbzK40ihqwT3ETRSgb6p0SOqBCOrDG1bv1Udb0EYmPTvjShH0kzZoKh_6lNqjhACk5gOZs__GCIY8eJ5EZuk0Ijg-PN97jth_ny7dH2CHJLuWm8-cu1uekhHXAD_UawRleQYrt6zqJKFGf-hFNJHd0AgpLkKk30-cHEXwPrK2zOFOxpBOoHU7unGW9OAuf4X5FIvGfb4L0qIMNpL2rtHilAm6-f4CPOhXPxTkVEGXg-Jc_yQCUAPZxiGhL-02pJ_Y2G4R8aSUEZS0rVIDWx2DwUd6oodxFhEotZhgfvq3EaOSK8OxHrp6IyKzA
This header is nothing but as JWT token. the token comes as "Authorization Bearer "
Hope this makes sense to undersatnd what I want to achieve
I have added the right checks using setFailedAuthenticationDetails method
Reinstalled the custom realm using
bin/x-pack/extension install file:///Users/jvemugunta/git_repos/shield-custom-realm-example/build/distributions/x-pack-custom-realm-extension-example-5.6.0.zip
When I start up, I don't see any more errors
However, when I call the rest API using JWT token, I end up this error
public static final String TYPE = "jwt-realm";
public static final String AUTHORIZATION_HEADER = "Authorization";
Logger logger = ESLoggerFactory.getLogger(JWTRealm.class);
/**
* Constructor for the Realm. This constructor delegates to the super class
* to initialize the common aspects such as the logger.
*
* @param config
* the configuration specific to this realm
*/
public JWTRealm(RealmConfig config) {
super(TYPE, config);
}
/**
* This constructor should be used by extending classes so that they can
* specify their own specific type
*
* @param type
* the type of the realm
* @param config
* the configuration specific to this realm
*/
protected JWTRealm(String type, RealmConfig config) {
super(TYPE, config);
}
@Deprecated
@Override
public User authenticate(AuthenticationToken token) {
throw new UnsupportedOperationException("Deprecated");
}
/**
* Method that handles the actual authentication of the token. This method
* will only be called if the token is a supported token. The method
* validates the credentials of the user and if they match, a {@link User}
* will be returned as the argument to the {@code listener}'s
* {@link ActionListener#onResponse(Object)} method. Else {@code null} is
* returned.
*
* @param authenticationToken
* the token to authenticate
* @param listener
* return authentication result by calling
* {@link ActionListener#onResponse(Object)}
*/
@Override
public void authenticate(AuthenticationToken authenticationToken, ActionListener<User> listener) {
try {
JWTToken token = (JWTToken) authenticationToken;
final String jwtToken = (String) token.credentials();
UserInfo userInfo = new JWTValidator().parseAndValidateJWTToken(jwtToken);
if (userInfo != null) {
listener.onResponse(new User(userInfo.getSubject(), userInfo.getRoles()));
} else {
listener.onResponse(null);
}
} catch (Exception e) {
listener.onFailure(e);
}
}
@Override
public User lookupUser(String arg0) {
return null;
}
@Override
public boolean supports(AuthenticationToken token) {
return false;
}
@Override
public AuthenticationToken token(ThreadContext threadContext) {
String header = threadContext.getHeader(AUTHORIZATION_HEADER);
if (Strings.isEmpty(header)) {
logger.log(Level.INFO, "Authorization header should not be empty");
setFailedAuthenticationDetails("Authorization header should not be empty", null);
return null;
}
String token = null;
if (header != null && header.startsWith("Bearer")) {
token = header.substring("Bearer".length()).trim();
}
if (Strings.isEmpty(token)) {
logger.log(Level.INFO, "token is not valid " + header);
setFailedAuthenticationDetails("JWT Token should be passed as authorization header.e.g Authorization Bearer <Token>",
new IllegalArgumentException("token is not valid " + header));
return null;
}
return new JWTToken(token);
}
@Override
public boolean userLookupSupported() {
return false;
}
Thanks Tim. I was able to fix the permissions. Can you advise me on the root cause of the problem
My roles.yaml file looks as below
role1:
indices:
- names: [ 'index1' ]
privileges: [ 'read' ]
role2:
indices:
- names: [ 'index2' ,'index1']
privileges: [ 'read' ]
[2017-11-13T21:37:24,711][DEBUG][r.suppressed ] path: /index1/tweet/1, params: {pretty=, index=index1, id=1, type=tweet}
org.elasticsearch.ElasticsearchSecurityException: unable to authenticate user [jvemugunta] for REST request [/index1/tweet/1?pretty]
at org.elasticsearch.xpack.security.support.Exceptions.authenticationError(Exceptions.java:39) ~[?:?]
at org.elasticsearch.xpack.security.authc.DefaultAuthenticationFailureHandler.failedAuthentication(DefaultAuthenticationFailureHandler.java:37) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$AuditableRestRequest.authenticationFailed(AuthenticationService.java:607) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeUser(AuthenticationService.java:385) ~[?:?]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:59) ~[elasticsearch-5.6.0.jar:5.6.0]
at org.elasticsearch.xpack.common.IteratingActionListener.onResponse(IteratingActionListener.java:108) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$consumeToken$13(AuthenticationService.java:318) ~[?:?]
at org.elasticsearch.xpack.common.IteratingActionListener.onResponse(IteratingActionListener.java:111) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$consumeToken$13(AuthenticationService.java:318) ~[?:?]
at org.elasticsearch.xpack.common.IteratingActionListener.onResponse(IteratingActionListener.java:111) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$consumeToken$13(AuthenticationService.java:318) ~[?:?]
at org.elasticsearch.xpack.common.IteratingActionListener.run(IteratingActionListener.java:93) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeToken(AuthenticationService.java:326) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$9(AuthenticationService.java:275) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:284) ~[?:?]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$null$0(AuthenticationService.java:218) ~[?:?]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:59) ~[elasticsearch-5.6.0.jar:5.6.0]
That means what it says: the request could not be authenticated.
This is an issue for you to debug - why didn't your realm authenticate that request?
Given that the error message contains a username, it indicates that something (presumably your realm) generated a token, but then the token was considered to be invalid. It would seem that your authenticate(AuthenticationToken, ActionListener<User>) method, isn't doing the right thing, but the issue is within your JWTValidator, and I can't help with that.
Sorry, Tim. I didn't get your point. Apologies. This is the code you are suggesting to look into.
I don't see this method being called in the stackTrace or in the logs.
Can you help me on where you saw the reference to JWTValidator in the logs.
@Override
public void authenticate(AuthenticationToken authenticationToken, ActionListener listener) {
Hi Tim, I have figured out the issue. I have not implemented supports() method in the custom realm. Once I have implemented this method, things start flowing
@Override
public boolean supports(AuthenticationToken token) {
return token instanceof JWTToken;
}
However, the authentication is still not successful when the code tries to load a resource(public certificate in pem format). I can see the public certificate is in the plugin class path. This is my final distribution jar file and you can see the public_key.pem file in the jar file
jar tvf x-pack-custom-realm-extension-example-5.6.0-all.jar | grep public 2329 Tue Nov 14 16:22:02 EST 2017 public_key.pem
This is how I am trying to load the public_key.pem file in my code
InputStream in = Thread.currentThread().getContextClassLoader().getResourceAsStream("public_key.pem");
The input stream is coming as null. I do the run similar code with my test cases and they all are working good
Can you give me some pointers on why the resources is not getting loaded
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.