HOAB

Problem :

Following the previous post (this one and this one) I configured a Authorization and Resource server on a a same JVM.

All was working well in my local machine, but when I send the springboot on the server, I get an "Invalid access token". The authorization request was accepted, I get an access token but it was refused by Resource server.

Solution :

I activate remote DEBUG, perform tests (I also had a reverse proxy but that's was not the problem). The issue was due to the tokenStore which :

• on my local machine was the same instance
• on the server was two different instance

In fact depending on the bean initialization order, the token store could be shared or not according to optional Autowire field of the authorization and resource server. If not available at init time, it could use a local instance.

So here is my updated configuration :

My AuthorizationServer is now :

package com.example;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.oauth2.config.annotation.configurers.ClientDetailsServiceConfigurer;
import org.springframework.security.oauth2.config.annotation.web.configuration.AuthorizationServerConfigurerAdapter;
import org.springframework.security.oauth2.config.annotation.web.configuration.EnableAuthorizationServer;
import org.springframework.security.oauth2.config.annotation.web.configurers.AuthorizationServerSecurityConfigurer;
import org.springframework.security.oauth2.provider.token.TokenStore;
import org.springframework.security.oauth2.provider.token.store.InMemoryTokenStore;

@Configuration
@EnableAuthorizationServer
public class AuthorizationServerConfig extends AuthorizationServerConfigurerAdapter {


    @Autowired
    public TokenStore tokenStore;

    @Override
    public void configure(ClientDetailsServiceConfigurer clients) throws Exception {
        clients.withClientDetails(new MyClientDetailsService());
    }

    @Override
    public void configure(AuthorizationServerSecurityConfigurer security) throws Exception {
        security.allowFormAuthenticationForClients();
    }

    @Override
    public void configure(AuthorizationServerEndpointsConfigurer endpoints) throws Exception {
        endpoints.tokenStore(tokenStore);
    }
}

And my resource server :

package com.hexagon.hpa.security;

import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.oauth2.config.annotation.web.configuration.EnableResourceServer;
import org.springframework.security.oauth2.config.annotation.web.configuration.ResourceServerConfigurerAdapter;
import org.springframework.security.oauth2.config.annotation.web.configurers.ResourceServerSecurityConfigurer;
import org.springframework.security.oauth2.provider.error.OAuth2AccessDeniedHandler;

@Configuration
@EnableResourceServer
public class ResourceServerConfig extends ResourceServerConfigurerAdapter {

    private static final String RESOURCE_ID = "RESSOURCE_ID";

    @Autowired
    TokenStore tokenStore;

    @Override
    public void configure(ResourceServerSecurityConfigurer resources) {

        resources.resourceId(RESOURCE_ID);
        resources.tokenStore(tokenStore);
    }

    @Override
    public void configure(HttpSecurity http) throws Exception {
        http.anonymous().disable()
                .authorizeRequests()
                .antMatchers("/api/**").authenticated()
                .and().exceptionHandling().accessDeniedHandler(new OAuth2AccessDeniedHandler());
    }

}

A simple configurer :

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.oauth2.provider.token.TokenStore;
import org.springframework.security.oauth2.provider.token.store.InMemoryTokenStore;

@Configuration
public class TokenStoreProvider {

    @Bean
    public TokenStore tokenStore() {
        return new InMemoryTokenStore();
    }
}

Problem :

We are testing a springboot application in AWS with ELB in front.

After a while of load-testing, the application was hanging :

• HTTP 504 error code from Jmeter client
• HTTP 502 if we raise ELB timeout
• Once logged on the server :
  • telnet localhost 8080 was OK
  • sending GET / on this socket was not responding
  • plenty of CLOSE_WAIT socket
  • wget was also hanging (normal)
  • connection was established during wget hang
  • nothing in the log

Solution :

I initially think about the keepAlive timeout and pool of tomcat but

1. SpringBoot copy the connectionTimeout parameter to keepAliveTimeout
2. new socket is accepted and established
3. CLOSE_WAIT wasn't shutdown after hour

Doing the test many times, I finally so a classical "Too many open files" in the log. That's why I could not see more log during the hang.

So we change the nproc and nofile in /etc/security/limits.conf

And taadaaaa ! Nothing change in :

cat /proc/<$PID>/limits

Thanks to blogs over the world like this one :

• the service is start with systemd
• to override ressources limits with systemd :

[Service]
...
LimitNOFILE=500000
LimitNPROC=500000

At last but not least, the value of Tomcat NIO socket queue is around 10000 + other files + other process... choose wisely your limit