The core microservice presented in the parts from 1 to 3 was assuming that everything happens as expected and all the events and conditions are on the happy path. This is not always the case. For example, our ActiveMQ broker may experience issues and be only partially available. Or it may simply be stopped. Or the network failure might prevent the connections to get established, etc.

In all these cases the consumer calls the service which will finally timeout but, before doing that, might waste resources without eventually being able to fulfill its role. Entry Netflix Hystrix service.

The Netflix Hystrix service can be used to provide the following patterns:

• Failling fast. This pattern is also called "circuit breaker". It monitors endpoints and, when it detects a non responding one, it fails fast without calling it any more, avoiding this way to waste resources. It is further able to detect that the endpoint becomes responsive again.
• Fallback. This pattern is a variant of the previous one and consists in gracefully failing a non responding service, by providing an alternative endpoint to be called in this case.
• Bulkhead. This pattern consists in segregating remote services calls to separated threads such that, in the case of non responsive microservices, to avoid blocking the main thread.

The samples we are looking at here only shows the first two patterns: the fail fast and the fallback. The code of our REST API has been modified such that to add Hystrix annotations. Let's look at the code:

@RequestMapping(value = "/subscribe/", method = RequestMethod.POST)
@HystrixCommand(fallbackMethod = "gtsFallbackSubscribe")
public ResponseEntity gtsSubscribe(@RequestBody SubscriberInfo si) throws JMSException
{
   logger.debug("*** HmlRestController.gtsSubscribe(): Simulating a long running process.");
   try
   {
     Thread.sleep(5000);
   }
   catch (InterruptedException e)
   {
     e.printStackTrace();
   }
   subscribers.put(si.getSubscriptionName(), si.getSubscriberInfo());
   logger.debug("*** HmlRestController.gtsSubscribe(): Have subscribed to events {}, {}, {}",
     si.getSubscriptionName(), si.getSubscriberInfo().getClientId(),
      si.getSubscriberInfo().getMessageSelector());
   return ResponseEntity.accepted().build();
}


The code above  shows the subscribe endpoint that, by now, should be quite familiar. It has been annotated with the @HystrixCommand annotation. The effect of this annotation is that, when the execution takes too long, meaning that the microsrvice is degradated, an alternative endpoint will be called. This alternative endpoint is, in our case the private method gtsFallbackSubscribe which simply displays a message saying that the service is not available.

The Hystrix library is fully configurable and it has dozens of properties. Here we are just using the default parameters, suitable in the most simple cases. If the subscribe endpoint execution takes longer then 1 second, then it is interrupted and the alternative endpoint is called. here 1 second is the default value of the circuitBreaker.requestVolumeThreshold property and it can be customized, of course. The Hystrix documentation shall be used in order to get the full understanding of all the properties and of the way that they should be combined such that to customize the behavior.

We've just seen the fallback pattern. Now in order to experience the fail fast we just comment out the fallbackMethod property above and, rebuilding the docker containers and performing the test again we should experience a timeout exception instead of a gracefully fallback.

In order to build and start the microservices, the same instructions presented in the previous Part 3 (http://nicolasduminil.blogspirit.com/archive/2018/06/08/basic-of-microservices-part-3-3106892.html) shall be applied here as well. The only specific instruction is that the name of the Git branch to be checked-out is now "resilience".

Netflix Hystrix also provides a dashboard to be used for monitoring purposes. You can open it by going to http://<service-ip-address>:<service-tcp-port>/hystrix. In our case these URL are http://172.19.0.5:8080/hystrix for the ms-core microservice and http://172.19.0.6:8081/hystrix for ms-core2. Once in the dashboard, you need to navigate to the hystrix.stream URI (http://172.19.0.5:8080/hystrix.stream for example) and click on the Monitor Stream button. Enjoy !