Skip to content
This repository has been archived by the owner on Jul 18, 2024. It is now read-only.

Home

Jump to bottom
Animesh Singh edited this page Aug 8, 2017 · 10 revisions

Welcome to the resilient-java-microservices-with-istio wiki!

Short Name

Make your Java microservices resilient with Istio

Short Description

Enable your Java microservices with advanced resiliency and fault tolerance leveraging Istio

Offering Type

Cloud

Introduction

Twelve factor apps make a strong case for designing and implementing your microservices for failure. What that means is with the proliferation of microservices, failure is inevitable, and applications should be fault tolerant. Istio, a service mesh can help make your microservices resilient without changing application code.

Authors

Tommy Li, Animesh Singh

Code

Demo

  • N/A

Video

  • Coming Soon

Overview

Building and packaging microservices is one part of the story. Given a highly salable and distributed microservices deployment is going to face failures at different layers, how do we make these microservices resilient and fault tolerant? How do we enforce policy decisions such as fine-grained access control and rate limits? How do we enabled timeouts/retries, health checks etc? Even though some language specific frameworks address these issues, the implementation is often framework or language specific. If the underlying framework or language changes, the resiliency features need to be reimplemented or ported over. And in some cases, applications also have the responsibility of implementing the code and configuration required for resiliency and fault tolerance. A Service-mesh architecture attempts to solve these issues by extracting the common resiliency features needed by a microservice framework away from the applications and frameworks and into the platform itself. Istio provides an easy way to create this service mesh.

In this code we demonstrate how to build, deploy, connect your Java MicroProfile microservices leveraging Istio service mesh. We then show how to configure and use circuit breakers, timeouts/retries, rate limits and other advanced resiliency features from Istio without changing the application code.

Flow

Flow

  1. In this step, the user deploys a Java application that is configured to run in a Kubernetes Cluster The application, MicroProfile, is composed of five microservices. The application is written in Angular and Java. The 'Vote' microservice has two versions. V1 stores its data locally and V2 stores its data with the Cloudant Database. User ensures Istio service mesh control plane is running on Kubernetes.

  2. To enable the application to use Istio features, user injects Istio envoys on the application. Envoys are deployed as sidecars on each microservice. Injecting Envoy into microservice means that the Envoy sidecar would manage the ingoing and outgoing calls for the service. User also creates ingress to enable ingress traffic from Istio ingress.

  3. Having deployed the application, user now goes and configures some advanced Istio features for the MicroProfile. User creates a Circuit Breaker for the Cloudant database. We specify maximum active connections to Cloudant database, and maximum pending requests. If we sent more than allowed maximum requests at once to Cloudant, it will have pending requests up to the specified threshold and deny any additional requests until the pending requests are processed.

  4. Another type of circuit breaker which can be used is based on the health of the instances in a load balancing pool. User creates a load balancing pool with two instances of Cloudant, and then uses a circuit breaker which can detect and eject one of the instances when its no longer healthy.

  5. User creates a timeout and retries rule for the ‘Vote’ microservice connection to Cloudant. Once applied, this rule timeouts all the responses that take more than the allowed response time threshold. User also applies retries rule by telling Istio how many retries he wants, and what should be the timeout for retry. Moreover, the user creates a fault injection on Cloudant database to simulate the failure situation and verify the timeout and retries rule works properly.

Included components

Featured technologies

Cloud

Database

[Containers]

[Microservices]

[Java]

Blog post

by Animesh Singh

How to make your microservices resilient and fault tolerant using Istio

Microservices and 12 factor applications solved a lot of issues with monolithic applications, but as mentioned in my previous [post](inert link), as the number of these microservices continue to grow, they also introduce some new challenges, such as service discovery, routing, failure handling etc.

In this blog we dive into how Istio can be leveraged to make your application fault tolerant and resilient? How do we introduce health checks, timeouts, retries, ensure request buffering or reliable communication between microservices? Some of these features are coming built in microservices framework, but often they are language specific, or you have to accommodate for it in your application code. How do we introduce it without changing the application code? Service-mesh architecture attempts to solve these issues. Istio provides an easy way to create this service mesh by deploying a control plane and injecting sidecars containers alongside your microservice.

Istio adds fault tolerance to your application without any changes to code. Some of resiliency features it supports are:

  • Retries/Timeouts
  • Circuit breakers
  • Health checks
  • Control connection pool size and request load

Circuit Breakers Circuit breaking is a critical component of distributed systems. When we apply a circuit breaker to an entity, and if failures reach a certain threshold, subsequent calls to that entity should automatically fail, without applying additional pressure on the failed entity and paying for communication costs. If done right, there should be a fallback for that entity which handles the response for failure It’s nearly always better to fail quickly and apply backpressure downstream as soon as possible. Envoy enforces circuit-breaking limits at the network level as opposed to having to configure and code each application independently.

Timeouts and Retries Istio allows you to create route rules for your destination microservices where you can specify the timeout and retry policies. For e.g. you can create a route rule if your microservice does not respond within N seconds. Once applied, this rule will timeout all the responses that take more than N seconds in the destination service. You also can apply retries rule by telling Istio how many retries you want if a particular microservices is not reachable, and what should be the timeout for your retry. So if at attempt first your destination microservice is not reachable in N seconds, you can tell Istio to do M number of retires, and also increase the timeout for retries beyond N.

Health checks Istio can also perform health checks against a load balancing pool. A microservice in a load balancing pool can have multiple deployed instances, and Istio distributes the traffic across those instances. If some of those instances are broken, Istio can perform health checks and eject any broken instance in your load balancing pool to avoid any further failure.

Control connection pool size and request load

Istio can also be used to specify maximum active connections to a microservice, or maximum pending requests. We can set destination microservice maximum connections to N and maximum pending requests to M. Thus, if we sent more than N+M requests at once to the microservice, it will have M pending requests and deny any additional requests until the pending request is processed.

Give it a try

In our new journey we show how you can deploy a Java MicroProfile microservices application leveraging Istio capabilities to introduce resiliency. MicroProfile is an open source baseline platform definition that optimizes Enterprise Java for microservices architecture

You’ll learn how Istio resiliency features such as health checks, timeouts, retries, ensure request buffering or reliable communication between microservices can be enabled without requiring changes to your microservice code. We’ve also integrated the journey with Bluemix DevOps Toolchain to provide a one-click deployment for anyone who wants to try it out quickly. Please use the journey, extend it, and send your feedback. As with every IBM Code journeys, all pull requests are welcome!

Links

Clone this wiki locally