The Next Generation Service Gateway

Cloud Native Application

Today, all the software architectures are migrating to Cloud Native architecture. The Cloud Native architecture has the following characteristics:

  • It’s built by Microservice architecture with resilience design.
  • On-demand performance and scale capability.
  • Decoupling the control logic and business logic completely.
  • Aiming the high availability and >99.9% uptime SLA.

And four type important things that need to be well constructed and scheduled by the Cloud Native way.

  1. Infrastructure Runtime Resource. The underlying computing node, storage, and network are all managed and scheduled by container and Kubernetes.
  2. Application Data & State. Application data and state need to be persistent or synced up cluster-wide. We can see the mainstream database(SQL/NoSQL), cache, queue, etc now support distributed cluster and HA solutions.
  3. Application Service. Those services are now orchestrated by Kubernetes and managed by a microservice architecture, including service discovery, service configuration management, service gateway/proxy, resilient & fault-tolerant, liveness or readiness health checking, observability( metrics, logs, tracing ), etc.
  4. Traffic Orchestration. Most of the online services need to deal with user requests from all over the world. That traffic would have different purposes and access different resources. So, it’s very important to manage that traffic and align it with Cloud Native backend services.

Traffic Orchestration

Regarding traffic orchestration, to be a cloud-native gateway, it must have the following features.

  • Align with the Microservice architecture.
  • High Availability with distributed system technologies.
  • Orchestrating traffic with a pipeline of various filters and API aggregation.
  • Compliant with Cloud Native architecture, such as Service Mesh, FaaS, etc.
  • Service Observability — tracing, metrics(throughput, latency, errors, etc), access logs.

To achieve this, here, we are going to introduce a next-gen service traffic gateway — Easegress

This software is open-source software(Apache 2.0 license) written in Go, with the Go programming language, it naturally has high performance for high concurrent scenarios.

Easegress Overview

The major design of Easegress is as below:

The traffic handling functions or filters can be developed by a plugin mechanism.

The functions or filters can be organized to a pipeline in runtime by admin API.

There are two types of controllers that help to manage and integrate to the whole Cloud Native architecture.

  • Traffic Controller — Mesh, FaaS, etc.
  • System Controller — Service Discovery, Monitor, Cluster, etc.

Application-level protocols support — HTTP 1/2/3, WebSocket, MQTT

Easegress Architecture

Easegress Features

Furthermore, Easegress has the following Cloud Native features.

Microservice architecture compliance

  • Spring Cloud compatibility
  • Restful API protocol — API Routing & API Aggregation.
  • Service Discovery integration — Eureka, Consul, Nacos, Etcd, and Zookeeper.
  • Resilience & Fault Tolerance (porting from resilience4j project ) — circuit breaker, rate limiter, retryer, time limiter, etc.

Cloud-Native architecture integration

  • Kubernetes Ingress Control
  • Service Mesh — Sidecar & Ingress Controller
  • FaaS — Knative
  • High Availability & High performance

Built-in Raft consensus & leader election provides 99.99% availability.

  • Load Balance — round-robin, random, weighted random, IP/header hash.
  • Cache — cache the back-end service response.

Observability

  • Service Tracing — built-in Open Zipkin, and Open Tracing for vendor-neutral APIs.
  • Throughput — QPS/TPS m1, m5, m15 and error rate
  • Latency — P999, P99, P98, P95, P75, P50, P25
  • Bandwidth — request and response data size
  • Response — HTTP Status Code statistics.

Easegress Extensibility

Besides, Easegress has very good extensibility, it’s very easy to add customized features or functions into Easegress, there are three ways to do this.

  • Filters or Controller. Following the development guide, you can develop your filters and controllers in the Go language. But it needs to recompile the software.
  • WebAssembly. Easegress supports the WASM runtime engine, so, you can develop the filters in any programming language that supports WASM, and Easegress can load it in runtime.
  • Function as a Service. Easegress supports Knative integration, so, you can deploy the function service besides the Easegress. This way seems a bit heavy, by working with Kubernetes, but it’s a scalable solution.

So, Easegress could be a development framework, you can focus on your own business on traffic gateway.

Ease Cookbook

Easegress is capable to do the following works(including but not limited to):

  • Reverse Proxy (Load balancing) — Rich strategies of load balancing [doc]
  • Service Discovery — Zookeeper, Eureka, Consul, Nacos, etc. [doc]
  • FaaS — Knative integration [doc]
  • Kubernetes Ingress Controller [doc]
  • Pipeline — Orchestrating HTTP filters for requests/responses handling. [doc]
  • API Aggregation — Aggregating many APIs into a single API. [doc]
  • Resilience & Fault Tolerant — Circuit breaker, Rate limiter, Retryer, Time Limiter, etc. [doc]
  • Distributed Tracing — Supporting APM tracing — Zipkin. [doc]
  • Performance optimization such as compression, cache, etc. [doc]
  • Flash Sales. High concurrent promotion sales. [doc]
  • Service Mesh — This is leveraged by [Ease Mesh] — a Spring Cloud compatible Mesh system.
  • Workflow(IFTTT) — running a number of APIs as a workflow. [doc]
  • WebAssembly — Using AssemblyScript to extend the Easegress. [doc]

Today, we are looking forward to your suggestion or concern about this new traffic gateway, and even help us make this gateway great. Please free feel to file an issue or contribute the code in our Github — https://github.com/megaease/easegress