byrajesh04-11-201810:52 AM - edited 04-11-201810:59 AM
Zuora developers need to be able to test the microservices they are building against the Zuora Central Platform and other dependent Zuora microservices.
Currently, developers write Docker Compose files for their microservice and Zuora Central Platform; this allows them to test their code quickly against a local stack on their laptop. Although this gives developers some level of flexibility and speed, as the number of microservices continues to increase, running the stack on a commodity laptop is no longer feasible as the laptop does not have the resources to run all the microservices.
To head off this problem, we are trying to provide a framework for developers to launch quick, dynamic and customizable development environments in the cloud. In particular, we are testing a solution that reuses the existing Docker Compose files on a Docker Swarm cluster in AWS, with connectivity from the local laptop’s microservices.
The list below represents some of the requirements that we have collected from developer teams with respect to developer environments. In no particular order:
Minimize changes developers need to make to adopt new environments
Ability to deploy all or subset of microservices and components into an isolated environment for development and testing
Dependency management (i.e developer pulls in microservice A, automatically pull in the Zuora Central platform and any dependent microservices)
Ability to specify a desired version of the microservice for testing
Run services on their laptop that can interact with the Docker Swarm cluster in the cloud
Focus on a low cost solution
Easy to stand up/tear down
Use existing or create a standard for describing a microservice environment setup and dependencies
Proof of Concept & Results
We implemented a proof of concept (POC) of the Docker Swarm solution, by running the Zuora Central Platform with some of our production ready microservices. One particular goal of the POC was to validate whether we can reuse existing microservice Docker Compose files to run in a Docker Swarm cluster in the cloud.
One of the goals was to be able to stand the cluster up and minimize any changes that teams need to make to their configuration files. Another major goal of the POC was to vet out the needs and understand scope for the development environments in order to derive a strategy to auto scale or automate the environments.
Figure 1- Integrating local and cloud based developer environments
Based on the POC, as displayed in Figure 1, we were able to address most of the initial requirements including avoiding changes that are required from the microservice teams to adopt the new framework.
Furthermore, the POC vetted out that developers would be able to run their in-development containerized microservices locally while standing up any dependencies in the Swarm cluster quickly. Developers were also able to run the entire stack on the shared Swarm cluster if they wanted to - Figure 2 represents one such configuration:
With the POC completed, we continue automating the Swarm based framework. As part of our future work, we have identified two cost-based improvements that we think will make the adoption of the framework more economical:
Ephemeral clusters - transient clusters that will expire after a certain amount of time
Spot fleets - leveraging spare AWS capacity to run clusters on the cheap like we do for our CI environments