By｜Zhao Zhenling (work alias: Youji)
Technical Expert at Ant Group, Project Lead of the Koupleless Project
This article 3724 words Reading time 10 minutes
Contact the author / join co-construction / use the product
This article is one of the Koupleless Advanced Series Articles, and we assume readers are already familiar with the basic concepts and capabilities of Koupleless. If you are new to Koupleless, you can check out the official website.
Under the Koupleless modular architecture, there are four core features: Fast, Resource-efficient, Flexible Deployment, and Smooth Evolution. These core advantages come from vertical and horizontal layered decoupling of application architecture, finding the optimal balance between isolation and sharing. It also upgrades the entire application lifecycle (requirement -> development -> testing and verification -> release -> online operation and scheduling, etc.) process, including base pre-warming, independent module iteration, and machine consolidation and reuse.
This article will analyze the advantages of Koupleless modularity from the perspective of isolation and sharing, provide corresponding performance benchmark comparisons, and also introduce in detail the challenges behind modularization and their solutions.
Analysis of Advantages Brought by Sharing
Traditional SpringBoot applications are completely isolated from each other and communicate via RPC. The modular approach enables more direct sharing capabilities between modules, including sharing of classes and objects, and more direct and efficient in-machine JVM service calls, thus achieving a better balance between isolation and sharing. To deeply analyze the effect of enhancing sharing on the basis of isolation, we split the community application eladmin into one base + three modules for the experiment, and collected data for reference. You can download and verify it yourself if you are interested.
Class Sharing
Koupleless uses SOFAArk to delegate classes in modules to the base for loading, which allows the module ClassLoader to only load module-specific classes at runtime. These reused classes are not included in the module package, thus reducing the size of the build artifact and the Metaspace memory consumption at runtime. The amount of size reduction varies depending on the proportion of shared classes between the module and the base. In the eladmin example, the traditional image-based build artifact is 300MB, while the modular build artifact is only 100KB.
Build artifact size comparison (assuming base image is 200M)
Object Sharing
Object sharing mainly reuses objects, logic, resources and connections on the base in the following two ways:
1. Sharing of static variables or objects
2. Modules call base logic via JVM Service, which is similar to API calls with no serialization overhead
static Object Sharing
Multiple modules reuse base classes through the class delegation loading mechanism. Some static variables in these reused classes have already been initialized when the base starts. When a module starts, it will directly use the values initialized by the base if it finds that the static variables in these classes have already been initialized. This allows static variables or objects defined in base classes to be shared by modules, especially singleton objects, which are very common in middleware. During module initialization, the framework automatically checks whether an instance of these objects exists, and reuses it if it does, for example, the CacheManager in ehcache.
Module Calling Base via JVM Service
General logic in the base is defined in base beans or interfaces, and modules can call these beans or interfaces via annotations (annotations for Dubbo and SOFARPC) or API calls. This allows modules to start without reinitializing infrastructure services or connections, reducing module resource consumption and improving module startup speed. At the same time, since these interfaces and beans are in the same process, API or JVM Service calls have no serialization and deserialization overhead, so there is no call performance degradation.
Through the class and object reuse introduced above, we can see that compared with traditional applications, the memory consumption of the module has dropped from 200MB to 30MB. At the same time, because the initialization logic of some classes and objects is reduced, the startup time has also dropped from 8 seconds to 4 seconds.
Memory consumption comparison
Startup time comparison
Detailed data table
We can see that sharing brings exponential benefits. The amount of benefit is related to how many base classes and resources are reused. If more logic is accumulated in the base, the startup speed of modules can be further improved.
Inside Ant Group, there are many infrastructure SDKs. After sinking these into the base, the startup speed of most applications has been reduced from the order of minutes to around 10 seconds.
Analysis of Problems Brought by Sharing
In addition to bringing benefits, sharing also brings some problems in certain special cases. These problems mainly fall into two categories: multi-application and hot deployment:
1. For multi-application scenarios, the main issues are static variable sharing and multiple ClassLoader switching
2. For hot deployment, the main issue is that some resources are not released during dynamic unloading
Based on 5 years of experience accumulated within Ant Group, Koupleless has summarized the list of encountered problems and provided corresponding solutions. Here we list the encountered problems to let everyone understand the problems and challenges of modularization, and work with the community to improve the modular framework together.
Class Sharing
The current class sharing design is that modules delegate some common classes to the base, and modules can reuse base classes during class lookup.
Static Shared Variables
In the single-process multi-application mode, modules reuse classes in the base. Static variables in these classes are initialized once when the base starts. When the module starts again, there are two possible cases:
1. Directly reuse the value from the base
- This is consistent with expectations in most cases. In a small number of cases where the module expects to use its own value, it will end up using the base's value, which does not meet the actual expectation
2. Directly overwrite the value of the base
- For cases where the module expects to use its own value, if it directly overwrites the original value, an overwriting problem occurs: static variables will only retain the value from the last installed module
Solution: This problem can be solved by adding an additional layer of map with the classLoader as the key to the original static variable.
Multiple ClassLoader
We define a class delegation relationship: prioritize looking up classes in the module first, and only look up classes in the base if they are not found in the module. However, apart from classes that can be delegated to the base, there must be some classes that cannot be delegated to the base, which means this is a partial delegation approach. Therefore, there are 5 possible cases during the class lookup process.
Among these 5 cases, the following abnormal scenarios occur in a small number of situations:
-
A class exists in both the module and the base: if some classes are loaded by two ClassLoaders at the same time, and involve judgments such as instanceof, this will cause LinkageError or "is not assignable to" errors.
-
A class exists in the module but not in the base: if execution enters from the module ClassLoader to the base ClassLoader, and then the
Class.forName("module class name") method is executed in the base ClassLoader, the class cannot be found.
Solution: We need an adapter to ensure that the correct module ClassLoader is passed in when looking up a class. In addition, for thread pools where threads are reused, it is necessary to correctly bind the thread to the corresponding ClassLoader.
Some resources are not automatically unloaded (only for hot deployment)
Shutting down a modular SpringBoot actually only calls the Shutdown method of SpringContext. It relies on Spring's Context management to unload services and Beans, and relies on JVM GC to reclaim objects, and does not perform a real JVM shutdown. Framework and business developers generally pay little attention to resource cleanup during Shutdown, and mostly rely on JVM shutdown to automatically complete resource cleanup. Therefore, during hot deployment, when a module is uninstalled and then reinstalled, some resources such as threads, resources, and connections may not be cleaned up.
Solution: Users only need to actively listen to the SpringContext Close event and actively perform cleanup. There is also another problem where metaspace grows with each installation; Koupleless detects the metaspace threshold during installation, and if the threshold is exceeded, it can fall back to restart installation to solve the metaspace growth problem.
Solutions to Problems Caused by Sharing**
To solve these problems more systematically, we have designed and implemented tools for the full process of problem discovery -> governance -> prevention, which will be introduced in subsequent articles of this series. Of course, through user demand analysis, we also found that not all problems need to be solved, and only some of the problems need to be solved for different scenarios.
Middle Platform Modules
In the middle platform mode, the goal is fast startup and resource saving by consolidating into one JVM. Modules are relatively lightweight, usually just code snippets, and generally do not need to handle static / classLoader / resource unloading problems. This mode has the smallest problem domain, and the problems mentioned above basically do not exist, so you can directly integrate and use Koupleless.
Application Modules
In contrast to middle platform modules are application modules. Application modules are heavier, and can use various middleware capabilities just like ordinary applications, so this mode will have some of the problems mentioned above. For existing application modules, there are two scenarios: long-tail scenarios and non-long-tail scenarios. Fewer problems are encountered in non-long-tail scenarios, so we will look at non-long-tail scenarios first.
Non-long-tail Applications
Non-long-tail applications refer to applications where the traffic of each machine is relatively sufficient, and the computing capacity of each machine is already fully utilized. These applications generally do not have the problem of resource waste caused by splitting into microservices, and are more concerned about startup speed and iteration efficiency. We can solve part of the problems through reasonable deployment and scheduling strategies. First, we can adopt a 1:1 approach where only one module is installed on one base machine, which achieves fast startup while avoiding problems caused by consolidating multiple applications together. At the same time, through scheduling, each time a new version of a module is installed, it can be installed on an empty base (no modules have been installed before) machine. The old version can then be uninstalled via asynchronous restart, which solves the problem of residual resources after hot deployment uninstallation. Taking the upgrade process of a single module as an example, the specific process is as follows:
Step 1, initial state:
Step 2, select a machine from the buffer to install version 2 of module 1;
Step 3, schedule the base machine running version 2 to base group 1, and schedule the base machine running version 1 to the buffer;
Step 4, when the buffer cluster finds that a machine has a deprecated module instance, it initiates a restart to clean up the module instance on that machine;
By that analogy, all machines of base 1 are upgraded to version 2.
The effect of this approach is shown in the table below:
Long-tail Applications / Private Delivery
This scenario pursues resource efficiency, which requires consolidating multiple applications into one JVM, and will encounter problems with static variables and ClassLoader in multi-application scenarios (similar problems also occur when consolidating multiple applications to reduce RPC call consumption). This mode is also divided into two types according to whether efficient iteration is required:
1. If efficient iteration is not required, you can directly use static merged deployment, which does not have the problem of residual resources after hot deployment uninstallation.
2. Dynamic merged deployment: this mode has multi-application and hot uninstallation problems. Multi-application problems cannot be avoided, and need to be gradually solved through the complete set of tools we provide. The hot uninstallation problem can be solved by restarting the base synchronously or asynchronously during module deployment.
In the future, we will build ModuleController and corresponding platform capabilities, provide different "packages" for different scenarios, and help everyone choose a suitable mode on demand. This is Koupleless's approach to helping existing applications solve their actual problems, working from both the framework and platform perspectives. If you are interested, welcome to visit the official website koupleless.io join our group and build together.
Learn more…
Give Koupleless a Star✨:
https://github.com/koupleless/koupleless
Koupleless Official Website:
Further Reading
This is a discussion topic separated from the original topic at https://juejin.cn/post/7368725622681944083