(↓TL;DR↓)
We published a post last month that covered our journey to Celery as well as our intent and reasons for going back to a custom-crafted job engine. Since then, our team has made substantial progress and we want to share the results of our efforts with you, our respected customers.
8.4—Tallman is Built on the CloudBolt Job Engine
After installing or upgrading to CB 8.4—Tallman, your jobs will be executed by the CloudBolt Job Engine (CBJE) instead of Celery.
The following sections give an overview of the advantages this brings.
Full Active-Active HA
Previously, there were a few barriers to running multiple CBJEs simultaneously—some of these (in particular, the locks in provisioning job) were obstacles for any CBJE (Celery and CB-crafted). In 8.4—Tallman, we have improved these locks and moved them to the DB so that they are shared and respected across CB VMs.
We have also made improvements to the flow of code that is used to pick up pending jobs so that they will be more evenly distributed across active CBJEs. In turn, CBJEs will not enqueue jobs when they do not have sufficient free threads to execute them.
New CloudBolt System Status Page
In 8.4—Tallman, you can navigate to Admin > Support Tools > System Status to see an overview of the active CBJEs, how many jobs each has run in the last hour & day, a series of health checks, and a button to enable & disable the admin-only maintenance mode.
We will be adding quite a bit more to this page over time.
Other CBJE-Related Improvements
In addition to the above, we have added ambitious automated scalability tests to our labs to ensure that there are no regressions and only improvements to our performance and stability. We run tests of single orders with a large number of provision jobs, many orders each with one provisioning job, large sync jobs, and also large numbers of simultaneous decommission jobs. We also streamlined the handling of CB IP Pools (which will speed provisioning and sync jobs), improved job cancelation, enhanced availability checking for IP addresses, made our locking around CB resource pools more granular, and started tracking which CBJE runs each job and storing this in the DB.
Other 8.4—Tallman Improvements
Even though this post is about the CBJE, we can’t resist bragging about some of the other features in this release. In particular, we now have arbitrary resource discovery and sync (where discovery plug-ins can be defined on blueprints), improvements to Nutanix & Azure support, enhancements to regenerated parameter options, the aforementioned admin-only maintenance mode, and a plethora of import/export improvements.
Next Up
Here at CloudBolt, we will continue making scalability and robustness improvements in 8.5 and will also be releasing benchmark numbers for the CBJE. Along the way, please let us know if you have any specific scalability requirements, as we’d love to implement more real-world use cases in our automated scale testing, and also reach out to us if you have any feedback on 8.4—Tallman, we want to hear the good, the bad, and the ugly.
Already using CloudBolt? Click here to upgrade to the latest 8.4—Tallman RC
If not, click here to download our 25 VM Lab
(↑To top↑) Upgrade your dev/test CloudBolt instance to the latest 8.4—Tallman RC, it will make you and your colleagues happy
