It’s the convenient untruth: There is a common misconception that software that is flexible and extensible is also necessarily complex and hard to use. The thinking goes, if a product allows you to extend and customize it, it must be hard to set up and use.
SAP is the classic example of this. Their ERP has long been the archetype of extremely powerful software that can do anything you want. However, lots of time is required from SAP professional services to get it functional within an organization.
On the other end of the spectrum lie most consumer-facing mobile apps. They are simple to use, but generally have limited configurability or extensibility. I can’t program my own extensions for the Strava app on my phone (unfortunately).
This misconception saves product managers, designers, and software engineers a lot of work. If they can make the case that extensibility and simplicity cannot co-exist, it makes their job much easier.
Just because most software products lie on the line in the graph above does not mean that it is impossible to create a product that is both easy to use and extensible.
When Auggy da Rocha and I founded CloudBolt, we did so with the specific intention of building a product that was both usable out-of-the-box without any customization, but that also allowed more advanced administrators to customize the product to look and act exactly the way required for their organization. Achieving both of these simultaneously was not easy, and did not come for free, but we knew that this combination of attributes was necessary to provide a solution customers would love.
When we in the software industry get past the misconception that simplicity and flexibility cannot co-exist, we find it’s an intriguing and worthy challenge to figure out how to achieve both at once.
Simplicity and powerful extensibility are not unrelated, but they can be independently improved and evaluated. Here’s a better way to visualize the graph above:
I propose that we hold software to a higher standard, and that we use this list of attributes as a scorecard to help in the evaluation of software. This scorecard is also valuable for software development teams to evaluate the quality of their own software, and to strive for improvement. Though these attributes are focused on enterprise software, they are applicable for many consumer-facing products/services as well.
The 11 Attributes of Extensible & Consumable Software
- One can set up a new account (or, if on-prem, one can install the product) in 30 minutes or less
- Value seen within the first hour of use
- Full external-facing API
- The product’s UI uses the user-facing API exclusively. In other words, the UI does not depend on any API calls or interfaces that are not documented and available to customers.
- Authentication is customizable within the product’s UI
- The UI has a customizable look & feel
- The UI can be extended with custom modules
- Back end logic that can be extended with custom modules in various hook points
- Customizations & extensions are upgrade-safe
- Full documentation and support for the API, extension framework, and hook points
- Rich library of readily-available examples of extensibility
Techniques for Achieving All 11 Attributes
Some tips on how to achieve more of the attributes above:
- Setting up a public location with many extensibility examples. At CloudBolt, this takes the form of public github repository (called the CloudBolt Forge) and a CloudBolt server (called the CloudBolt Content Library), which each installation of CB knows how to query and display available examples so that users can browse extensions and find one to start with.
- Include time into the software design and implementation schedule to build the right API for new features, build a simple UI, and also make the feature extensible & customizable.
- If you do not have sufficient time to make it easy and extensible, start with extensibility, and schedule a project to make it easy in the immediate next release. This is a more specific incarnation of Ken Beck’s rule for software: “Make it work, make it right, make it fast. ” One example of this within CloudBolt is synchronizing user permissions. We had very little time to get this feature in initially, so we just exposed a new hook point at the time users log in which allowed customers to write their own synchronization logic, starting with some examples we provided. Not long after that, we followed up with a full UI for configuring synchronization of user permissions that obviated the need for looking at or changing any code.
- Collaborate in brainstorming. Get a group of creative people together around a [virtual] whiteboard, throw around ideas good and bad for ways to make the software both customizable and easy to use. You may be surprised at what you’re able to come up with together, and at how fun it is.
- Use the extensibility framework internally! Any time a new feature needs to be developed, ask the question: what if we had to implement this without any changes to the core product code, just using the extensibility framework. If it’s too hard, the extensibility of your product could use improvement.
- Constantly ask what the easiest way to provide customizability and extensibility would be. Instead of making the customer learn a domain specific language (DSL), could we allow them to program extensions in a language they already know? Instead of requiring them to code from scratch, could we give them a working chunk of code and then let them change it to meet their needs? Or, instead of requiring them to code, could we create a configuration UI for customizing the product?
Today, IT professionals have precious little time to administer enterprise systems, but they are also expected to integrate many of them with each other, and also to customize each. Only through enterprise software solutions that are simple to use and administer, and also easy to extend and customize, can we empower IT professionals to succeed in their work. The importance of delivering software that achieves this combination of attributes previously thought to be incompatible is growing constantly, with rising user expectations and the proliferation and heterogeneity of systems that need to be managed.
Experience the leading hybrid cloud management and orchestration solution. Request a CloudBolt demo today.
CloudBolt integrates with a great myriad of technologies in many different categories, with more integrations being added in every release. However, the defining characteristic of CloudBolt’s integrations is not their breadth, but their depth.
For IT administrators, it is not enough for their cloud platform to have surface-level integration; it needs meaningful, substantive support for other technologies so the IT organization can provide a fully-automated self-service experience to their internal (and sometimes external) customers.
One illustrative example is CloudBolt’s support for VMware. I will list all the ways in which CloudBolt integrates with VMware. The main takeaways are that CloudBolt has the most complete integration with VMware of any cloud management platform, and without all of these features within the integration, the IT team will be left to handle situations manually. The full benefit of automation is only realized when the integration goes this deep.
The net effect of having this depth of automation for and integration with VMware (and other on-premises virtualization systems) is that IT teams have their private datacenters leveled up with functionality traditionally relegated to the public cloud – self service provisioning, management, and decommissioning of environments by end users without IT’s intervention, modeling cost tracking and implementing shameback/showback, policy-based features such as order approval workflows, enforcement of expiration dates for resources, and power scheduling for VMs to ensure systems do not consume resources when they are not needed.
Features of CloudBolt’s Integration with VMware
- VM Build process:
- Discovery of available clusters, networks, datastores, datastore clusters, VM templates
- Provisioning using:
- templates stored in the VMware Content Library
- normal VM templates
- a blank VM and a network-boot based build system such as Razor or Cobbler
- Creation & management of multiple disks
- Creation & management of multiple network interfaces
- support for specifying adapter type
- Guest OS customization
- Static IP
- Setting annotations
- Auto-selection of the datastore with the most free space
- Datastore clusters
- Following progress of the build task
- Linked clone builds
- Templatized specification of which VMware folder in which to place new VMs
- Storage management
- Define rate multiplier for premium storage backends
- Limit access to storage backend per group or environment
- VM management:
- Creating new ones
- Auditing existing ones and deleting any past expiration
- Adding and removing disks, NICs, CPU, memory
- All the policy-based management features that CloudBolt supports on other platforms, including execution of remote scripts, tracking of expiration dates, chargeback, power scheduling, the ability to log into VMs remotely from the CloudBolt web UI
- VM Discovery:
- Detection and storage of 26 distinct attributes on virtual machines
- Automatically updating CloudBolt’s dynamic inventory database every 30 minutes with any changes to these attributes on VMs
- History tracking for all changes, including those made in CloudBolt and those discovered. This allows reporting on change history over time, including the built-in ability to compute how CPU and GB memory hours each server used over the course of a month.
VMware Cloud on AWS
- Everything supported by CloudBolt’s vCenter integration above
- Deployment and configuration of fenced networks
- Support for load balancers
- Support for edge gateways
- Micro-segmented firewall rules (tagging)
- Extension-friendly API wrapper to enable any NSX use cases like NAT’ing, custom route configurations, etc.
- Discovery of flows:
- Automatic discovery of flow inputs
- Ability to map those inputs to CloudBolt attributes
- Ability to set up a flow at any trigger point in CloudBolt, which allows it to run at many points during the provisioning process, be presented as a button on the server details page, run as a recurring job and many other places.
- This allows reuse of existing investment in vRO workflows for the CloudBolt customers who have switched from vRA to CloudBolt
- Deploying VMs via vCloud Director
- Discovery of existing VMs
- Management of existing VMs
- Useful for orgs that want to modernize their cloud platform, but are not yet ready to remove vCloud Director
For a cloud platform to deliver on its promise of self-service IT, it needs to have deep interactions with external systems, feature-rich integrations which do IT’s previously-manual work for them. This post covers the depth of CloudBolt’s integration with VMware, but a similar dive could be taken into all the other integrations CloudBolt provides out of the box and as importable content in its hosted Content Library.
As always, if there are more aspects of integration you would like to see, we would love to hear from you. The list above has been fueled by excellent ideas from our customers over the last nine years.
See these powerful VMware integrations in action. Request a CloudBolt demo today.
CloudBolt sponsored and attended VMworld 2019 in San Francisco (with 12 CloudBolters in attendance!) and it was an energy-packed event. I’ll summarize some of the news and talk from the conference here.
VMWare’s main announcements
Last week, VMware announced the release of:
- Tanzu – their Kubernetes orchestrator, essentially an answer to Google’s Anthos.
- Project Pacific – the effort to embed a container runtime into vSphere and provide visibility into both containers and VMs from within the vSphere UI.
- Updates to VMware Cloud on Amazon Web Services (AKA VMC on AWS) – including accelerated GPU Services and a new study showing cost savings in moving to VMC on AWS.
Analysis of VMware’s Direction
Shift of focus from IT to developers
VMware has traditionally focused on selling products and services to IT departments, but their messaging and product direction are steering toward selling to developers. This is likely in response to VMware’s observation that the locus of decision-making and the budget for technology are shifting toward development teams over time.
Embracing of containers
With both Project Pacific and Tanzu, it’s clear that VMware is now betting on containers and does not want to miss that train. These two projects will embed a container runtime in vSphere and provide a Kubernetes cluster management tool (playing in the same space as Google’s Anthos).
Emphasis on VMC on AWS
VMC on AWS is a key part of the hybrid cloud story that VMware is delivering. The idea is to keep running workloads on VMware ESXi, and using vCenter to manage them, but the servers run in data centers owned by AWS instead of customer-owned and operated data centers. This is appealing as it allows large organizations to swap their capex spend out for opex, and to do it without making major changes to applications to run using modern public cloud services and/or containers. The possibility remains that organizations could move applications off VMC on AWS to just AWS or a different cloud, so it will be interesting to see how VMware handles that long-term.
vRA 8 Announced
VMware officially announced vRealize Automation 8, the rewrite of their Cloud Management Platform. We talked with a lot of vRA 7.x customers who are wondering what the path forward looks like for them. vRA 8.0 will have some good new features (like more agnostic public cloud support, more flexible blueprints than vRA 7, and potentially enhanced extensibility), but that it will have a subset of the features of vRA 7. It remains to be seen when upgrades from vRA 7 to 8 will be supported, or how difficult they will be when that day comes. It’s also unclear whether old-style extensions will be supported.
What this Means for CloudBolt
Since 2011, CloudBolt has been focusing on meeting the needs of both:
- Empowering developers with a simple self-service way to obtain the resources they need to do their job AND
- Turning the central IT team into superheroes, giving them unmatched visibility and the ability to orchestrate and automate everything
At CloudBolt, we are passionate about the themes VMware brought up. Here’s how CloudBolt stacks up in these themes:
|Empowering developers & IT admins
|✅ Since 2011
|Easy upgrades of CloudBolt
|✅ Since 2012
|Agnostic hybrid cloud support
|✅ Since 2013
|Infinite and easy extensibility
|✅ Since 2013
|Solid support for GCP and Azure
|✅ Since 2014 (plus 6 other public clouds in the ensuing years)
|✅ Since 2014
|✅ Since 2015, and getting deeper in every CloudBolt version
|VMC on AWS support
|Coming in CloudBolt 9.1 in December
Summarizing, CloudBolt has been focused on the themes that matter most to IT and developers and the product has matured over many years of releases and management of production environments for global 2000 companies.
We look forward to heading back to VMworld in 2020, and in the meantime you can find us at upcoming VMware User Group (VMUG) gatherings in Boston (9/25), Atlanta (10/2) and Phoenix (10/30) this fall. Stop by to chat with us!
Want to see how CloudBolt stacks up with vRA? Download our datasheet today.
At CloudBolt, we believe that software solutions should be easy to maintain, manage, and understand. We also believe they should be self-regulating and self-healing, when possible. You will see a focus on this starting in 8.4—Tallman but also continuing through our 9.x releases, which will give you better visibility into CloudBolt’s internal status, management capabilities directly from the web UI, and reduce the number of times you need to ssh to the CB VM to check things or perform actions.
CloudBolt 8.4—Tallman introduces a new Admin page called “System Status” which provides several tools for checking on the health of CloudBolt itself.
The System Status Page in 8.4—Tallman
To see the System Status page in your newly installed/upgraded CloudBolt 8.4-Tallman, navigate to Admin > Support Tools > System Status. You will see a page that looks a bit like this:
There are three main parts of this page.
1. CloudBolt Mode
This section provides a way to put CloudBolt into admin-only maintenance mode. This prevents any user who is not a Super Admin or CloudBolt admin from logging in or navigating in this CloudBolt instance. This is useful for times when you need to perform maintenance on CloudBolt (eg. upgrading it, making changes to the database, etc), and you want to prevent users from accessing it while in an intermediate state, but you yourself need to perform some preparation and verification within the CB UI before and after the maintenance.
2. Job Engine
This section shows the status of each job engine worker, each running on a different CloudBolt VM now that active-active Job Engines are supported. It also shows a chart of all jobs run in the last hour and day per job engine. When things are healthy, and the job engines are not near their max concurrency limit, there should be a fairly even split of how many jobs are being run by each worker.
3. Health Checks
This section has several kinds of checks:
- Indications of the health of a specific service, as would be seen from the Linux command line when running `service <name> status`
- Tests of OS-level health, such as a check of available disk space on the root partition
- Functional tests, which perform some basic action to make sure systems are working properly. Functional tests in 8.4—Tallman include writing a file to disk and deleting it, creating an entry in the database and deleting it, and adding an entry to memcache and deleting it.
Ensuring the health of the systems that underlie CloudBolt can help you quickly hone in on the root cause of an issue, and we hope that the system status page will help narrow the time it takes to troubleshoot and resolve issues with CloudBolt.
What’s Next for the System Status Page
We have some ideas for what we might add next:
- Uptime metrics for each job engine worker
- The average time for jobs to complete for each worker
- Disk space checks for all partitions on the CB VM
- CPU, memory, I/O, and network utilization for the CB VM
- Uptime for the CB VM as a whole
- Network health checks, including:
- testing DNS lookups
- testing pinging the gateway
- testing connections to any configured proxies
If there are any of these that seem like they would be especially useful to you, we’d love to hear that to help us prioritize. We’d also love to hear any additional ideas you have for this new page!
You can contact us here, and if you don’t already have CloudBolt to try out the new features download our free 25 VM Lab
Public clouds provide an easy path to deploying virtual machines (VMs), but this ease of deployment, if not properly managed, can lead to a proliferation of uncategorized, mysterious, and often expensive VMs. This reviled situation is known as VM Sprawl.
Networking Appliance Company: Case Study
Case in point, a major networking appliance company had adopted public cloud usage team by team, and wound up with more than a dozen AWS, Azure, and GCP public cloud accounts. The result was multiple interfaces to see and manage all of their VMs, and no unified way to track each VM’s purpose, owner, or lifecycle. The inability to gain visibility into their VMs was costing the enterprise IT team money and time, and exposing them to unnecessary security risks.
Step 1: Automatically Set Expiration Dates
The first step they took to get control over the situation was to install CloudBolt and to connect it to all of their public cloud accounts. When connected to a virtualization system or public cloud account, CloudBolt automatically discovers all of the VMs, networks, images, etc, and begins tracking and reporting them. Standing up CloudBolt and connecting all of their public cloud accounts took less than half a day and gave the IT admins a single web interface where they could see all of their resources across their various public cloud accounts. It’s worth noting that CloudBolt synchronizes with the providers’ inventory every 30 minutes, so if a user powers off a VM, creates a new one, or changes an existing one (ex. adding memory to it), that change will appear in CloudBolt within half an hour.
Step 2: Plug-in
Next, the customer added a post-sync plug-in to CloudBolt that automatically set expiration dates on all of their VMs (here’s a sample CloudBolt plug-in that can be used as a starting point). They then emailed all of their developers and other users of the public clouds to let them know that their VMs would be turned off in 60 days, unless they went into CloudBolt’s web UI, claimed their VMs, and changed or removed the expiration date.
This step was extremely valuable to the organization as it allowed them to know which of their VMs were still needed, and which users and groups they belonged to. By the 60 day mark, they saw that half of their public cloud VMs were powered down, drastically reducing their public cloud costs. CloudBolt’s discovery and expiration date management had enabled the equivalent of sticking a post-it note on the shared refrigerator saying “Unclaimed food will be discarded Friday at 5pm”. The end result: chaos became order, their costs decreased drastically, no important VMs were lost, and the clean up took place without incident.
Step 3: Compare Before & After Costs
After these two steps, the IT department was able to contrast their pre-CloudBolt public cloud bills with their post-CloudBolt bills and present the savings to their peers and management. The strong accolades and recognition received for their work enabled the IT team to continue with additional improvements.
Self-Service IT & Power Schedules
Another improvement made was the roll-out of self-service provisioning of resources in any public cloud or any of their private virtualization systems (VMware and OpenStack). Depending on the group and the environment for the new resources, expiration dates with maximum lengths were set up to be enforced by CloudBolt (here’s a video on how those expiration dates are set up).
In addition, the team set up automated power schedules in CloudBolt so it would turn off VMs when they were not in use (saving even more money), implemented limits & quotas on groups and environments, and added approval processes for any orders in production environments.
For this IT department, striking back against VM sprawl saved money, time and increased visibility, and also greatly enhanced navigation of the infrastructure. Now, anyone with appropriate permissions can see which VMs belonged to who, how they are changing over time, plus notes detailing their purpose.
CloudBolt also allowed the entire IT organization to collaborate and troubleshoot more effectively. It established a common interface for cross-team collaboration, where IT and their internal customers gained a common control plane for infrastructure and applications. Their DevOps practices were advanced and CloudBolt provided them with a place to automate and standardize their practices and processes.
To learn more download our Solution Overview or get a free 25-server lab license today.
History of Technology: From Mainframe to Hybrid Cloud
From Mainframe to Hybrid Cloud
For Tech’s Future, the Only Constant is Change
Computing infrastructure has come a long way in the last 50 years and the rate of change continues to rise. In order to inform our view of where infrastructure and computing platforms are going in the future, we need to take a look at the past and how far we’ve come…
1960s – Mainframes & Timesharing
During the mainframe epoch, companies had only a few large computers, occupying entire rooms. They usually required physical access to use (though logging in over phone lines with primitive remote terminals started to emerge in the ‘60’s). Maintenance of these machines required several physical operators, and automation of this maintenance was not widely considered.
1970s – Advent of Personal Computers
During the ‘70’s, computers, started showing up on people’s desks, albeit in a limited manner, instead of filling entire rooms. Administration of these ‘desktops’ was still done locally.
1980s – Networks Arise
The ’80’s saw a growth, dare I say widespread connectivity of computers, including modems appearing everywhere, DNS’ creation in ’83, usenet, gopher, and even the inception of the World Wide Web (WWW) in ’89. Still, servers that large organizations owned and operated remained on a relatively small scale, and administration of these servers was done on a one-off basis.
1990s – The Web and the Proliferation of Physical Servers
The ’90’s saw drastic change to computing infrastructure. The first popular web browser (Mosaic) was released in ’93 and suddenly more servers were needed than before. Companies quickly moved from having 10’s or 100’s of servers, to having tens of thousands of servers.
This shift required a new approach to server management. No longer could you hire one system administrator to manage each server, (or a few), instead they each needed to manage hundreds, an insurmountable task without automation. Cfengine was released as the first configuration management tool in 1993, but the demand for automation greatly outpaced available solutions.
2000s – Virtualization
In the early ‘00’s, The maturation of Linux catalyzed a shift from traditional, proprietary Unix systems (such as Solaris, HP-UX, and IBM AIX, which ran on proprietary hardware) to RedHat and other Linux variants, running on commodity hardware. This was fueled by, and in turn further fueled the expansion of the number of servers that needed to be managed.
The early ‘00s saw the emergence of a new class of products called Data Center Automation (DCA) products, including Opsware and Bladelogic, however, Sun and HP had their competitors too.
Virtualization became accepted in the mid-’00s, first in dev/test labs, but increasingly in production environments. This let companies slow the growth in physical hardware, while still adding virtual servers, each of which with its own running OS. This resulted in more operational efficiency, but management nightmares abounded as it became harder to track, patch, upgrade, and secure all of the operating systems running in a datacenter.
More mature configuration management tools rose from this chaos, notably Puppet and Chef (founded in 2005 and 2008, respectively), and the term DevOps was coined in 2007.
2010s – Public Cloud
While AWS was released (in non-beta form) in 2008, it wasn’t widely adopted as a platform for enterprise computing until the 2010s. Suddenly, all of the frustrations of dealing with one’s own data centers could be solved with a credit card. No more waiting for the IT team to create your VM, no more dealing with the possibly obstructionist networking, database, or security teams…
Companies throughout the 2010s have shifted back and forth between bearishness and bullishness on the public cloud (often depending on how recent and shocking their last bill was). However, serious, enterprise IT shops are realizing that hybrid cloud is truly the best solution, and the end goal. Hybrid cloud enables them to use a mix of public clouds and their own datacenters, choosing the best environment for each workload. Hybrid also allows IT admins to use public cloud during times when demand is above average, scaling back down when demand subsides, so they do not wind up with a gigantic bill.
The Advent of a Hybrid Cloud Management Platform
The first pre-release of what is now CloudBolt was created in 2010. The gap that my co-founder Auggy and I saw was that, in larger companies, the interface to the IT organization was broken. Developers (and other folks who needed IT resources), would submit a ticket to IT, and then wait weeks to get what they needed (often a VM, but sometimes a physical server, network change, storage allocated, etc). This problem was exacerbated in the ‘90s and ‘00s with the rise in demand of access to servers and VMs, and was made highly contentious by the advent of public cloud (“If I can get a VM in minutes from AWS, why do I have to wait weeks for my IT people to get me one?!”).
CloudBolt brings the experience of using a company’s private datacenter up to par with the public cloud experience, and spans the eras from the physical server age, through VMs and public clouds, to the container-based and serverless age of computing. Most large companies today have a bit of each of these eras represented, and now CloudBolt provides them with a unified, easy-to-use interface to provision, manage, and control all of the artifacts of past eras in a consistent way, and from a common web interface and API.
2020s – Hybrid, Containers, and Serverless
So that brings us to the next decade. The one thing we know for sure in today’s world is that the only constant is change.
Want to learn more? Download the CloudBolt Solution Overview or check out our videos for more info!