History of Technology: From Mainframe to Hybrid Cloud
From Mainframe to Hybrid Cloud
For Tech’s Future, the Only Constant is Change
Computing infrastructure has come a long way in the last 50 years and the rate of change continues to rise. In order to inform our view of where infrastructure and computing platforms are going in the future, we need to take a look at the past and how far we’ve come…
1960s – Mainframes & Timesharing
During the mainframe epoch, companies had only a few large computers, occupying entire rooms. They usually required physical access to use (though logging in over phone lines with primitive remote terminals started to emerge in the ‘60’s). Maintenance of these machines required several physical operators, and automation of this maintenance was not widely considered.
1970s – Advent of Personal Computers
During the ‘70’s, computers, started showing up on people’s desks, albeit in a limited manner, instead of filling entire rooms. Administration of these ‘desktops’ was still done locally.
1980s – Networks Arise
The ’80’s saw a growth, dare I say widespread connectivity of computers, including modems appearing everywhere, DNS’ creation in ’83, usenet, gopher, and even the inception of the World Wide Web (WWW) in ’89. Still, servers that large organizations owned and operated remained on a relatively small scale, and administration of these servers was done on a one-off basis.
1990s – The Web and the Proliferation of Physical Servers
The ’90’s saw drastic change to computing infrastructure. The first popular web browser (Mosaic) was released in ’93 and suddenly more servers were needed than before. Companies quickly moved from having 10’s or 100’s of servers, to having tens of thousands of servers.
This shift required a new approach to server management. No longer could you hire one system administrator to manage each server, (or a few), instead they each needed to manage hundreds, an insurmountable task without automation. Cfengine was released as the first configuration management tool in 1993, but the demand for automation greatly outpaced available solutions.
2000s – Virtualization
In the early ‘00’s, The maturation of Linux catalyzed a shift from traditional, proprietary Unix systems (such as Solaris, HP-UX, and IBM AIX, which ran on proprietary hardware) to RedHat and other Linux variants, running on commodity hardware. This was fueled by, and in turn further fueled the expansion of the number of servers that needed to be managed.
The early ‘00s saw the emergence of a new class of products called Data Center Automation (DCA) products, including Opsware and Bladelogic, however, Sun and HP had their competitors too.
Virtualization became accepted in the mid-’00s, first in dev/test labs, but increasingly in production environments. This let companies slow the growth in physical hardware, while still adding virtual servers, each of which with its own running OS. This resulted in more operational efficiency, but management nightmares abounded as it became harder to track, patch, upgrade, and secure all of the operating systems running in a datacenter.
More mature configuration management tools rose from this chaos, notably Puppet and Chef (founded in 2005 and 2008, respectively), and the term DevOps was coined in 2007.
2010s – Public Cloud
While AWS was released (in non-beta form) in 2008, it wasn’t widely adopted as a platform for enterprise computing until the 2010s. Suddenly, all of the frustrations of dealing with one’s own data centers could be solved with a credit card. No more waiting for the IT team to create your VM, no more dealing with the possibly obstructionist networking, database, or security teams…
Companies throughout the 2010s have shifted back and forth between bearishness and bullishness on the public cloud (often depending on how recent and shocking their last bill was). However, serious, enterprise IT shops are realizing that hybrid cloud is truly the best solution, and the end goal. Hybrid cloud enables them to use a mix of public clouds and their own datacenters, choosing the best environment for each workload. Hybrid also allows IT admins to use public cloud during times when demand is above average, scaling back down when demand subsides, so they do not wind up with a gigantic bill.
The Advent of a Hybrid Cloud Management Platform
The first pre-release of what is now CloudBolt was created in 2010. The gap that my co-founder Auggy and I saw was that, in larger companies, the interface to the IT organization was broken. Developers (and other folks who needed IT resources), would submit a ticket to IT, and then wait weeks to get what they needed (often a VM, but sometimes a physical server, network change, storage allocated, etc). This problem was exacerbated in the ‘90s and ‘00s with the rise in demand of access to servers and VMs, and was made highly contentious by the advent of public cloud (“If I can get a VM in minutes from AWS, why do I have to wait weeks for my IT people to get me one?!”).
CloudBolt brings the experience of using a company’s private datacenter up to par with the public cloud experience, and spans the eras from the physical server age, through VMs and public clouds, to the container-based and serverless age of computing. Most large companies today have a bit of each of these eras represented, and now CloudBolt provides them with a unified, easy-to-use interface to provision, manage, and control all of the artifacts of past eras in a consistent way, and from a common web interface and API.
2020s – Hybrid, Containers, and Serverless
So that brings us to the next decade. The one thing we know for sure in today’s world is that the only constant is change.
Want to learn more? Check out our Solution Overview or check out our videos for more info!
Case Study: US State Government Eliminates VM Sprawl
[vc_row][vc_column][vc_column_text]
US State Government
- Department: Centralized IT Team
- Location: United States
[/vc_column_text][/vc_column][/vc_row][vc_row][vc_column width=”1/2″][vc_column_text]
Their Challenge
The state IT organization struggled with centralizing a large number of virtualization resources that were spread across separately managed vCenter and XenServer clusters. VM sprawl was costing the state thousands of dollars a month in infrastructure and licensing charges, and delays in resource provisioning resulted in a sharp increase of public cloud-based shadow IT environments that put sensitive agency data and security at risk.
Their Technology
- Virtualization: VMware vCenter, Citrix XenServer
- Cloud: Amazon Web Services, Microsoft Azure
- Configuration Management: Puppet
- Operating Systems: RHEL, CentOS, Windows
CloudBolt’s Competition
- VMware vCloud Automation Center
- Embotics
- Red Hat CloudForms
[/vc_column_text][/vc_column][vc_column width=”1/2″][vc_column_text]
CloudBolt’s Solution
Summary
- Provided self-service IT to consumers
- Reduced VM sprawl
- Enabled selecting best-of-breed configuration management platform
- Provided accurate chargeback reports
- Enabled IT org to increase agility by adding new service offerings
Eliminate VM Sprawl
CloudBolt’s ability to ingest information about all of the VMs running across the state enabled the IT team to identify high-cost VMs that were being under-utilized. Unneeded VMs were terminated, freeing underlying server resources and software licenses.
Self-service IT reduced shadow IT
Shadow IT refers to unsanctioned IT solutions that grow in reaction to an IT organization’s inability to meet their consumer’s needs. Using CloudBolt to provide a self-service portal that automates the end-to-end provisioning and management of servers helped meet IT consumer’s need to receive on-demand access to servers.
Flexibility to choose configuration management solution
The state IT team wants to move past managing multiple VM templates and manual management of software application installation and configuration. They’re currently evaluating Puppet and Chef. CloudBolt’s integration with many industry-leading technologies gives them the ability to select the configuration manager that is best for their specific use cases.
Chargeback and showback reporting
The state’s previous chargeback reporting required significant staff effort to collect data from multiple systems, correlate it, and then assign it to the appropriate group for invoicing. CloudBolt streamlined this entire process, savings dozens of staff hours each month.
Increased agility and automated testing
Implementing CloudBolt enabled the IT organization to focus on tasks and projects that enabled them to create new offerings for their IT consumers, who can easily select from a list of available offerings from CloudBolt’s service catalog. Through the Continuous Infrastructure Testing (CIT), CloudBolt automatically tests the underlying service offering components to ensure they’re working properly, improving the end user experience.
Customer Findings
- Attained goal of sub-10 minute provisioning
- Rapidly eliminated VM sprawl
- Created an “IT vending machine” from existing virtualization environments
- Gained control of environment governance
- Easier to install and maintain than the competition
- Integrating chargeback exposed gaps in previous reporting
- Can grow into a configuration management tool of choice
The state’s evaluation and selection of CloudBolt was an easy one. Rather than “select the best of the worst”, they continued to search for a cloud manager that was both easy to install and configure, but would also grow with their requirements. Of the evaluated competition, only CloudBolt effectively integrated their existing infrastructure, making it easy to model and apply a rate structure.
The CloudBolt rate engine enabled the state to offer different rates on a per-environment basis. On install, CloudBolt reported a large cost delta between what the customer expected and what CloudBolt actually reported. After a bit of investigation, CloudBolt was deemed to be correct: the IT team discovered several TiB of allocated storage that had been routinely missed from manual chargeback accounting.
CloudBolt’s intuitive interface enabled the state to rapidly develop new offerings for their end users. The state could easily modify the ordering process to match the level of understanding of the end user, meaning the state could expand the self-service IT portal to serve nearly everyone who regularly requested IT resources or applications.[/vc_column_text][/vc_column][/vc_row]
[vc_row][vc_column width=”1/2″][vc_column_text]
Large Software and Services Company
Employees: 2,000
Annual Revenue: $450M
Location: United States
Their Technology
Virtualization: VMware vCenter, Amazon Web Services
Configuration Management: Chef (multiple instances)
Orchestration: vCloud Orchestrator
Ticketing: ServiceNow
IPAM: InfoBlox
Operating Systems: RHEL, CentOS, Windows
VMs: 7,000+
CloudBolt’s Competition
- VMware vCloud Automation Center
- Homegrown Application
- Red Hat CloudForms
[/vc_column_text][/vc_column][vc_column width=”1/2″][vc_column_text]
Their Challenge
Before CloudBolt, the company relied on a complicated integration of vCloud Orchestrator, Chef, and a ticketing system to run their server provisioning processes. They struggled with slow turn-around time, a lack of environment governance, and difficulty understanding the monetary costs of their infrastructure.
CloudBolt’s Solution
Summary
- Dramatically faster VM provisioning and hosted customer onboarding times
- Showback/Chargeback with cost tracking and transparency across lines of business
- Policies and quotas for restricting usage and access to resources
- Policy-conformant deployments
- Real-time software license management and tracking
- Self-testing environment
Building a Hybrid Cloud with CloudBolt didn’t mean starting over
Beyond discovering the company’s pre-existing servers, CloudBolt also connected to their Chef installation and automatically imported the available Chef recipes, cookbooks, and roles. The Chef content was then ready to re-use from within CloudBolt as applications that could be added and removed to servers. CloudBolt did the same for their vCloud Orchestrator environment, exposing the flows so that they can be executed manually or automatically at any point during a server’s lifecycle.
Self service IT reduced shadow IT
Shadow IT refers to unsanctioned IT solutions that grow in reaction to an IT organization’s inability to meet their consumer’s needs. Using CloudBolt to provide a self-service portal that automates the end-to-end provisioning and management of servers helped meet IT consumer’s need to receive on-demand access to servers.
Faster turn-around with smart automation and integrations
CloudBolt’s orchestration hooks feature enabled the company to tightly integrate with existing components of their flow. For example, CloudBolt was made to use InfoBlox for IP address management during provisioning and modification of networks. Orchestration hooks also enabled CloudBolt to create ServiceNow tickets when provisioning requests were approved, and to update those tickets as servers are modified or deleted. CloudBolt allows existing infrastructure to be partitioned into “environments” by their hypervisors, networks, applications, OS templates, and more. These environments were exposed to end-users via simple order forms and a unified API that let them request servers and more– without regard to where those resources would be deployed. The time required to provision a single VM was reduced from 1 business day to just 15 minutes. The time to onboarding new hosted clients was reduced from 7 days to under 30 minutes.
Customer satisfaction through better availability
Because the company utilized CloudBolt’s multi-tenant capabilities to expose provisioning capabilities to their hosted customers, it was critical that every step of the process worked perfectly. They used CloudBolt’s Continuous Infrastructure Testing (CIT) to perform end-to-end testing of the entire provisioning process. If a hypervisor or orchestration tool fails, CIT’s early warnings give system administrators the time to proactively address the issue before end-users have a chance to notice a problem.
Improved margin reporting and pricing
Interactive reports of infrastructure usage allowed IT to provide accurate actionable information to business about the cost of resources required to develop various products, which is used to set pricing and report on margins. IT was also able to differentiate high-value external clients from the ones that require more resources than what they are currently paying for.
Customer Findings
✦ Time-to-value — 2.5 days to implement
✦ Simple and powerful solution
✦ Lower OPEX — CloudBolt was less expensive to operate and maintain
✦ Capability — CloudBolt successfully met all 8 use cases out of the box
✦ Flexibility — CloudBolt provided customer with maximum choice
✦ Governance — Unique cost & license tracking unify management visibility
✦ Value — Unique features such as license management and self testing added significant value
During the company’s evaluation of CloudBolt, they were impressed by the speed of implementation. After day one, CloudBolt was integrated with Chef, vCloud Orchestrator, and vCenter. After day 2, all required use cases had been met. Subsequent days were spent delivering nice-to-have integrations with the company’s workflows by using CloudBolt’s upgrade-safe orchestration hooks feature.
Compared to the competition, the customer preferred CloudBolt’s powerful yet intuitive interface and API. The consumable interface meant that CloudBolt would not require significant ongoing maintenance to make changes and alterations as the customer’s needs evolved, and the API meant that devops teams could access IT resources programmatically. The seamless upgrade process was also attractive: CloudBolt is updated just as easily as it is installed, and even demonstrated that customizations were upgrade safe.
Overall, the customer cited a combination of CloudBolt’s capabilities as being both compelling and accessible. Coupled with the superior OpEx, and a time-to-value that was measured in hours, the choice was simple.
[/vc_column_text][vc_single_image image=”30608″ img_size=”full” alignment=”center”][/vc_column][/vc_row]
[vc_row][vc_column width=”1/2″][vc_column_text]Public University and Research Institution
Location: Eastern United States
Their Technology
Virtualization: VMware vCenter
Public Cloud: Amazon Web Services, Acquia Cloud Platform for Drupal
Configuration Management: Puppet, Homegrown Scripts
Operating Systems: RHEL, CentOS, Windows
VMs: 1500+
CloudBolt’s Competition
- VMware vCloud Automation Center
- Red Hat CloudForms
[/vc_column_text][/vc_column][vc_column width=”1/2″][vc_column_text]About the University
The University Office of Information Technology’s cloud team is responsible for fielding and supporting cloud-enabled technologies to faculty, staff, and students. They attempt to support instruction whenever possible.
Their Challenge
The cloud team is tasked to implement various cloud-based services, but was challenged to expose them to students and staff in a controlled and reliable manner. They needed a unified interface that would allow end users to find and request cloud services regardless if the services were provided by a public cloud provider or by existing university assets, and enable tracking and reporting of resource usage to better understand the impact of their work
CloudBolt’s Solution
Summary
- Automate provisioning and management of IT resources to faculty, staff, and students
- Present a tightly-controlled but flexible self-service IT platform
- Centralize control of resource utilization and reporting
- Eliminate VM sprawl
- Re-purpose surplus corporate IT to advance educational mission
- Flexible and burstable CloudBolt license model makes for predictable cost
Empower educators with surplus IT
With CloudBolt, the university was able to re-purpose surplus capacity from their corporate environments, and put it to use by educators — enabling students to access server and application resources to support their class work.
Customer satisfaction through better availability
As the demand for compute resources in support of Big Data and other compute-intensive workloads increases, faculty and staff are impatient in getting access to additional resources. CloudBolt replaces time-and resource-intensive manual processes by allowing end-users to drive the provisioning and server management process on their own.
End-to-end automation
Automating the routine provisioning and management meant systems administrators could do more with less, increasing the server to administrator ratio. Removing the people from the process also improved compliance, reduced OpEx costs, and significantly increased the pace at which the IT team was able to offer new services.
Cost and utilization transparency
Once CloudBolt imported the customer’s existing environments and servers, the university was able to determine which colleges and departments were using resources. The university was then able to re-assign budgets to better support demonstrated need.
Self-service access to non-traditional resources
Beyond just provisioning VMware and AWS EC2-backed servers, CloudBolt was easily configured to create S3 storage buckets in AWS, as well as provision Drupal accounts in the Acquia Cloud Platform for Drupal.
Multiple authentication points
University faculty and staff authenticate against a different Active Directory endpoint than the student population. CloudBolt’s multi-tenant capabilities ensure that every user is authenticated to the correct environment, and presented with the expected branding when logged in.
Customer Findings
✦ Seamless support of instructional needs
✦ Consolidated shadow IT systems reduces costs
✦ Student environment usage tracking, complete with automatic cleanup at semester’s end
✦ Rapidly integrate unique infrastructure and platforms offerings
✦ Multi-tenant solution unified management of disparate groups’ environments
✦ Provide access to enterprise instructional technology that mimics what students encounter in the career marketplace
CloudBolt’s selection was based on CloudBolt’s unique ability to rapidly integrate with non-traditional service providers. In the University’s case, they needed to make cloud-provided Drupal instances available to end users in an on-demand basis, while still accounting for cost transparency and access rights. While the competition claimed to be able to rapidly do this, CloudBolt was the only tool that was able to demonstrate this capability in the University’s own environment. CloudBolt enabled the University to consolidate shadow IT environments that groups turned to when they were unable to get access to centralized resources in a timely manner. Providing on-demand centralized access, management, and provisioning of IT resources also enabled the University to achieve better economies of scale as they extended their offerings to a larger faculty, staff, and student population. Furthermore, the ability to add elastic public cloud-based resources enabled the cloud team to flexibly account for peaks in demand based on enrollment. Integrated reporting ensured accurate cost accounting.
The cloud team’s use of CloudBolt to provide on-demand IT resources from a variety of managed environments has made them the de-facto IT architecture thought leaders at the University. IT teams such as the Windows group now look toward the cloud team’s offerings as the reference architecture for how to properly implement cloud-based services in varied and disparate environments. Thanks to its selection of CloudBolt, the cloud team has increased their organizational capital.
[/vc_column_text][vc_single_image image=”30608″ img_size=”full” alignment=”center”][/vc_column][/vc_row]
[vc_row][vc_column width=”1/2″][vc_column_text]
Consumer Retail Company
Company: Consumer Retail
Employees: 25,000
Annual Revenue: $3.5 billion
Location: United States
Their Technology
Virtualization: VMware vCenter, OpenStack, HP Virtual Connect
Configuration Management: Puppet
Continuous Integration: Jenkins
Orchestration: Rundeck
Ticketing: ServiceNow
IPAM: InfoBlox
Operating Systems: CentOS, Windows, Ubuntu
VMs: 1,800+
CloudBolt’s Competition
- VMware vCloud Automation Center
- Cloud360
- Red Hat CloudForms
- HP CloudSystem Matrix
[/vc_column_text][/vc_column][vc_column width=”1/2″][vc_column_text]
Their Challenge
At this company, the IT group primarily services the e-commerce group, which is responsible for the web storefronts of all the corporate brands, and for a significant portion of the company’s revenue. To speed delivery of new and innovative capabilities, the e-commerce team needed a platform that would allow on-demand access to production and development/test resources, with high compliance to PCI and other standards. Initially, they experimented with a homegrown solution based on OpenStack, but this was rejected by the CIO because it failed to effectively take into account needs of other lines of business. The potential for infrastructure fragmentation, extra costs, and large amounts of duplicated work caused this company to search for a better solution.
CloudBolt’s Solution
Summary
- Aligned IT and e-commerce teams
- Unifies management and access through UI and API
- Enables IT to offer customize offerings to different lines of business
- Deployed services meet strict standards and configuration requirements
- IT regains the ability to make effective technology decisions
Aligned IT with lines of business
CloudBolt enabled the IT team to offer distinct solutions to various lines of business without adding significant OpEx. Increased service to the lines of business ensures the IT team is a partner to success rather than merely a service provider, and ultimately results in more productivity and innovation from all parties.
Expanded configuration management
CloudBolt’s integration with Puppet configuration management made it simple to pair VMware virtualization with Puppet configuration management. Even non-technical users can now use the Puppet resources and applications IT developed because they’re exposed as installable and manageable applications on requested servers. The company now spends less time managing dozens of VM templates, and more time realizing the added value of effective configuration management.
Provided IT flexibility in technology choice
The customer has a fairly large amount of legacy infrastructure in the form of existing VMs, deployed applications, and other stand-alone physical server environments. CloudBolt has become the foundation for a transformative IT process that will enable IT to begin deploying services that can help them retire legacy infrastructure that carries along significant OpEx requirements. CloudBolt also enabled the IT team to reduce their dependence on single-vendor virtualization solutions, saving money, and reducing risk to the business.
Obsoleted their expensive HP CloudSystem Matrix deployment
Even after significant investments in professional services and painful upgrade experiences, HP CloudSystem Matrix was not able to meet this company’s requirement of a user-ready portal. When the company started investigating CloudBolt, they were excited by the contrasts: painless upgrades and a feature-set thoughtfully designed to meet their needs.
More accurate test environments
When working to test new software releases, the e-commerce team had to rely on environments that were not exact replicas of production systems. By using the CloudBolt API, the
e-commerce team enabled their continuous integration environment to use production-replica systems to perform build and functionality tests, which increased productivity, as well as code quality.
Customer Findings
Summary
✦ Rapid time to value — up and running in less than a day
✦ Easy installations, customizations, and upgrades means CloudBolt grows with them
✦ High-value product roadmap will allow further innovation
✦ Immediate delivery of test environments reduced time to deployment by 50%
✦ Compelling price point with low barrier to entry
CloudBolt’s ease of use was one of the customer’s favorite features. A key reviewer of CloudBolt mentioned that they felt more confident with users ordering servers from CloudBolt than with any other system. After struggling to use CloudForms and Cloud360, the reviewer felt that they were not viable portals for end-users and that both required too much expensive customization to meet requirements.
The well rounded feature-set that CloudBolt provided helped them re-unify their infrastructure and allowed them to avoid investing more into their failed HP CloudSystem Matrix deployment by dropping it entirely. This company now has a self-service portal that allows IT to be successful in providing their consumers simple access to needed resources.
[/vc_column_text][vc_single_image image=”30608″ img_size=”full” alignment=”center”][/vc_column][/vc_row]
Neustar Case Study
CloudBolt – IHG Case Study