CLOUD MIGRATION
CLOUD DATA WAREHOUSE ARCHITECTURE
Heritage reporting and analytics systems that rely on a monolithic on-premises data warehouse appliance will not be able to satisfy most organizations’ rapidly increasing demands for reporting, business intelligence, and analytics. The future is in the cloud, and there is no doubt that the purported benefits using cloud computing services influence the decision to modernize the enterprise data architecture. Many organizations are modernizing their environments by migrating data and applications to the cloud and for good reasons, such as:
Cloud Economics
The traditional approach to on-premises hardware acquisition involves estimating peak system loads and sizing an acquired system that exceeds the anticipated peak demand. This approach drains corporate capital yet acquires computing bandwidth that often remains unused. When moving to the cloud, your organization only pays for the cloud services it uses, enabling an Operational Expense (OpEx) model that appeals to the corporate finance team.
Performance and Scalability
Cloud-based resources can be configured to automatically expand to meet the performance needs.
Simplified Management
Cloud service providers (CSPs) handle the management and maintenance of the computing environment. This eliminates the need for space, power, and cooling to support on-premises hardware configurations.Innovative Value-add Services
Cloud vendors have accelerated the development of services supporting advanced analytics, such as machine learning and artificial intelligence capabilities that are easily integrated into applications.
The cloud revolution provides the perfect opportunity to modernize how your organization’s applications meet the current and future business process requirements. In this paper, we begin with an overview of the rapid evolution of hybrid cloud data architectures that span multiple cloud instances across different cloud service providers (CSPs) and SaaS providers as well as existing on-premises systems and data. We consider some key challenges of migrating to the cloud, and why while the “lift and shift” may provide short-term value, ultimately it does not afford the opportunity to refactor old processes and bring them up to date. Alternatively, migrating to the cloud provides an opportunity to rethink what your business processes are and how the data and applications support those processes. However, to reduce the complexity of renovation and modernization, you need to understand your enterprise data landscape, and that provides an opportunity to review the characteristic data governance and modeling technologies that support process renovation through cloud migration.
HYBRID MULTICLOUD DATA ARCHITECTURES
The future enterprise data architecture uses the cloud as a platform for data warehouses and data lakes, but why should your organization be constrained to only using one type of virtual cloud environment provided by a specific cloud service provider? Organizations should be free to migrate their capabilities to more than one cloud environment using the same CSP, or even use services from more than one CSP, Software as a Service, and Platform as a Service (PaaS) providers.
Different cloud service providers provide different services and capabilities, and the same economic and performance factors that has driven organizations to the cloud are often employed to seek out the best of breed across a variety of cloud service providers and vendors for particular types of services, and balancing costs vs. performance. When the organization has a clear understanding of key performance criteria and how the architecture can be designed to optimize for those criteria, hybrid environments can be used to provide the broadest support for end-user/analyst needs (see Figure 1).
CHALLENGES OF CLOUD MIGRATION
The drive for cloud modernization has driven project managers and their teams to migrate data and processes out of the traditional on-premises data center and to a cloud-based environment. Yet despite the promises of lowered costs and simplified administration, the process of cloud migration is complex, making it difficult to realize the anticipated savings and reduction in administrative effort. In other words, there are risks in attempting to completely move all systems at once that can complicate the migration effort.
The rapidly accelerating introduction of new technologies and the demand for cloud migration can confound data practitioners whose job is to migrate operational processes and accompanying data to a new cloud-based platform. This complexity is compounded by the increasing breadth of the data consumer communities including traditional data analysts as well as more sophisticated data scientists, an increased demand for operational data pipelines, and the need to speed data onboarding and integration lifecycles. Some specific challenges include:
- Growing data chaos that emerges as more data sets are distributed across different cloud host object storage services.
- Unexpected data costs associated with repeated requests for the same data assets,replicated invocation of similar services from different CSPs, and replicated computing.
- Inability to provide enterprise-wide data awareness of all the data assets managed both on-premises and across the different CSP storage services.
- Performance bottlenecks associated with cross-cloud data latency.
- Increased data integration complexity when data sets are piped across cloud platforms.
When the organization has a clear understanding of the key performance criteria and how the architecture is designed to optimize for those criteria, a hybrid cloud environment fits the bill. Just be aware that as hybrid cloud environments offerings continue become more complex, data practitioners will require additional support to maintain data awareness across the data landscape.
“LIFT AND SHIFT” CLOUD MIGRATION
Organizations with operating systems might consider simply replicating their existing system’s footprint directly using the same system architecture on the cloud platform. This “lift-and-shift” approach involves configuring a cloud-based image of the software stack supporting the on- premises application deployed on a cloud provider’s computing platform. This approach is presumed to minimize migration risks because no changes are made to the system.
While lift-and-shift does accomplish the goal to migrate to the cloud, it does little to improve the way their business processes perform. The reason is that existing systems are likely to have been built a long time before and are constrained by the extent of what the IT developers knew about the business process when the application was developed. In some cases, the existing applications are decades old, using technologies that have radically changed in the interim. If you want to lift and shift a 20- or 30-year-old system to the cloud, you will face challenges that limit the overall value of migration, such as:
- Cost: The cost of the cloud services may also turn out to be shocking/prohibitive, as people underestimate both the computing resource costs that, when unmanaged, can spiral up rapidly, as well as the costs of data storage and importantly, data egress.
- Effort: When you lift and shift, you extend significant effort in replicating the environment, ensuring that the right versions of tools are installed and working, moving the data, and implementing and testing the application on the new platform. However, after all this effort, you are no closer to having renovated the application than you were before.
- Missed opportunities for improvement: Lifting and shifting deploys the existing application on a cloud platform but will not enhance your application with any new capabilities. You have not modernized or enhanced the application to address new and emerging business scenarios, and you will miss out on performance if the underlying software stack is not designed to take advantage of cloud storage, computing, and other native services.
CLOUD MIGRATION MODERNIZATION
When your organization has made the decision to migrate their applications to the cloud, instead of lifting and shifting, use migration as an opportunity to renovate and modernize the application landscape. Modernization involves a concerted effort in evaluating business requirements and devising an extensible architecture that establishes the framework for future innovation. That process incorporates these steps:- Differentiate between the business process and the application implementing that business process. As legacy applications age and the original developers have long disappeared, existing team members often conflate the application with the business process it was originally designed to implement. This perilous approach creates artificial barriers to renovation, so by separating the process from the code frees the designers to reconsider alternatives for modernization.
- Understand the key business drivers of today. It is doubtless that the business process requirements have changed since the original application was developed. Engage the existing business process owners and review their perceptions of the existing business process equirements.
- Consider tomorrow’s future business drivers. At the same time, ask the business process owners their opinions about how the requirements will change in the near and longer-term timeframes.
- Assert a data architecture that is engineered to adapt to a hybrid cloud environment. That includes determining which cloud-native resources and services that best meet current and future needs.
- Identify the key techniques and methodologies that are necessary to support current and future business needs.
Taking these steps will afford the ability to remodel the data management environment to take best advantage of the cloud services and optimize the environment to avoid unexpected costs, unnecessarily hampered performance, and increased system complexity.
CLOUD MIGRATION STRATEGY
The key to effective environmental and application refactoring is through raising the level of data awareness. The modernization process assumes some degree of proficiency in enterprise architecture and modeling. However, this sometimes can be a challenge. While many of the individuals involved in a cloud migration strategy are data specialists, data engineers, and data scientists, these roles do not generally demand a level of expertise in data modeling. Yet the different facets of data modeling can help these practitioners even if they have not done it before. In particular, it is valuable to gain some level of familiarity with the types of modeling techniques.
TYPES OF DATA MODELING TECHNIQUES
Business process modeling, which enables analysts to develop graphical representations that illustrate the organization’s business processes and provide a means for communicating, reviewing, analyzing, and making decisions about how the organization runs the business.
Data discovery, which combines coarse-level discovery of data assets and their object- level metadata (such as their names, owners, size, last update, location) as well as analyzing the contents, ascribing a categorization to the data assets, and document what has been learned in a data catalog.
Metadata management, which provides fine-grained information about the data elements, structured data assets, as well as the contents of semistructured and unstructured data.
Data lineage, which maps the flow of data from the origination points coming into the environment to the various destinations coupled with descriptions of the processing applied along the way.
Database structure, which details the physical structures of the data assets across the environment.
All of these different aspects of data modeling provide a way to understand the data landscape and simplify the process of both migration of data assets from the on-premises environment to the cloud as well as establishing data awareness across the hybrid cloud environment. Establishing good practices for managing data standards and operational norms in the enterprise helps solidify end-to-end data pipeline management, increase levels of confidence in consistency in final products, and reduce overall complexity.
CLOUD COMPUTING CONSIDERATIONS
Cloud computing is quickly becoming the standard for enterprise data architecture, especially for reporting and analytics applications. The cloud provides reasonable economics while supporting the performance, scalability, and simplified management expectations of the modern environment. And as more cloud service providers differentiate their service offerings, organizations are likely to evolve hybrid multicloud environments that take advantage of cloud- native services.
Migration to the cloud poses some challenges, though, and although the default approach is to “lift and shift” existing systems from their on-premises environments to a similar cloud-based configuration, this approach poses unexpected risks causing increased costs and complexity, yet does not lead to any significant differentiation of the application’s capabilities. Instead, moving to the cloud affords the possibility of renovating and modernizing enterprise applications.
The best way to modernize is to reframe the contexts associated with how applications support business processes. You can employ modeling techniques to support the modernization process, and it is beneficial to assemble a suite of tools that not only enable experienced modelers, but also help engage a variety of different personas and roles to support enterprise application renovation. Look for vendors that provide enterprise-grade tools that incorporate techniques for business process modeling, data modeling, data discovery, metadata management, data lineage, and data catalog capabilities to support the renovation and modernization effort.
ER/Studio Data Architect
Document, design, and communicate data assets faster and easier. Round-trip database support gives ER/Studio Data Architect users the power to easily reverse- and forward-engineer, compare and merge, and visually document data assets across multiple relational, NoSQL, and ETL platforms and data sources. Built-in facilities automate routine data modeling tasks so users can analyze and optimize database and data warehouse designs faster than ever.
Start a free, fully-functional, 14-day trial today!
IDERA Database Tools for the Cloud and Hybrid Environments
Powerful Technology for Efficient Database Management Be proactive not reactive, by finding and fixing issues before they negatively affect the organization. Whatever the cloud configuration, IDERA provides a wide range of database tools to help you manage and monitor your environment. Learn how IDERA’s solutions help you in the cloud.