What it is and the value it drives
Over the past decade, cloud computing has removed the need for businesses to have their own IT infrastructure and radically changed the way companies deploy software. A decade ago, the platform on which to build software applications were enterprise operating systems deployed on proprietary servers– it is now “the cloud”.
Cloud refers to a rich ecosystem of technologies that vastly simplify the process of setting up infrastructure, databases, security, and integration services, and deploying software. The full continuum of technologies required to roll out applications is available in the cloud today, as exemplified by the exhibit below.
Front end. Code or tools to create user interfaces. Flexible programming frameworks such as React, for example, permit reusing the same code to build websites and apps; “no-code” or “low-code” solutions enable non-engineers to create simple applications quickly
APIs and back end. Software running on servers, constituting the core “business logic” of applications, e.g., to access data or execute transactions
Data analysis and visualization. Tools that allow users to connect to large data sets and run analyses and visualizations, often in the form of “notebooks” or dashboards
Databases, data warehouses, data lakes, and data storage. Traditional databases store data in logical “tables”; “bucket” data storage solutions are best used to store unstructured data, such as images or text. Data lakes and warehouses refer to large enterprise-wide catalogs of data that teams across a company can leverage for a diverse range of business applications
Security and monitoring. Suites of automated tools that monitor usage patterns of cloud systems to ensure correct functionality, detect anomalies, and possible cybersecurity breaches
Containers. A modern engineering concept that makes it possible to write platform-agnostic software, removing the need to tailor programs for the server or operating system they are deployed on
CI/CD pipelines. “Continuous integration” and “continuous deployment” tools enable programmers to safely deploy their code to users quickly and frequently without ad hoc human intervention
Data governance. Tools that ensure the validity and security of enterprise data, so that it can be reliably available for business applications, while protecting companies from inappropriate access or data corruption
Infrastructure. A service model, also known as “Infrastructure as a Service” (IaaS), where providers offer computing, storage, and network resources, removing the need for businesses to own and maintain server rooms or data centers
Platforms. Also known as “Platform as a Service” (PaaS), these include full-fledged solutions which companies can leverage to deploy software without worrying about configuring specific computing and storage resources
Collaboration. Modern productivity tools that enable sharing knowledge in “wikis”, communicating flexibly and instantly, and collaborating on shared documents and code.
An increasing number of companies are embracing the "cloud-native" paradigm, where the majority of their IT infrastructure is hosted in the cloud, whether private, public, or hybrid (also known as "multi-cloud"). The two prime candidates for cloud migration are modern web applications, which can be easily deployed via PaaS services, and data-driven use cases, which benefit from the scalability and cost efficiency of cloud. The exhibit below shows the continued shift away from on-premise and into various types of models from public cloud to multi-cloud.
Cloud computing provides value for organizations in four ways: agility and flexibility; innovation adoption; resiliency and quality; cost savings.
Agility and flexibility. With self-service tools and software components that "snap" in place, engineers can set up entire cloud architectures in days rather than weeks and automate maintenance. An idea that would take 20 months to be released can now be production ready in under five, and if entering a new market used to require 26 weeks of contracting and infrastructure deployment, it can now be done in as little as one. Deploying "cloud-native" architectures on public cloud infrastructure, companies can easily provide services in new countries, while safely isolating their customers' data.
Innovation adoption. The public cloud makes it easy to adopt innovative solutions at scale: enterprises can benefit from advanced use cases provided by the Software as a Service (SaaS) solutions on cloud platforms, from pipeline automation to advanced analytics to ML, which would be difficult to reproduce in-house.
Resiliency and quality. Cloud-based applications can be designed for high availability and resiliency, resulting in fewer incidents and reduced downtime: service continuity, in turn, increases customer satisfaction and minimizes negative business impact.
Cost savings. Companies no longer need to invest significant capital to purchase infrastructure; instead, they make use of scalable IaaS resources, and pay for the processing power of CPUs used or gigabytes of data stored, thus eliminating redundancy and trading off capital investment for operating expenses.
Where it is today
Adoption of the public cloud has been steadily growing, and the global market for IaaS and PaaS reached $233 billion in 2019. IDC identifies three players as established leaders: Amazon Web Services (with a 34% market share), Microsoft Azure (18%), and Google Cloud Platform (5%).
Beyond the top three, Alibaba holds a 5% market share, with adoption concentrated in the Chinese market. It is worthwhile to note how all major providers are concentrated in the US and China, with little competition from Europe or the rest of Asia. In a strategic bid to create a regional alternative, several European companies and governments have formed GAIA-X, a partnership aiming to deploy Europe-based cloud infrastructure by 2021.
Even though cloud technology is becoming prevalent globally, in 2019 Japan was classified as a "cloud-resistant" economy by Gartner. With smaller IT spending on cloud compared to other economies, it would take seven years to reach the US’s current level of cloud maturity, at the current pace of growth. Key reasons for this include deeply rooted legacy systems and processes, limited talent skilled in cloud architecture and migration, “vendor lock” contractual or financial restrictions, and data regulations.
In order to unlock the full value of a cloud migration, companies need sound and security-minded DevOps and Agile processes. When product teams are empowered to self-organize, allocate budget, and decide on product direction, they can move quickly and efficiently, making the most of the flexibility offered by the cloud. A similar transformation will be required of aspiring Japanese cloud champions.
A METI report warned in 2018 that Japan is approaching a "digital cliff", where the gap of available digital talent could reach 430,000 workers in 2025; moreover, companies report a significant shortage in cloud architects able to design company-wide transformations. So far, language barriers have deterred businesses from large-scale hiring of international experts but, as the talent gap widens, experienced professionals from overseas could be crucial in kick-starting the cloud journey and developing internal capabilities.
On the regulatory side, many companies have strict rules about data handling and software architectures, particularly in the financial sector. In the public sector, METI introduced ISMAP (Information System Security Management and Assessment Program) in 2020, an assessment framework that aims to create a centralized registry of cloud providers approved for government procurement, modeled after the analogous FedRAMP in the US.
However, this system has several shortcomings that make the adoption of cloud services by government agencies more difficult. First, while there is no fee to enter the registry, an audit by one of a few auditing companies is required, which according to executive interviews typically cost hundreds of thousands of dollars. Such fees may be prohibitive for companies of all sizes, and even larger multi-national companies may not be able to justify this expense, which competes with internal resources needed for other certifications. Second, despite this centralized system, cloud vendors report difficulties in complying with the multilayered system of regulations set by various prefectures and regions, which makes the process of qualifying for service complex and expensive.
While most Japanese businesses have yet to embark on cloud migrations, some local champions have led the way in recent years.
In September 2020, convenience store chain 7-Eleven announced the launch of Seven Central, its new data analytics platform, developed in partnership with Google. A move away from "data silos", this platform will allow the retailer to aggregate data from the POS in store by leveraging Google Cloud Platform's BigQuery and other data processing solutions and gain real-time visibility on its business performance.
In response to a major earthquake in 2018 that caused significant spikes in website traffic and access difficulties, Hokkaido Electric Power Company decided to migrate its websites to the cloud, in order to prevent malfunctions at critical times. Thanks to Azure's PaaS offering, the company was able to deploy a system within a few days, and the success of this use case inspired a cloud-driven update to the utility's digital strategy.
Since 2018, the Japanese government has been promoting Digital Transformation within the social sector. In support of this effort, NEC and Amazon’s AWS partnered in July 2020 to launch a cloud-based platform, which will enable government agencies to leverage modern cloud applications in compliance with security standards.
Cloud solutions are emerging in the healthcare sector as well, with the promise of unifying data records across hospitals, clinics, and patients. In September 2020, TIS announced the launch of its "Health Care Passport" project, developed with the Chiba University Hospital: the project aims to create an open platform that would allow healthcare professionals to connect and securely exchange clinical data.
How the technology will continue to evolve
Cloud technology will continue expanding across areas of the stack.
Cloud providers keep expanding their service portfolios, by enabling innovative use cases to be deployed as SaaS at a massive scale early on. Recent innovations have been around IoT infrastructure, ML as a service, quantum computing, and infrastructure monitoring: while these are often developed internally, acquisition of promising cloud-based startups has also proven to be an effective way for cloud providers to add new capabilities. Hybrid cloud solutions continue to expand, such as Morpheus, VMware, and CloudBolt. This development is likely to be underpinned by the increasing popularity of multi-cloud architectures, especially for critical use cases in heavily regulated sectors such as financial services. Sustained development in open-source software will drive innovation in writing, testing, deploying, and monitoring software. For example, one of the most popular solutions for deploying microservice architectures, Kubernetes, is an open-source project, supported by multiple developers and companies, including several cloud providers.
The key future applications
The realization that maintaining proprietary IT architecture is costly, complex, and often avoidable is creating strong business cases for cloud migration for players across every industry and geography, and it is likely that more Japanese companies will follow this trend.
In order for successful cloud migrations to occur, three key enablers need to be considered.
First, companies should have a clear strategy for cloud adoption, and understand how to securely leverage single or multiple cloud providers to maximize performance and savings. Second, they will need a sophisticated understanding of the financial aspect of cloud pricing, in order to negotiate contracts and leverage cloud solutions in the most cost-effective way. Third, synergies between technology and business teams will be crucial, in order to identify what use cases to migrate, and to fully capture the possible complexities of business-critical applications.
Enterprises should also be aware of the pitfalls when embarking on a cloud migration. First, a truly Agile organizational setup is required to allow product teams to make the most of the flexibility offered by self-service cloud infrastructure. Second, the advanced use cases supported by cloud providers will require specialized talent, without which these capabilities will be moot. Third, "lift-and-shift" migrations, i.e., replication of on-premise applications without re-architecting for the cloud, will not bring about savings or performance enhancements. Fourth, in order to unlock maximum availability and security offered by the cloud, a solid DevSecOps process is required, without which projects may experience more frequent issues and suboptimal applications.