How Japan Can Deal With AI Energy Demand

Japan’s semiconductor prowess key to breaking free of power constraints

15 November 2024

Japanese version: 1 August 2024

Kentaro Abe
Center for Policy and the Economy
Naoki Nishikado
Center for Policy and the Economy

Generative AI’s rapid advances of late are causing worry as to whether we will ever be able to satisfy its seemly insatiable appetite for energy. This is especially true amid global ambitions to also achieve carbon neutrality—the anxiety is rising worldwide. But the appearance of new challenges is also the appearance of new opportunities for growth. Key will be technical development of hardware and software that will enhance power efficiency. Japan, with its advanced technology in these areas, cannot afford to miss this golden chance to transform its power-constraints challenge into something game changing.
Quantitative analysis sheds light on the implications of generative AI on the future of electricity supply and demand in Japan. The challenge to make generative AI more energy efficient can serve as a perfect opportunity to jumpstart a recovery in the international competitiveness of Japan’s digital industries.

New issues emerge from wider presence and growing size of generative AI

ChatGPT had 140 million unique visitors1 from around the world shortly before the first anniversary of its November 2022 release. Going forward, generative AI promises to be a huge driver of change in industry and society as it gets put to work in diverse applications in just about every industrial sector. Meanwhile, concerns are rapidly mounting about yet another challenge: an increase in power consumption fueled by demand from power-hungry ICT-sector infrastructure, like network facilities and data centers. Behind this are two issues: Generative AIs’ spread (increase in use) and their exponential growth in size (increase in number of parameters).
Figure 1: Schematic of burgeoning power consumption due to generative AIs’ growing presence and size
Schematic of burgeoning power consumption due to generative AIs’ growing presence and size
Source: Mitsubishi Research Institute, Inc.

1. Generative AI’s presence (increase in use)

Our forecasts show that in 2040 networks will be transporting approximately 350 times the volume of data they carried in 2020.2 Further, with larger use and presence, generative AI will come to process ever more data.

For example, take services delivered by smartphone catering to people’s routine needs: Down the road, the combination of generative AI and smartglasses promises to offer advanced personal-assistant services combining wearers’ activity history and what they see in front of them. There is also the possibility of generative AI-powered robots and customer-service avatars, which number in the hundreds of thousands in Japan alone, taking over interactions requiring conversation.

2. Growing size of generative AIs (increase in parameters)

The foundation models,3 which constitute the core of generative AI technology, continue to grow in size. Generative AI is subject to scaling laws,4 which say that a model’s performance improves proportionately to the number of its parameters (variables that can be set for training it) and other factors. Increasing parameters two or three orders of magnitude, combined with the improving performance of hardware like GPUs,5 allows developers to enhance their generative AIs’ performance dramatically.

Increasing use and parameters each lead to increases by magnitudes in the volume of computations generative AIs perform when inferencing. Their synergistic effect means that the volume of computations at data centers will reach a level in 2040 that is 5 orders of magnitude greater than today’s. Generally speaking, if power efficiency remains constant, power consumption would rise proportionately to computational volumes. So it is obvious that an explosive increase in the volume of computations by generative AI could lead to an explosion in power consumption.

How to keep power constraints from hindering generative AI utilization

Better power efficiency, and in turn development of related technologies, will be required to maintain utilization of generative AI amid power concerns.

Considering domestic power supply margins cited in current power-facility plans,6 Japan’s total electricity demand in 2040 should be kept below 1.5 times today’s level. We estimate that current ICT-sector consumption accounts for about 2 percent of total domestic demand. Taking electrification of other industries into account, we believe it will be necessary to keep the ICT sector’s 2040 consumption below 13 times its 2020 level.
Figure 2: Total domestic power generation and ICT-sector power consumption maximums
Total domestic power generation and ICT-sector power consumption maximums
Source for 2020 (historical) figures: Mitsubishi Research Institute, Inc. from IEA and Japan’s Agency for Natural Resources and Energy
Source for 2040 figures: Mitsubishi Research Institute, Inc. forecasts
The role of semiconductor technology development plays a particularly important role to that end.

We estimate that innovations in semiconductor technology could keep growth of ICT-sector power demand between five and thirteen times its current level. This holds true even in the case of a computational volume 5 orders of magnitude larger than today. This would make it possible to pursue full-throttle utilization of generative AI within projected power constraints. There are several technologies with potential to boost the power efficiency of semiconductors, giving Japan a way forward.

Further, if Japan can succeed in developing and spreading more power-efficient generative AI models, it could even be possible to keep ICT-sector power consumption to about only twice its 2020 level.

*Mitsubishi Research Institute, Inc. “インサイト:生成AI電力効率化を導く半導体開発の最前線” (Insight: Pioneering developments in semiconductor technology leading generative AI power efficiency)

Powering generative AIs with local renewables

From a carbon neutrality standpoint, utilizing renewable energy is just as important as reducing power consumption. Many types of generative AI computations can be done anywhere, so the work could be distributed across data centers in remote locations. Exploiting this characteristic of these tasks enables maximum use of locally produced renewable energy.
Figure 3: Utilizing local renewables to power generative AI computing
Utilizing local renewables to power generative AI computing
Source: Mitsubishi Research Institute, Inc.

1. Powering generative-AI training with surplus renewable energy

One option worth contemplating is utilizing surplus solar and wind energy, which are prone to big variations in availability, to power the training of generative AIs. If wind farm operators in Hokkaido were to anticipate a surge in output due to strong winds, people training AI in Tokyo could shift their training processes and data to data centers in Hokkaido. That would let them take advantage of the surplus wind power, which would otherwise be discarded.

There are already examples of this in computationally intensive fields: A German car manufacturer, for instance, has run crash simulations at data centers in Ireland, where renewable energy is plentiful.7 The same is possible with generative AI too.

2. Run inferencing for local consumption

Employing a local-consumption approach for generative AI inferencing is an effective means for utilizing locally produced renewable energy. If the foundation model is in Tokyo, then generative AI users in other areas of the country would have to have their data transported to and from Tokyo. But if generative AIs were sited where their users are, they could enjoy the benefits of having them close by, like low latency, and—if the AIs are running on local renewable energy—help take pressure off Tokyo’s grid when supply is tight.

Making this kind of flexible utilization of local renewables happen calls for a comprehensive approach to designing and running ICT and power infrastructure, two types that are by nature disparate. Big tech abroad has already started moving toward finding and building solutions8. In Japan, too, ICT players and those in power generation and delivery will have to work together to make things happen in this area.

Power constraints can be a catalyst for boosting Japan’s competitiveness

Deep-pocketed big tech players are at the forefront of generative AI, but Japanese companies have plenty potential to transform power constraints into game-changing opportunities. And this in turn could very well become a catalyst for recovering the international competitiveness of Japan’s digital industries. Japan has two things to do here: develop software technologies that help generative AI perform with less electricity, and leverage its prowess in innovating semiconductor and peripheral technologies.

1. Develop more energy-efficient generative AIs

Growing concerns over power constraints is fueling interest in AIs that are more frugal.

For example, there is a technique called distillation for developing smaller AI models with performance similar to that of large ones by training them to emulate the large models’ behavior9. And there are numerous reports of successes in developing small yet well-performing generative AIs geared for use in specialized fields like medicine.

Rather than putting all their eggs in one giant model’s basket, some players are pushing ahead with research into computational methods10 that join together a number of smaller, field-specific or specialized models (“experts”) to get around the issue of power constraints. AI constellations like the ones NTT and Sakana AI collaborate on are one such example. If successful, they could have paradigm-shifting implications for the current trend toward the intensifying concentration of use of ever-larger models.
Figure 4: Diverse small AI models working together to get around constraints
Diverse small AI models working together to get around constraints
Source: Mitsubishi Research Institute, Inc.

2. Utilize specialized generative AIs

Another crucial approach is to develop AI services focused on specific fields and particular applications. These should be built from the perspective of the industries and members of society that will be utilizing them, which will ensure successful development and operation. For example, Preferred Networks, a Japanese AI venture, and petrochemical major ENEOS collaborated to develop a predictive AI capable of autonomously running a processing unit in a petrochemical plant, a world’s first. Theirs is a case study in collaboration between an AI developer and a major enterprise—with domestic production facilities, mountains of highly confidential data, and industry knowhow—to build a specialized AI service. Collaborations of this kind hold much promise in the generative AI field as well.

In the 1970s, two oil shocks threw the world’s auto industry into turmoil. But Japan’s car makers turned the adversity into a catalyst, bringing out highly fuel efficient cars that propelled them to the top of their industry worldwide when they eventually outproduced even US carmakers in the 1980s. Now it is time for Japan to once again turn a challenge—power constraints—into the impetus for boosting its competitiveness, this time in the generative AI field.

Works Cited

1According to SimilarWeb, ChatGPT (chat.openai.com) had 140 million unique visitors as of October 2023
https://www.similarweb.com/blog/insights/ai-news/chatgpt-birthday/ (Accessed: 12 July 2024)

2Mitsubishi Research Institute, Inc. “ICTインフラの三重苦を回避する” (Getting around the three shackles on ICT infrastructure) MRI Monthly Review, July 2023 issue

3Foundation models are AI models, like GPT-4o, that form the principal constituent element of generative AIs. The Large Language Model (LLM) is one such foundation model.

4Scaling laws are empirical laws postulated in 2020 by OpenAI researchers that show how foundation models’ performance improves proportionally as the number of parameters, training data sets, and number of learning calculations get larger. See Kalplan, J., McCandlish, S., et al., “Scaling Laws for Neural Language Models,” 2020, arXiv:2001.08361, available at https://arxiv.org/abs/2001.08361 (Accessed: 12 July 2024)

5Graphics Processing Unit: With their great power efficiency and processing speeds, GPUs excel at the kind of large-scale calculations that real-time graphics rendering, AI training, and subsequently inferencing entail

6We set our 2040 supply capacity scenario on a simple interpretation of Organization for Cross-regional Coordination of Transmission Operators, Japan (OCCTO) facility plans for the coming 10 years. The scenario maintains 2033 facility capacity per OCCTO’s supply forecasts, and it posits certain growth in renewables, assumes an increase in thermal utilization, and covers the number of nuclear restarts and plants for which building permits have been applied for as of 30 April 2024

7Germany’s BMW shifted a portion of its advanced, non-urgent (“highly latency tolerant”) compute tasks (like those for crash simulations and aerodynamics calculations) to Verne Global’s data centers in Ireland. Nearly all Ireland’s electricity is derived from renewable sources

8An example is Google’s piloting of a “carbon-intelligent computing platform.” It functions to optimize utilization of renewables like wind and solar, which are prone to dramatic availability swings, by dynamically shifting compute tasks across time and location to when and where energy supply is plentiful

9Google says that it used distillation to train its Gemini 1.5 Flash low-latency model:
https://blog.google/technology/ai/google-gemini-update-flash-ai-assistant-io-2024/ (Accessed: 12 July 2024)

10Technology known by names like Mixture of Experts (MoE)

Author profile

Author

Kentaro Abe

Center for Policy and the Economy

Kentaro Abe has been involved in advising the Japanese government on formulation of national policy on communications, broadcasting, and frequency allocation issues with a focus on 5G, Beyond 5G, and other next-generation communications infrastructure, along with helping with their societal implementation. He is also involved in distinctive research on societal constraints affecting uptake and utilization of generative AI, such as accuracy, public trust, and power consumption.

Author

Naoki Nishikado

Center for Policy and the Economy

Naoki Nishikado joined Mitsubishi Research Institute in 1997. He has been involved in advising governments on formulation of competition, rates, and other policies in the information and communications technology (ICT) sector and providing strategy consulting to broadband- and mobile-services–sector players. At present he is Head of Research, Information and Communications, conducting original research projects, engaging in joint research with academic institutions, and putting together policy recommendations.