1According to SimilarWeb, ChatGPT (chat.openai.com) had 140 million unique visitors as of October 2023
https://www.similarweb.com/blog/insights/ai-news/chatgpt-birthday/ (Accessed: 12 July 2024)
2Mitsubishi Research Institute, Inc. “ICTインフラの三重苦を回避する” (Getting around the three shackles on ICT infrastructure) MRI Monthly Review, July 2023 issue
3Foundation models are AI models, like GPT-4o, that form the principal constituent element of generative AIs. The Large Language Model (LLM) is one such foundation model.
4Scaling laws are empirical laws postulated in 2020 by OpenAI researchers that show how foundation models’ performance improves proportionally as the number of parameters, training data sets, and number of learning calculations get larger. See Kalplan, J., McCandlish, S., et al., “Scaling Laws for Neural Language Models,” 2020, arXiv:2001.08361, available at https://arxiv.org/abs/2001.08361 (Accessed: 12 July 2024)
5Graphics Processing Unit: With their great power efficiency and processing speeds, GPUs excel at the kind of large-scale calculations that real-time graphics rendering, AI training, and subsequently inferencing entail
6We set our 2040 supply capacity scenario on a simple interpretation of Organization for Cross-regional Coordination of Transmission Operators, Japan (OCCTO) facility plans for the coming 10 years. The scenario maintains 2033 facility capacity per OCCTO’s supply forecasts, and it posits certain growth in renewables, assumes an increase in thermal utilization, and covers the number of nuclear restarts and plants for which building permits have been applied for as of 30 April 2024
7Germany’s BMW shifted a portion of its advanced, non-urgent (“highly latency tolerant”) compute tasks (like those for crash simulations and aerodynamics calculations) to Verne Global’s data centers in Ireland. Nearly all Ireland’s electricity is derived from renewable sources
8An example is Google’s piloting of a “carbon-intelligent computing platform.” It functions to optimize utilization of renewables like wind and solar, which are prone to dramatic availability swings, by dynamically shifting compute tasks across time and location to when and where energy supply is plentiful
9Google says that it used distillation to train its Gemini 1.5 Flash low-latency model:
https://blog.google/technology/ai/google-gemini-update-flash-ai-assistant-io-2024/ (Accessed: 12 July 2024)
10Technology known by names like Mixture of Experts (MoE)