AI infrastructure
AI infrastructure comprises the physical and software systems used to develop, train, deploy and operate artificial intelligence (AI) models and applications. In a narrow sense, the term refers to the physical computing resources that underpin AI, including semiconductors, processors, memory, servers, storage, networking equipment and data centres. Broader descriptions also include the software that makes this hardware usable, such as drivers, compilers, machine-learning frameworks, resource-management systems and cloud platforms.[1][2]
AI infrastructure is one part of a wider AI supply chain that also includes training data, models and applications. It may be owned and operated by an organisation, obtained through cloud computing, or provided through shared public or national computing programmes.[3][4] The expansion of AI has made access to advanced chips, cloud capacity, electricity and suitable sites a subject of industrial policy, competition policy and environmental assessment.[1][5]
Scope and terminology
[edit]There is no single universally used boundary for AI infrastructure. Policy reports sometimes use the term chiefly for physical resources, particularly advanced chips and the data-centre systems in which they operate. In computing practice, the related concept of the compute stack is commonly divided into hardware, software and infrastructure, with the layers affecting one another's performance and cost.[1][2] Other analyses place hardware and cloud infrastructure as the first two layers of a supply chain that continues through data, foundation models and applications.[3]
Under these definitions, data, algorithms, trained models and applications depend on infrastructure but are not necessarily treated as infrastructure themselves. An AI data center is a facility-level component of AI infrastructure, rather than a synonym for the entire stack. Infrastructure may support both AI and non-AI workloads: general-purpose computing and accelerated computing share facilities, storage and some network equipment, while differing in processors, interconnect requirements, power density and cost.[6]
Components
[edit]Computing hardware
[edit]AI systems run on central processing units (CPUs) and on specialised accelerators that perform large numbers of operations in parallel. Common accelerators include graphics processing units (GPUs), application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs). Different workloads use different combinations of processors: large-scale model training commonly relies on accelerators, while some inference and supporting tasks can run on general-purpose CPUs.[2][6]
Accelerators are assembled into servers and then into clusters. Their usefulness depends on associated memory, storage, networking and systems software, so processor counts alone do not describe usable AI capacity.[2] The semiconductor supply chain behind these systems includes chip design, manufacturing equipment, fabrication, packaging and materials. It is globally distributed but contains geographically concentrated inputs and specialised production stages, creating dependencies and potential bottlenecks.[7]
Memory, storage and networking
[edit]Training and operating AI models requires moving data between processors, memory and storage. Servers use memory close to the processors for active calculations, while storage systems retain training data, intermediate results and deployed model files. High-bandwidth connections link processors within a server, servers within a cluster and clusters to storage and external networks.[2]
Large training workloads are often distributed across many machines. Distribution can reduce the time needed for a computation, but it introduces coordination and communication problems, including how to parallelise training efficiently and maintain a coherent model across workers.[8] Consequently, cluster performance depends not only on arithmetic capacity but also on memory bandwidth, network latency, network bandwidth and the way work is divided.
Data centres and cloud services
[edit]Data centres provide the buildings, racks, servers, storage, network equipment, power distribution, backup systems and cooling needed to operate computing hardware. Some organisations build private clusters, while others rent virtual machines, accelerator instances or managed AI services from public cloud providers. Cloud regions are physical hubs that place computing capacity in particular geographic and legal jurisdictions.[2][9]
Cloud platforms combine infrastructure with model-development tools, databases, security services and application marketplaces. This integration can make large-scale resources available without building a private data centre, but it can also create technical and commercial dependence on a provider's interfaces, services and software ecosystem.[6][10]
Software
[edit]The software layer translates model operations into work that processors can execute. It includes device drivers, compilers, programming libraries and machine-learning frameworks, as well as software for assigning accelerators, scheduling jobs and monitoring clusters. Deployed systems also need data pipelines, model-serving systems and access controls.[2][11]
Production machine-learning systems must handle requirements such as large and heterogeneous data flows, low-latency processing, resource management, security and privacy. Software architecture therefore affects how efficiently physical capacity can be used and how reliably a model can be integrated into a larger application.[11]
Electricity and cooling
[edit]AI computing equipment requires electrical supply, power conversion and distribution, uninterruptible power supplies or other backup systems, and cooling. These supporting systems take up space and consume energy in addition to the processors themselves. Accelerated clusters can require more power and cooling than general-purpose servers, and rapid changes in workload can create large variations in electrical load.[12][5]
The siting of infrastructure is therefore affected by access to grid connections, reliable electricity, cooling technology, water, telecommunications links and suitable land. These constraints can determine whether announced projects are built and how quickly capacity becomes available.[2][5]
Training and inference
[edit]Machine learning training uses data to adjust a model's parameters. Advanced training runs may occupy large accelerator clusters for extended periods and use distributed software to divide calculations and exchange intermediate results. A study of historical machine-learning systems estimated that training compute doubled about every six months after the emergence of deep learning in the early 2010s, although trends vary by model class and time period.[13]
Inference applies a trained model to new inputs. Its infrastructure requirements range from small, latency-sensitive deployments to high-volume services running large models in data centres. Some inference workloads use accelerators, while smaller or more efficient models may run on general-purpose processors; cost, response time, throughput and availability influence the choice.[6] Software and hardware efficiency can reduce the resources needed for a given level of performance. Between November 2022 and October 2024, the inference cost of a system performing at the level of GPT-3.5 fell more than 280-fold.[14] Lower unit costs do not necessarily reduce total infrastructure demand if the number, scale or complexity of uses grows.[5]
Economics and access
[edit]Scale and market structure
[edit]AI infrastructure has high fixed costs at several layers. Advanced semiconductor production requires specialised facilities and equipment, while cloud and data-centre operators invest in land, buildings, power systems, networking, storage and large numbers of processors. Scale can lower unit costs and support a wider range of services, but it can also make entry difficult for smaller firms.[7][3][6]
Large technology companies operate across several layers of the AI supply chain, including cloud infrastructure, model development and applications. Vertical integration can simplify deployment and fund large investments, while also raising competition concerns about access, switching, preferential treatment and dependence on a small number of providers.[1][10] The UK Competition and Markets Authority found that accelerated cloud computing required significant investment in accelerator chips and specialised infrastructure, and that large sunk costs and economies of scale were barriers in cloud services.[6]
Geographic distribution and public capacity
[edit]AI computing capacity is distributed unevenly. Public cloud capacity is organised into regions, so users' practical access can depend on whether specialised hardware is available domestically or in a nearby jurisdiction. Location also affects latency, data-governance obligations, resilience and exposure to trade or supply restrictions.[9]
Governments have consequently considered public, shared or nationally supported computing capacity. The OECD has described national compute planning in terms of capacity and utilisation, effective access and skills, and resilience issues including security, sovereignty and sustainability.[4] Such programmes may provide access to researchers, public institutions and smaller firms that cannot procure large clusters directly, but they still depend on global semiconductor, networking and energy supply chains.[7]
Environmental impacts
[edit]The environmental effects of AI infrastructure occur across the equipment life cycle. They include energy and water used in semiconductor production and data-centre operation, greenhouse-gas emissions associated with electricity generation, materials used in servers and supporting equipment, and electronic waste at end of life. Environmental assessment can distinguish these direct effects from the indirect environmental costs or benefits of applications that use AI.[15]
AI-specific electricity use is difficult to isolate because data centres host mixed workloads and operators disclose limited comparable data. The International Energy Agency estimated that global data-centre electricity consumption was about 485 terawatt-hours in 2025 and projected roughly 950 terawatt-hours in 2030; in its central projection, electricity use by AI-focused data centres grows faster and triples over that period.[5] These figures cover scenarios rather than a fixed outcome and depend on efficiency, demand, grid connections and the rate at which proposed facilities are completed.
Water and emissions also vary by location, electricity mix and cooling system. A 2025 scenario study of AI servers in the United States estimated an annual water footprint of 731–1,125 million cubic metres and additional annual emissions of 24–44 million tonnes of carbon-dioxide equivalent between 2024 and 2030, depending on expansion; the authors emphasised deep uncertainty from server locations, efficiency measures and grid decarbonisation.[16] Proposed responses include more efficient hardware and software, higher equipment utilisation, lower-carbon electricity, water-aware siting and cooling, greater disclosure, and life-cycle measurement standards.[15][12]
See also
[edit]References
[edit]- 1 2 3 4 OECD (14 November 2025). Competition in artificial intelligence infrastructure (Report). OECD Roundtables on Competition Policy Papers. Paris: OECD Publishing. doi:10.1787/623d1874-en. Retrieved 11 July 2026.
- 1 2 3 4 5 6 7 8 Gupta, Aadya; Ranjan, Adarsh (30 April 2024). "A Primer on Compute". Carnegie Endowment for International Peace. Retrieved 11 July 2026.
- 1 2 3 Gambacorta, Leonardo; Shreeti, Vatsala (March 2025). The AI supply chain (PDF) (Report). BIS Papers. Bank for International Settlements. ISBN 978-92-9259-843-3. Retrieved 11 July 2026.
- 1 2 OECD (28 February 2023). A blueprint for building national compute capacity for artificial intelligence (Report). OECD Digital Economy Papers. Paris: OECD Publishing. doi:10.1787/876367e3-en. Retrieved 11 July 2026.
- 1 2 3 4 5 International Energy Agency (16 April 2026). Key Questions on Energy and AI (Report). Paris: International Energy Agency. Retrieved 11 July 2026.
- 1 2 3 4 5 6 Competition and Markets Authority (31 July 2025). Cloud infrastructure services: Final decision report (PDF) (Report). Competition and Markets Authority. Retrieved 11 July 2026.
- 1 2 3 OECD (24 June 2025). Mapping the semiconductor value chain: Working towards identifying dependencies and vulnerabilities (Report). OECD Science, Technology and Industry Policy Papers. Paris: OECD Publishing. doi:10.1787/4154cdbf-en. Retrieved 11 July 2026.
- ↑ Verbraeken, Joost; Wolting, Matthijs; Katzy, Jonathan; Kloppenburg, Jeroen; Verbelen, Tim; Rellermeyer, Jan S. (2020). "A Survey on Distributed Machine Learning". ACM Computing Surveys. 53 (2). Article 3377454. doi:10.1145/3377454. Retrieved 11 July 2026.
- 1 2 Lehdonvirta, Vili; Wu, Boxi; Hawkins, Zoe Jay; Caira, Celine; Russo, Lucia (29 October 2025). Measuring domestic public cloud compute availability for artificial intelligence (Report). OECD Artificial Intelligence Papers. Paris: OECD Publishing. doi:10.1787/8602a322-en. Retrieved 11 July 2026.
- 1 2 van der Vlist, Fernando; Helmond, Anne; Ferrari, Fabian (12 March 2024). "Big AI: Cloud infrastructure dependence and the industrialisation of artificial intelligence". Big Data & Society. 11 (1). doi:10.1177/20539517241232630. Retrieved 11 July 2026.
- 1 2 Cabrera, Christian; Paleyes, Andrei; Thodoroff, Pierre; Lawrence, Neil D. (20 November 2025). "Machine Learning Systems: A Survey from a Data-Oriented Perspective". ACM Computing Surveys. 58 (5). doi:10.1145/3769292. Retrieved 11 July 2026.
- 1 2 International Energy Agency (10 April 2025). Energy and AI (Report). Paris: International Energy Agency. Retrieved 11 July 2026.
- ↑ Sevilla, Jaime; Heim, Lennart; Ho, Anson; Besiroglu, Tamay; Hobbhahn, Marius; Villalobos, Pablo (2022). Compute Trends Across Three Eras of Machine Learning. 2022 International Joint Conference on Neural Networks. pp. 1–8. doi:10.1109/IJCNN55064.2022.9891914. Retrieved 11 July 2026.
- ↑ Stanford Institute for Human-Centered Artificial Intelligence (2025). The 2025 AI Index Report (Report). Stanford University. Retrieved 11 July 2026.
- 1 2 OECD (15 November 2022). Measuring the environmental impacts of artificial intelligence compute and applications: The AI footprint (Report). OECD Digital Economy Papers. Paris: OECD Publishing. doi:10.1787/7babf571-en. Retrieved 11 July 2026.
- ↑ Xiao, Tianqi; Nerini, Francesco Fuso; Matthews, H. Damon; Tavoni, Massimo; You, Fengqi (10 November 2025). "Environmental impact and net-zero pathways for sustainable artificial intelligence servers in the USA". Nature Sustainability. 8: 1541–1553. doi:10.1038/s41893-025-01681-y. Retrieved 11 July 2026.
