The Green Energy Transition Is Moving Faster Than Most Forecasters Predicted — And Market Research Must Catch Up
A Market in Overdrive: The Scale of the Energy Transition
I have spent the better part of fifteen years conducting market research in the energy sector. I have watched solar power go from a subsidized curiosity to the cheapest source of electricity in human history. I have seen offshore wind transform from a niche Scandinavian experiment into a global industry. And I have watched the forecasting community — including some of the world's most sophisticated energy modelers — consistently underestimate the pace and scale of the clean energy transition, year after year after year.
The global energy and power market was valued at approximately $9.8 trillion in 2023, making it one of the largest industries on earth. Renewable energy's share of new power capacity additions has now exceeded fossil fuels in every major region. According to the International Energy Agency (IEA), renewable energy sources are expected to account for 90% of new electricity capacity additions globally between 2023 and 2030. The global clean energy investment figure hit a record $1.77 trillion in 2023, surpassing fossil fuel investment for the first time, per BloombergNEF data.
These are not incremental changes. They represent a structural transformation of the foundational infrastructure of the global economy. And yet, when I look at the market research frameworks most practitioners are still using to study the energy sector, I see methodologies built for a slower, more predictable world. That needs to change — urgently.
Opinion: The energy transition is not a future scenario to be planned for. It is a present reality being underestimated by forecasters and underserved by researchers. The gap between what is happening and what our research instruments can capture is growing — and that gap has real consequences for the clients and industries we serve.
Where the Forecasting Community Keeps Getting It Wrong
The most striking empirical pattern in energy market research over the past two decades is the systematic underestimation of renewable energy deployment. The IEA's own World Energy Outlook — the most cited energy forecast in the world — has consistently projected lower solar and wind growth than actually materialized. In 2010, the IEA projected that solar PV would generate roughly 210 GW of capacity globally by 2024. The actual figure exceeded 1,400 GW. That is not a rounding error. That is a categorical failure of the forecasting methodology.
Why does this happen? Several structural biases are at work:
- Linear extrapolation of exponential trends: Most energy models are built on linear growth assumptions. Solar and wind have followed learning curves — each doubling of cumulative capacity has driven consistent cost reductions of 20–30% — that are fundamentally non-linear and therefore systematically underestimated by conventional models.
- Anchoring to legacy energy economics: Models built around the economics of coal, gas, and nuclear plants embed assumptions about capital cost trajectories and financing structures that do not apply to modular, rapidly iterating technologies like solar panels and lithium-ion batteries.
- Regulatory pessimism: Forecasters have repeatedly underweighted the speed and ambition of policy intervention — from China's solar manufacturing subsidies to the U.S. Inflation Reduction Act to the EU's REPowerEU program. Policy acceleration has been the rule, not the exception, in clean energy markets.
- Demand-side conservatism: Corporate renewable energy procurement (power purchase agreements, or PPAs) has expanded far faster than most models projected, driven by corporate sustainability commitments from major buyers like Google, Microsoft, and Amazon — entities that did not feature prominently in traditional utility-centric energy demand models.
The Emerging Research Frontiers That Demand Attention
If existing frameworks are struggling to capture the pace of change in established renewable technologies like solar and wind, the challenge becomes even more acute in the emerging segments that will define the next phase of the energy transition. I want to highlight three areas where I believe market researchers must urgently develop new competencies:
Energy storage and grid flexibility: The global battery energy storage market is growing at a CAGR of over 30%, driven by the need to manage grid variability as renewable penetration increases. Research in this space must go beyond capacity projections to understand the complex interplay between utility procurement models, merchant market revenue stacking, and technology performance guarantees. Standard B2B survey methodologies are poorly suited to capture the sophistication of these decision-making processes.
Green hydrogen: The green hydrogen economy — producing hydrogen via electrolysis powered by renewable electricity — remains early-stage but is attracting extraordinary investment. Approximately $320 billion in green hydrogen projects have been announced globally as of 2023, according to the Hydrogen Council. Yet meaningful demand-side research is sparse. Who will actually buy green hydrogen, at what price points, and for which applications? These questions are inadequately answered by the current research base, and the gap between announced supply and credible demand is a major market risk that better research could illuminate.
Distributed energy resources (DERs): Rooftop solar, residential batteries, smart thermostats, EV chargers, and virtual power plants are transforming consumers into active market participants — "prosumers" who both consume and produce energy. Traditional utility market research, designed to study passive customers, is entirely inadequate for understanding prosumer behavior, motivations, and barriers. Consumer researchers from adjacent sectors — particularly those with expertise in smart home technology and behavioral economics — have much to contribute here.
What Research Practitioners Must Do Differently
I am not suggesting that all existing energy market research is without value. Secondary research drawing on sources like Wood Mackenzie, S&P Global Commodity Insights, BNEF, and the IEA itself provides essential quantitative context. Regulatory analysis, technology benchmarking, and competitive intelligence all remain critical functions. But I believe the following shifts are non-negotiable for researchers who want to remain relevant in this sector:
- Adopt scenario planning as a core deliverable, not an afterthought: Given the pace of policy change and technology cost reduction, point-estimate forecasts in energy markets are often misleading. Shell's scenario planning methodology, developed over decades, offers a model for how to communicate genuine uncertainty while still providing actionable strategic guidance. Every major energy research deliverable should include explicitly articulated scenario assumptions.
- Build supply chain research capabilities: The energy transition is fundamentally constrained by physical supply chains — for critical minerals like lithium, cobalt, nickel, and rare earth elements; for specialized manufacturing capacity; and for skilled labor. Researchers who can integrate supply chain analysis with demand forecasting will generate significantly more durable insights than those who model these dimensions separately.
- Invest in behavioral research for prosumer and community energy segments: The decarbonization of the demand side requires understanding why households and communities do or do not adopt clean energy technologies. This is fundamentally a behavioral research challenge, drawing on disciplines — behavioral economics, social psychology, ethnographic methods — that traditional energy researchers rarely deploy.
- Develop fluency in carbon markets: The voluntary and compliance carbon markets are increasingly intertwined with energy procurement decisions. Researchers who understand how corporate carbon commitments translate into energy purchasing behavior will be far better positioned to serve clients navigating net-zero strategies.
A Final Provocation for the Research Community
The energy transition is the defining economic and industrial story of our era. The researchers, analysts, and strategists who develop genuine expertise in this space — who invest in understanding the technology, the policy, the supply chains, and the behavioral dimensions of decarbonization — will find themselves at the center of some of the most consequential decisions being made anywhere in the global economy.
But that opportunity comes with a responsibility. When our research instruments systematically underestimate the pace of change, we give decision-makers false confidence in the stability of the status quo. When our methodologies are calibrated for a fossil-fuel-centric world, we risk reinforcing the inertia we should be helping our clients overcome. The energy transition is moving faster than most forecasters predicted. Our research must catch up — not just in its conclusions, but in its methods, its assumptions, and its ambition.