The Energy Transition Is Not Moving Fast Enough — And Market Researchers Are Part of the Problem
A Sector at a Crossroads
The global energy and power sector is undergoing its most profound transformation since the electrification of the late 19th century. The numbers are genuinely staggering: global investment in clean energy reached a record $1.77 trillion in 2023, according to BloombergNEF, for the first time surpassing investment in fossil fuels. Renewable energy capacity additions set a new record of 295 GW in 2022, led by solar PV and onshore wind. The International Energy Agency (IEA) projects that renewables will account for 35% of global electricity generation by 2025.
And yet, the world is not on track to meet the Paris Agreement's 1.5°C target. Global CO₂ emissions hit a record high of 36.8 billion tonnes in 2023. The energy transition, for all its momentum, is not moving at the speed that climate science demands.
As an opinion, I want to make a case that is rarely articulated in our industry: the market research community — those of us who shape the intelligence that informs investment decisions, policy formulation, and corporate strategy — bear a meaningful share of responsibility for this inadequacy. We are measuring the wrong things, asking the wrong questions, and in doing so, providing a distorted picture of where the energy market actually is and where it needs to go.
The Measurement Bias Toward Incumbency
The dominant methodologies in energy market research were developed during an era of stable, centralised energy systems dominated by large utilities, fossil fuel majors, and regulated monopolies. Demand forecasting models built on historical consumption data, competitive analysis frameworks designed for oligopolistic markets, and customer satisfaction metrics calibrated for captive consumers are poorly suited to a sector being disrupted by distributed generation, peer-to-peer energy trading, virtual power plants, and prosumer behaviour.
Consider how we measure consumer attitudes toward energy. Most large-scale tracking studies in the sector still ask questions like: "How satisfied are you with your energy provider?" and "How likely are you to switch supplier in the next 12 months?" These are the right questions for a 2003 deregulated energy market. They are almost entirely irrelevant to understanding whether a household in California or Germany will invest in rooftop solar, participate in a demand response programme, purchase a home battery, or shift consumption to take advantage of dynamic pricing.
The Rocky Mountain Institute, IRENA, and academic researchers at institutions like Imperial College London and MIT Energy Initiative have been pushing for more sophisticated consumer research frameworks for years. But the bulk of the commercial market research commissioned by energy companies still leans heavily on legacy methodologies. This is not because researchers are incompetent — it is because clients are requesting what they know, and vendors are supplying what they are comfortable delivering.
The Prosumer Revolution Is Being Under-Researched
One of the most significant structural shifts in energy markets is the emergence of the prosumer — individuals and organisations who both consume and produce energy. In Australia, more than 3.5 million rooftop solar installations now cover over 30% of detached dwellings, fundamentally altering grid dynamics and creating a distribution network that operates in ways no central planner designed. In Germany, the Energiewende has produced over 1.5 million energy cooperatives and prosumer entities that are reshaping local grid economics.
Yet the market research literature on prosumer decision-making remains thin. We have reasonable data on adoption rates and system sizes, but we lack deep understanding of the psychological, social, and financial drivers that lead a specific household to become an active energy market participant rather than a passive consumer. Conjoint analysis studies examining the relative importance of feed-in tariff rates, battery economics, environmental motivation, and peer influence are urgently needed — and largely absent from the research canon.
"The energy industry is awash in technology data and desperately short of human data. We know everything about what the panels and turbines are doing and almost nothing about what the people are thinking." — A sentiment I have heard repeatedly from senior strategy executives at major utilities across Europe and North America.
Hydrogen: A Case Study in Research Hype Exceeding Evidence
The green hydrogen market represents perhaps the starkest example of research community failure in recent memory. Between 2020 and 2022, a wave of market reports — many from reputable consultancies — projected green hydrogen becoming cost-competitive with grey hydrogen by the mid-2020s and capturing substantial shares of industrial heat and heavy transport demand by 2030. These projections were based on optimistic electrolyser cost curves, assumed renewable electricity prices, and extrapolated from a very thin base of demonstrated projects.
By 2023, the reality was becoming apparent: electrolyser manufacturing costs were not declining at projected rates, green hydrogen project cancellations were mounting globally, and the IEA revised its near-term green hydrogen demand estimates downward significantly. Companies that had made strategic investments based on the optimistic research consensus were facing painful recalibrations. Air Products, which had committed to multi-billion dollar green hydrogen projects, faced significant investor scrutiny. Projects in Australia, Europe, and the Middle East stalled or were abandoned.
The research community — including major consultancies and investment banks — had allowed narrative momentum to substitute for rigorous primary research. Better-designed primary studies with industrial energy consumers, examining genuine willingness-to-pay for green hydrogen at realistic cost levels and current infrastructure limitations, would have provided a more accurate picture.
What Good Energy Market Research Looks Like
I am not arguing for pessimism about the energy transition — the structural forces driving decarbonisation are real and powerful. I am arguing for research that accurately characterises both the pace and the complexity of that transition. Specifically, researchers in the energy sector should:
- Invest in longitudinal panel research tracking actual household energy behaviour over time, not just stated intentions. The gap between intention and behaviour is enormous in energy decisions, driven by capital constraints, inertia, and information asymmetries.
- Apply behavioural economics frameworks systematically. The work of researchers like Hunt Allcott (NYU Stern) on energy efficiency gaps demonstrates that standard economic models dramatically overpredict uptake of cost-saving energy technologies. Understanding the psychological barriers is as important as understanding the economic ones.
- Commission independent demand-side validation for technology markets before projecting commercialisation timelines. Talking to ten informed industry insiders is not sufficient — structured research with the actual potential buyers (industrial firms, utilities, municipalities) is necessary.
- Acknowledge and disclose uncertainty more rigorously. Energy markets are subject to policy discontinuities, technological surprises, and macroeconomic shocks that make point-estimate forecasting intellectually dishonest. Scenario-based approaches with explicit probability distributions are more honest and ultimately more useful.
The Stakes Are Too High for Comfortable Research
The energy transition involves the reallocation of trillions of dollars of capital, the restructuring of entire industries, and decisions with multi-decade consequences for both climate outcomes and energy security. In this context, market research that confirms existing biases, extrapolates aggressively from thin evidence, or fails to capture the genuine complexity of consumer and industrial decision-making is not just methodologically deficient — it is actively harmful.
The energy research community has an opportunity to be genuinely valuable at one of the most consequential moments in industrial history. Seizing that opportunity requires intellectual honesty, methodological evolution, and the courage to deliver findings that challenge rather than validate the assumptions of our clients. That, ultimately, is what professional market research is for.