ORIGINAL COMPANY-CONTEXT PRACTICE
Backtest wind-generation forecasts across sites
Original practice scenario: A hypothetical wind operator wants a model using weather predictions and turbine history. Explain a fair evaluation for newly added sites.
Written for independent practice using company context; this is not a verified question from a company interview.
A REASONED APPROACH
Model answer.
Use the weather forecast version available at prediction time and evaluate both future periods and unseen sites. Actual future weather or revised forecasts would make an unrealistically easy test.
Work through it
- Build time-indexed features with issue times, forecast horizons, turbine availability and site metadata, explicitly separating curtailment from weather-driven variation.
- Use rolling temporal backtests and held-out-site evaluation, comparing to simple persistence and seasonal baselines at each horizon.
- Report error and interval coverage by site, weather regime and availability, including missing forecast inputs and limited-history sites.
Tradeoffs to discuss
- Shared models borrow strength across locations but can underfit local terrain effects; site-specific models need sufficient representative history.
Common pitfalls
- Do not score only on periods when turbines operated normally.
- An aggregate average can conceal poor performance during high-impact forecast errors.