Comments on working group meeting

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

Summary:

The California Community Choice Association (CalCCA) appreciates the opportunity to comment on the California Independent System Operator’s (CAISO) December 6, 2023, RA Modeling and Program Design Working Group (WG).

The review and revision of the resource adequacy (RA) structure has become increasingly important.  As the CAISO notes in its discussion paper, the types of resources, expectations of load, and programmatic changes of RA all have varying impacts on the ability to reliably serve customer needs.  As the CAISO and stakeholders move forward in this WG and ultimately, a stakeholder process, the objectives of this effort and existing conditions on the grid should be made clear.  This clarity should be provided by:

• Evaluating the efficacy of the available fleet through a probabilistic assessment and whether the stack-type analysis of the RA program will find the same outcome;
• Using data available to the CAISO such as RA showings, net qualifying capacity (NQC) listings, and historical import levels and trends to estimate future RA showings;
• Evaluating the ability of mechanisms to better incent resource availability in RA, such as unforced capacity (UCAP), consistently with Local Regulatory Authority (LRA) programs or proposals;
• Ensuring that LRA and CAISO rules are consistent and do not result in outcomes that are counter to LSE incentives;
• Providing the market with more transparency into the use and availability of RA resources for compliance purposes; and
• Encouraging the department of market monitoring (DMM) and other balancing authority areas (BAA) in the Western Electricity Coordinating Council (WECC) to coordinate to provide additional transparency of capacity conditions and expectations in the west.

                                                                                              *** END OF SUMMARY ***

1. CalCCA Response:

In response to a question from Southern California Edison Company regarding why the CAISO did not backstop for central procurement entity deficiencies, the CAISO responded, in part, that resources may not be available because load serving entities (LSE) may be holding resources to avoid Resource Adequacy Availability Incentive Mechanism (RAAIM) charges.  However, RAAIM is a charge applicable to a Scheduling Coordinator for the generator, not an LSE.  Only in the case that an LSE is a Scheduling Coordinator for generation would the potential holding of excess resources to avoid RAAIM be a potential issue.  This is a narrow set of resources for which a local RA-eligible facility would be in the hands of a Scheduling Coordinator that is also an LSE, and the LSE has decided to hold the resource for potential RAAIM charges. Therefore, other causes (lack of incentive, locking in of a resource year-ahead (YA), showing a local resource for a system need, etc.) are more likely than RAAIM.  This distinction is important because as market participants examine issues like this, the causes of the issue must be identified precisely so that changes can be made to make the RA process more effective and efficient.  In this case, belief that the holding of RA capacity to serve RAAIM needs would suggest potential changes to mechanisms under the control of the CAISO.  However, if the causes were elements like the lack of incentive, locking in of a resource YA, or the showing of a local resource to meet system need by an LSE, the changes to those mechanisms must occur at the LRA instead.

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

The CAISO Should Ensure Assumptions are Consistent with Local Regulatory Requirement Setting and Planning

The CAISO presented a modeling approach that looks at three time frames: year-ahead, 2-4 years forward, and 5-10 years forward. Each would use probabilistic modeling to ensure the assumed fleet meets a 1-in-10 loss of load expectation (LOLE) planning target. The CAISO’s general approach and set of studies appear that they will offer valuable information about the sufficiency of the RA fleet to meet demand. It will be critical for the CAISO to use the same assumptions as the California Public Utilities Commission (CPUC) for use in their planning reserve margin (PRM) setting and integrated resource planning (IRP) to maximize the probability that RA showings will meet a 1-in-10 LOLE. To the extent the CAISO makes different assumptions than the CPUC, then the CAISO should make its assumptions public so LSEs know what they need to do to meet CPUC requirements and CAISO assessments.

The CAISO Should Issue Stack Analyses in addition to Probabilistic Modeling for Each Study Time frame

For transparency and ease of showing key drivers of the outcomes, the CAISO should use the same assumptions and data sources to conduct an RA Stack Analysis in each of the three time frames (YA, 2-4 years, 5-10 years) that parallels the probabilistic LOLE modeling. The stack should yield similar results, at least directionally, and any big differences will suggest the need for updating/refining the effective load carrying capability approach to ensure peak and net peak needs are satisfied. A stack is also more similar to how compliance is assessed, and it would be helpful to know if the supply stack expected to be available in those time horizons will satisfy LSEs compliance obligations.

Using only a probabilistic assessment for future RA periods (YA, 2–4 years, and 5–10 years in advance) will tell the CAISO whether the resource mix studied meets a probabilistic simulation of potential combinations of conditions over those future years.  What it will not tell the CAISO, the LRAs, and stakeholders is whether the same fleet of resources that either passes or fails the probabilistic simulation would pass or fail an RA showing which is based on a stack type of process.  If a set of resources fails the probabilistic simulation but passes a stack analysis, the conclusion is logically one of the following:

• An unlikely combination of inputs in a significant number of draws in the probabilistic simulation (a mathematically unlikely outcome);
• Resource counting that inaccurately predicts resource capacity for RA; or
• Inaccuracy of the PRM to account for ancillary services, forced outages, and load forecast error.

Unless the CAISO has identified a different manner to compare the results of its probabilistic modeling to the results of an RA showing, then evaluating these time frames on a stack basis (rooted in the showing requirements of the LRA) will be necessary to determine if the RA program will routinely produce a portfolio capable of meeting grid needs and capable of meeting RA showings compliance.

In the 2-4 and 5-10 year time frames, the comparison of probabilistic assessment and stack analysis will show whether there is a mismatch between methods.  In the event that the probabilistic assessment predicts a reliable fleet but the stack analysis does not, then the IRP should evaluate the cause of the misalignment.  Doing so will allow either; the IRP to account for these differences to produce a reliable fleet, or the RA program to consider changes to counting rules and PRMs to enable compliance under 1-in-10 conditions.

The CAISO Should Estimate a 100 Percent Annual Showing by using data and history already available to the CAISO

The CAISO explains that in its year-ahead RA sufficiency analysis, the CAISO will need to determine what resources to include in the analysis to get to a 100 percent shown capacity equivalent (because the year-ahead RA requirement is only 90 percent). 

The CAISO asked if LSEs could provide additional information year-ahead about their expectations for their month-ahead (MA) showings to complete the 10 percent need. Doing so is both impractical and burdensome. In the current market conditions of scarce capacity, it is unlikely that LSEs know the availability of internal and import resources that could fill the remaining 10 percent.  Without such knowledge, LSEs will be left to speculate without the benefit of knowing how much of the CAISO interconnected NQC is still available for procurement after the YA showing. 

Instead, the CAISO has access to all LSE YA showings and supply plans.  The CAISO also has the current NQC listing as well as knowledge of historical RA imports in the YA and MA time frame and the trend of those imports over time.  Equipped with that knowledge, the CAISO is in the best position to use its available data to understand what is likely to be available to CA LSEs to meet the remaining MA RA requirement.

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

Problem Statement 2 should be updated to include UCAP as a sub-issue. UCAP is interrelated with RAAIM, the lack of a tool to incentivize performance, and rules for substitution and planned outages sub-issues. Including UCAP is likely to address many of the problems associated with these sub-issues. UCAP is also listed as a topic that will be explored in the upcoming RA proceeding at the CPUC (Rulemaking 23-10-011), and it will be important for the CAISO and CPUC to align on UCAP proposal development and implementation.

Problem Statement 3 should be updated to include proposed revision request 1280/credited resources as a sub-issue. The CAISO’s tariff does not allow the CAISO to consider “credited” resources (resources count for RA under CPUC rules but are not shown on CAISO RA plans and supply plans) when allocating Capacity Procurement Mechanism (CPM) costs, a problem that while known of prior, became a reality after the CAISO’s August 2023 CPM.  In this circumstance, there were LSEs that had met their RA target when including the “credited resources” and LSEs that had not met their RA target even with their share of the “credited resources”. However, when the CAISO performed its backstop and cost allocation, some of the previously compliant LSEs became short of RA once the CAISO had removed the “credited resources” from their showing. Thus, LSEs that had otherwise met their RA requirement were allocated a portion of the CAISO backstop cost that would not have been allocated to them if other LSEs would have met their compliance requirements. The CAISO should work with the CPUC to ensure credited resources’ RA capacity is recognized, either through changes to the CPUC rules to have credit resources shown on supply plans or through a CAISO tariff change to support credited resources when allocating CPM costs.

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

The Monthly Summer Performance Report’s scope extends far beyond RA and includes detailed analysis that takes time to complete. The result is that the monthly RA showings’ reports in the Monthly Summer Performance Reports are not always timely relative to the showing (e.g., it is December and the September 2023 monthly report is still not posted). The CAISO should consider separating out the RA portion of this report and posting it separately (e.g., on the Open Access Same-time Information System (OASIS)) so that it can be posted in a timelier manner.  Timeliness of data should ensure that it is not posted too early so that it can be used to the detriment of the market (e.g., before the RA cure period concludes) but should also be posted soon thereafter so that the LRA can better assess market conditions that may have impacted LSEs ability to comply.

As discussed in question 2, the CAISO has access to information that would help the market determine the availability of capacity resources in California. The CAISO knows from supply plans the resources and quantities of RA committed to the CAISO. From the NQC list, the CAISO also knows all of the resources and quantities that are qualified to provide RA. The CAISO should publish aggregated information about the amount of resources available and the actual showings from those resources.  The aggregation could be as simple as listing the amounts by technology (e.g., gas, wind, solar, hydro, etc.) 

It has become increasingly difficult for LSEs to find available capacity resources. While this information will not provide a complete map to where capacity can be found, it can at least begin describing how much of the existing NQC list is being made available as RA.

In addition, the RA imports included in the reports do not include the pseudo-tie and dynamic transfers. Since the dynamic transfers are not in the NQC list posted to the CAISO site, it is difficult for third parties to use data in the reports to discern key information like what fraction of the eligible RA resources on the NQC list were shown. Earlier datasets posted by the CAISO include all RA imports (e.g., https://www.caiso.com/Documents/HistoricalResourceAdequacyImportAggregateData.xlsx).  It would be helpful if CAISO reported aggregate RA imports from both specified and unspecified resources.  

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

The DMM should track actual capacity transaction prices rather than assume how hypothetical Combustion Turbine (CT) or combined cycle gas turbine (CCGT) earns the CAISO soft-offer cap, as the CAISO shows on slides 61 and 62. CalCCA’s analysis of FERC Electric Quarterly Report (EQR) data shows that the weighted average price of capacity in California for the 12-months ending in September 2023 would add $101/ kilowatt (kW)-year to the revenue of a CT or CCGT, much higher than the $76-88/kW-year capacity revenue based on the CAISOs soft-offer cap. In fact, the weighted average price of capacity in the month of September 2023 was $13.2/kW-month which is 80-110 percent higher than the CAISO soft-offer cap.  Individual transactions for capacity in September 2023 were above $60/kW-mo. Such high prices should be monitored and analyzed by the DMM.  

Given the current market conditions for RA, it is imperative that the need for and availability of resources is examined more closely than it is today. In questions 2 and 4 of these comments, CalCCA has identified additional public information that should be made available. In addition, the DMM would be an ideal organization, due to its expertise in markets and independence, to examine and evaluate the cases in which CAISO interconnected NQC is not shown in an RA showing and the trend of import RA available to meet California LSE needs. Better still would be coordination between the DMM and other BAAs in the WECC to evaluate the complete set of resources, rules, and availability to better inform current RA needs as well as new resource build needs. A more complete analysis of the availability of RA resources is necessary to understand how best to ensure that resources are made available to meet grid needs.

6. Please provide any feedback not already captured.

CalCCA has no additional feedback at this time.

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

CAISO’s RA101 information is sufficient enough to introduce the existing RA processes.

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

On the slide #25, CAISO indicates that for future years (2-4 years and 5-10 years) modeling, Loss of Load Expectation (LOLE) and Expected Load carrying Capability (ELCC) approach for resource counting will be considered. However, in the slide #22, it is confirmed that for solar and wind resources ELCC based resource counting will be replaced by an exceedance based counting. Therefore, for solar and wind resources, preferred option on counting method should be clarified (in the slide #22, and #25) for consistency.

With regard to the shift in QC calculation methodology for solar and wind resources (slide #22), an assessment of reduction in wind and solar NQC in the past and future years in aggregate (systemwide) due to exceedance based approach may provide an indication on gap in the RA capacity for both past and future years as significant capacity from solar and wind may be lost. This assessment may provide information on what resources may be needed to meet the gross peak load due to reduction in the solar and wind QC values.

An illustrative numerical example (using a particular month in a year, e.g., August 2024) on CAISO methodology on derivation of 0.1 LOLE requirements (slide #27) systemwide and how the resources would stack to meet that requirement would be helpful to understand the 0.1 LOLE requirements.

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

No comment at this time.

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

Definition and illustrative examples of each metric in the RA trend curve may provide further clarity on how the numbers are derived. Additional RA metrics such as RA capacity requirements by hour based on the tariff default PRM (PRM RA requirements), and RA requirements for 0.1 LOLE (0.1 LOLE RA requirement) by hour (if feasible) may provide status on meeting systemwide criteria. These two metrics side by side may provide comparison between PRM and the 0.1LOLE requirements.

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

No comments at this time. 

6. Please provide any feedback not already captured.

No comments at this time. 

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

The Public Advocates Office at the California Public Utilities Commission (Cal Advocates) is the state-appointed independent ratepayer advocate at the California Public Utilities Commission (CPUC).  Our goal is to ensure that California ratepayers have affordable, safe, and reliable utility services while advancing the state’s environmental goals.  Our efforts to protect ratepayers include energy, water, and communications regulation advocacy. 

Cal Advocates makes the following recommendations:

• The CAISO should confirm whether the Resource Adequacy Availability Incentive Mechanism (RAAIM) will be reviewed in the January or February workshops;
• The CAISO should publish the Planning Reserve Margins (PRMs) of all Local Regulatory Authorities (LRAs) for whom the CAISO is the Balancing Area Authority (BAA);
• The CAISO should consider developing and requesting a non-binding showing from CPUC load-serving entities (LSEs) to report resource adequacy (RA) capacity that is owned or contracted by the LSEs but not used to meet the 90% year-ahead requirement of the CPUC’s RA program;
• The CAISO should provide additional public information on monthly import showing volumes;
• Energy sufficiency tests should be developed in this initiative and incorporated into the CAISO’s RA evaluation framework;
• The Today’s Outlook dashboard would be enhanced by enabling bulk data downloads across multiple months for data currently provided in the dashboard;
• The Today’s Outlook would also be enhanced by adding the ability to access and download price information for all three markets across any time interval;
• The Department of Market Monitoring (DMM) annual report would benefit by adding a CAISO-determined loss of load expectation supply metric in reported RA information; and
• The CAISO should not adopt problem statements that presuppose policy responses.

Cal Advocates appreciates the CAISO’s RA 101 overview and working group content in the Resource Adequacy Modeling and Program Design process.  This content has provided useful and accessible information that helps stakeholders understand the complex dynamics of resource adequacy compliance in the CAISO.  Looking ahead in the published working group plan, Cal Advocates notes that “resource counting and incenting availability” is scheduled for the January and February workshops.  Cal Advocates requests that the CAISO confirm that the January and February workshops include review of the RAAIM.[1]

Next, Cal Advocates requests that the CAISO publish the PRMs of all LRAs that participate in the CAISO RA program.   This information is needed to interpret grid-wide reliability assessments, conduct effective ratepayer advocacy, and will help hold LRAs accountable to the ratepayers they serve.  Likewise, this information is useful for verifying the cost allocation of grid-wide costs, such as capacity procurement mechanism designations.  The PRMs set by all LRAs are subject to public oversight and are, in theory, obtainable via a public records request from each individual agency.  As grid operator, the CAISO should publicize PRM information centrally and immediately. This will spare stakeholders the time and burden of submitting public records requests to the dozens of LRAs that are subject to the CAISO.   

It is imperative to prioritize transparency where possible and ample precedent exists for publishing this kind of information in one place.  For example, the California Energy Commission’s (CEC) 2023 Draft Integrated Energy Policy Report (IEPR) includes an Appendix summarizing the procurement position of 37 Publicly-owned Utilities (POUs).[2]  This data “compar[es] the total existing and planned supply resources for each year (2023–2031), and the projected peak demand plus a planning reserve margin (PRM) of at least 15 percent above the peak demand.”[3]  The table is reproduced below.

Table 1: Reproduction of CEC Table Summarizing POU Resource Adequacy

Cal Advocates acknowledges that the peak load of each of these LRAs should be considered confidential and market-sensitive information.  However, the actual PRM of each LRA should be public information.  At a minimum, the CAISO should publish:

1. summary statistics of the LRA PRMs that represent CAISO load beyond CPUC-jurisdictional LSEs.  These statistics should include the unweighted average, peak load share-weighted average, median, mode, minimum and maximum; and,
2. the CAISO BAA’s PRM built from the load-weighted average of all LRA PRMs, including the CPUC.  Cal Advocates estimates that this PRM is around 15.72%,[4] but providing the CAISO’s calculations would improve transparency and the planning process.

This information is valuable for promoting transparency at the CAISO and to support reliability modeling and planning conducted by LSEs and other stakeholders in the future.

[1] CAISO, Resource Adequacy Modeling and Program Design Working Group slides, December 6, 2023 at 79.  Available at: https://www.caiso.com/InitiativeDocuments/Presentation-Resource-Adequacy-Working-Group-Dec62023.pdf.

[2] Some of the POUs listed in Table 1 are not part of the CAISO BAA, such as Los Angeles Department of Water and Power and Imperial Irrigation District.  Cal Advocates’ request to publish PRM data only applies to POUs that are part of the CAISO BAA.

[3] CEC, 2023 Draft Integrated Energy Policy Report, Appendix E, November 13, 2023 at E-1 to E-2.  Available at: https://efiling.energy.ca.gov/GetDocument.aspx?tn=253086.

[4] On the basis of Electricity Deliveries to End Users by Agency (GWh), Cal Advocates estimates that 86.9% of CAISO load in 2022 was CPUC-jurisdictional; accordingly, Cal Advocates estimates (86.9%*16% PRM) + (13.1%*13.88% PRM) = 15.72% PRM.  The 86.9% share is estimated based on the CEC’s California Energy Demand 2022 LSE and BA Planning Forecast Tables – corrected 3-30-2023, Form 1.1c (available at: https://www.energy.ca.gov/data-reports/reports/integrated-energy-policy-report/2022-integrated-energy-policy-report-update-2.)  For the 13.88% POU PRM estimate, the joint agency Final Root Cause Analysis Mid-August 2020 Extreme Heat Wave 2020 report (January 13, 2021 at 41) found that the aggregate CAISO PRM during outages was 14.9% and non-CPUC LSEs accounted for 9% of the RA share.  Given that the CPUC’s PRM was 15% at the time, we can algebraically infer that the collective PRM of the 9% of CAISO that were POUs was 13.88%.  These assumptions are clearly stale, which only underscores the need for transparency on current PRMs.  

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

Cal Advocates recommends that the CAISO consider requesting a non-binding RA showing from LSEs to report any contracted or LSE-owned RA capacity above the 90% requirement in the CPUC’s year-ahead (YA) showing process.  In other words, if LSEs at the time of the YA showing control RA beyond what is needed to meet the 90% compliance requirement, LSEs could show their net-of-90% resources without that additional capacity being bound by any must-offer obligation to the CAISO.  This approach would acknowledge that there may be movement in the capacity market and LSE positions until the month-ahead showing time frame.  The benefit of this approach is that it would be a stepwise improvement over the use of pure assumptions to estimate the composition of the final 10% of the RA portfolio used for month-ahead compliance.  It is highly likely that modelers would still need to make assumptions about some amount of the final 10% of the RA portfolio; however, using a nonbinding showing to inform the CAISO’s modeling could reduce some of the potential errors. 

Beyond assumptions about capacity showings, one of the more unsettled aspects of any YA modeling exercise for CAISO capacity are the import availability assumptions.  To enhance these assumptions, Cal Advocates requests that the CAISO provide more detailed information on monthly import data.  Specifically, Cal Advocates requests that the CAISO provide monthly RA import showing volumes, disaggregated by resource-specific imports and non-resource specific imports.  While specified imports are contracted (or owned) for RA similarly to in-CAISO resources, non-resource specific imports are often contracted year-by-year, posing a challenge for developing assumptions. 

This request can be accomplished by integrating two workbooks already published by the CAISO on the Reliability Requirements webpage:[1]

• The Historical Year Ahead Resource Adequacy Aggregate Data workbook[2] provides monthly import RA showings, decomposed by intertie. 
• The Historical Resource Adequacy Import Aggregate Data workbook[3] provides annual aggregations of RA import showings disaggregated by resource-specific imports and non-resource specific imports as shown on the “Import RA in MW” and “% of RA By Type” tabs.

Incorporating the monthly cadence into the resource-specific and non-resource specific imports decomposition would help stakeholders understand the RA import dynamics and lay the foundation to potentially utilize import assumptions that vary by month.  This reporting would address Problem Statement 1’s “need for additional consistent, transparent, and timely information on the sufficiency of the RA fleet in the CAISO Balancing Authority Area (BAA).”[4] 

[1] Available at: https://www.caiso.com/planning/Pages/ReliabilityRequirements/Default.aspx

[2] Available at: https://www.caiso.com/Documents/HistoricalYearAheadResourceAdequacyAggregateData.xlsx.

[3] Available at: https://www.caiso.com/Documents/HistoricalResourceAdequacyImportAggregateData.xlsx

[4] CAISO, Resource Adequacy Working Group Discussion Paper, September 5, 2023 at 5.  Available at: https://www.caiso.com/InitiativeDocuments/DiscussionPaper-ResourceAdequacyWorkingGroup-Oct5-2023.pdf.

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

The December 6 Working Group slides’ summary of participant comments on Problem Statement 2 lists Cal Advocates as supporting a re-evaluation of RAAIM.[1]  Cal Advocates additionally supports the following themes:

1. Slice of Day: Evaluate impact on BAA (historical analysis), explore aligning with availability, and more;
2. Review substitution rules;
3. Assess Flex RA; and
4. Consider including / emphasizing UCAP.

A major motivation for developing the CPUC’s Slice of Day framework was the ability to integrate energy sufficiency evaluations into the RA planning framework.[2]  While one stakeholder has expressed opposition to the CAISO expanding its analysis to consider energy sufficiency,[3] there has been notable recent movement in the electricity industry towards considering and examining the issue of energy or fuel adequacy.  For instance, a recent Lawrence Berkeley National Laboratory paper on resource adequacy assessments noted that “RA may need to expand beyond capacity adequacy to ensure energy adequacy.”[4]  Likewise, the National Association of Regulatory Utility Commissioners (NARUC) recommended energy adequacy assessments as a potential tool for regulators doing reliability planning.[5]  The CAISO should continue to explore incorporating an energy sufficiency test into its RA evaluation framework. 

[1] CAISO, Resource Adequacy Modeling and Program Design Working Group slides, December 6, 2023 at 44.  Available at https://www.caiso.com/InitiativeDocuments/Presentation-Resource-Adequacy-Working-Group-Dec62023.pdf.

[2] Decision 22-06-050, Decision Adopting Local Capacity Obligations for 2023-2025, Flexible Capacity Obligations for 2023, and Reform Track Framework, June 23, 2022 at 55; issued in Rulemaking 21-10-002, Order Instituting Rulemaking to Oversee the Resource Adequacy Program, Consider Program Reforms and Refinements, and Establish Forward Resource Adequacy Procurement Obligations.

[3] Northern California Power Agency, Comments Resource adequacy modeling and program design Working Group 1, October 20, 2023 at #2.  Available at: https://stakeholdercenter.caiso.com/Comments/AllComments/5860a092-9299-4cf2-a3f7-60efa5105b32#org-c3c811b1-7775-4b2e-bb02-7aab7af1d42f.

[4] Carvallo, Juan, et al., A Guide for Improved Resource Adequacy Assessments in Evolving Power Systems: Institutional and Technical Dimensions, June 2023 at viii.  Available at: https://eta-publications.lbl.gov/sites/default/files/ra_project_-_final.pdf.

[5] NARUC, Resource Adequacy for State Utility Regulators: Current Practices and Emerging Reforms, November 2023 at 29-30.  Available at: https://pubs.naruc.org/pub/0CC6285D-A813-1819-5337-BC750CD704E3.

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

Cal Advocates appreciates the CAISO’s presentation on the RA metrics that the CAISO currently publishes in real-time on its website. Cal Advocates makes two recommendations regarding these metrics.  First, the Today’s Outlook dashboard would be greatly enhanced by incorporating the ability to do bulk data downloads across multiple months directly via the Today’s Outlook interface.  In the current iteration, Today’s Outlook only allows for downloads of a single day’s worth of data.  To perform bulk downloads, users must track down the corresponding metrics in the CAISO’s Open Access Same-time Information System (OASIS) platform and then reconstruct the Today’s Outlook metric.[1]  Incorporating the capacity to batch download across multiple months (or even just a single month at a time) on Today’s Outlook would be a substantial and welcome improvement in transparency and user-friendliness.

Cal Advocates’ second recommendation is for the Prices tab on Today’s Outlook to include the ability to download prices for all three markets (Day-Ahead, Fifteen-Minute, and Real-Time) across specified time intervals (or at least for a given 24-hour period).  This would allow for expedited analysis of pricing data when OASIS fails to load or when stakeholders do not have the expertise necessary to expeditiously navigate and utilize OASIS’ capabilities for solving problems.

[1] It is frequently unclear precisely how the OASIS metrics map onto the Today’s Outlook dashboard.  Even clarifying the exact OASIS metric (using its OASIS code) shown on a visualization would be a helpful addition.

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

Cal Advocates provides no response to this topic at this time.

6. Please provide any feedback not already captured.

In comments on the November 1, 2023 working group, Middle River Power (MRP) proposed an additional problem statement for consideration.  MRP’s problem statement provides that if LRAs’ RA programs collectively fail to meet a pre-specified standard in a CAISO LOLE study, the CAISO “shall” conduct backstop procurement.[1]  By virtue of its use of the directive “shall”, MRP’s proposed problem statement is a policy proposal rather than a description of a problem.  The proposal development phase of this initiative is the appropriate venue to consider such proposals.  Cal Advocates urges the CAISO to not adopt a problem statement that presupposes a specific policy response. 

[1] Middle River Power, Comments on November 1, 2023 Working Group, November 20, 2023 at 1.  Available at: https://stakeholdercenter.caiso.com/Comments/AllComments/cc4dfcc8-1db5-4ae3-9c13-77c1167d5dd2#org-a931df20-e861-44a6-9af9-eba09cc430bb.  See also discussion of this issue at the December 6, 2023 workshop recording at 3:32:06, available at: https://youtu.be/lXeqNwT2bRw?t=12726.

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

Middle River Power LLC (“MRP”) appreciates the CAISO’s “RA 101” review.  With regards to the CAISO’s implementation of the slice-of-day framework, MRP looks forward to the CAISO’s promised Frequently Asked Questions, to be published in the coming weeks.  MRP encourages the CAISO to strongly consider holding the additional workshop on “edge cases” and providing more Q&A referred to on slide 21 of the presentation for the December 6 working group meeting.

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

As MRP’s proposed problem statement observes, the ISO currently does not determine whether the RA programs of the Load-Serving Entities (“LSEs”) within the CAISO Balancing Authority Area (“BAA”) establish requirements which, if met, would maintain a 0.1 Loss-of-Load-Expectation (“LOLE”).  The CAISO states that it plans to perform RA sufficiency analysis for three time frames.  The first is the year-ahead time frame, the second is the two- to four-year-out planning horizon and the third is for a much longer term.  While MRP believes that the CAISO is correctly concerned with assessing the year-ahead shown RA capacity to determine whether sufficient RA may be contracted to meet RA program needs, the CAISO should acknowledge that this concern stems from the fact that LRA programs, both individually and collectively, do not meet the 0.1 LOLE reliability standard.    This is the primary reason why MRP proposed an additional problem statement highlighting this concern, to maintain reliability in both the short term as well as the mid-term time horizons.

To address the underlying concern that LRA programs do not meet 0.1 LOLE reliability metric, MRP strongly agrees that the CAISO must perform LOLE analysis for the short-term based on the year-ahead shown RA resources.  Additionally, it is critical for the CAISO to perform the same analysis for the mid-term to establish a forward-looking default Planning Reserve Margin (“PRM”).  The default PRM can be a guide for LRAs to base their own PRMs for their jurisdictional LSEs.  If LSEs do not procure sufficient capacity to meet the default PRM, then the CAISO has the capability to utilize its Capacity Procurement Mechanism (“CPM”) authority to backstop additional capacity up to the default PRM amount.  MRP holds that one of the current flaws is that the various LRA programs do not meet the 0.1 LOLE metric and, if and when the CAISO utilizes its CPM backstop authority, the CAISO can only procure backstop capacity up to the capacity requirements which the LRAs have set for their respective RA programs, which ultimately may not achieve the 0.1 LOLE reliability metric.  During the workshop meeting, stakeholders raised the concern that if the CAISO were to procure backstop capacity due to a showing deficiency, then to properly allocate the costs for such procurement, the CAISO must find the cause of the deficiency, which would be a difficult task to perform given that each LRA may utilize different qualifying capacity (“QC”) counting methodologies.  Since the PRM is a function of the QC values of the portfolio, then an LRA-specific PRM can be calculated based on the QC methodology of the LRA.  This is because the effective portfolio that achieves a 0.1 LOLE is the same regardless of the QC methodology.  If an LRA uses a different QC counting methodology, then the CAISO can convert that LRA’s PRM to an appropriate PRM based on that LRA’s QC counting methodology.  This method can be used to fairly allocate the cost of any CPM procurement for all LRAs.  MRP believes the working group should devote additional time to discussing the CAISO’s default PRM before delving into the inputs and assumptions of the year-ahead assessment process.

To respond directly to CAISO’s questions regarding how to conduct its sufficiency analysis for the year-ahead process, MRP suggests that the CAISO consider alternative processes that would allow for robust and meaningful sufficiency analyses.  The CAISO could, for example, modify its RA program to require an annual full year-ahead only showing rather than maintain a partial year-ahead showing and full month-ahead showings.  This would provide the CAISO with the necessary information to determine whether the shown fleet can ensure reliability for the next compliance year without relying on assumptions to assess sufficiency.  LSEs still can manage load migration through their LRA program.

Lastly, the focus surrounding non-RA capacity is only important when the LRA RA programs are unable to maintain 0.1 LOLE.  MRP strongly urges the working group process to focus on ensuring the establishment of RA capacity requirements that meet the 0.1 LOLE metric rather than focusing on non-RA capacity.  California’s Strategic Reliability Reserve program is intended to be utilized when the grid faces higher than 0.1 LOLE conditions. 

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

During the December 6 discussion, the CAISO suggested that MRP’s proposed new problem statement may belong, in a broader context, under problem statement 2.  MRP agrees that problem statement 2 properly notes that the CAISO’s requirements and tools have not been updated in light of evolving market and regulatory structures (e.g., the pending implementation of the slice-of-day framework), but observes that problem statement 2 has been structured to address how the RA fleet is “operationalized”, not how it is secured and determined to be sufficient to meet the 0.1 LOLE target.  Further, problem statement 1 is more focused on the lack of information for the CAISO to determine whether the shown fleet is sufficient, not on the CAISO’s current inability to assess the sufficiency of the shown fleet (though that inability is captured in the first sub-issue).  

MRP offers that the first sub-issue under problem statement 2, which relates to current requirements for RA capacity, better belongs under problem statement 1, to the extent that “current requirements for RA capacity” means “requirements that LSEs must meet for the program(s) to achieve 0.1 LOLE”.  If “requirements” in this sub-issue is intended to mean “those obligations that the RA capacity must meet to allow the CAISO to reliably operate the system”, MRP suggests the CAISO use the term “obligations” instead of requirements”. 

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

MRP appreciates all the metrics published by the CAISO and DMM but offers that those metrics are focused on (1) whether LSEs meet the capacity requirements; (2) how the requirements match up against actual operating conditions; (3) how the RA capacity meets its operational obligations; and (4) the effect the RA capacity has on CAISO energy market performance.  As noted, these metrics do not capture what might be the most important measures of RA program success – does the RA program meet the 0.1 LOLE metric on a forward basis and has the shown RA fleet met the 0.1 LOLE metric historically?  Comparing RA program performance against actual operating conditions may provide a valuable insight into the sufficiency of the program design(s), especially if the actual operating conditions are more severe than those assumed in the program design, but such comparison may not be especially useful if the operating conditions are not as severe as those assumed in the program design.    

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

MRP hopes that the data DMM is gathering can be utilized in the working group process to identify changes that are necessary to support solutions to the problem statements.  As such, MRP requests the data be presented in the working groups in response to the many questions raised in the working group meetings.  This will help inform the stakeholders and help the group to come up with solutions to construct the CAISO’s RA program.

6. Please provide any feedback not already captured.

MRP has no additional feedback.

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

Northern California Power Agency (NCPA) has no additional questions or feedback at this time. 

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

NCPA believes that the 90% annual showing requirement in place today is appropriate, providing CAISO with the information needed to evaluate the sufficiency of the RA fleet on a year-ahead basis. The annual showing requirements in place today have historically resulted in a high degree of reliability in the CAISO BAA, but also provides additional necessary flexibility for LSEs to further optimize and augment their portfolios to account for changing market and resource conditions prior to the summer period.

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

NCPA strongly believes that the current CAISO Tariff rules that delegate certain authority to each LSE’s applicable jurisdictional authority or Local Regulatory Authority (LRA) are still relevant and important.  NCPA strongly believes that each applicable LRA is best situated to establish rules and requirements that support system reliability and account for the unique operations or portfolios of differently situated LSEs.  A one size fits all approach is not appropriate for a subject as complex and detailed as Resource Adequacy policy, and recognition of individual LRA authority has resulted in a high degree of system reliability historically.  Please also see NCPA’s response to Question 6 below.

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

NCPA believes that the current RA metrics provided by the CAISO are useful and sufficient to track the status of the fleet.

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

NCPA has no comments at this time.  

6. Please provide any feedback not already captured.

NCPA understands concerns expressed by other stakeholders that visibility into what amount of installed capacity is currently not contracted as RA is limited.  This limited availability of information is commonly driven by the need to protect commercially sensitive information.  It is no surprise that operational capacity has become more scarce in recent years throughout the western interconnection due to a number of factors including, but not limited to, resource retirements, backlogged interconnection queues delaying new resource availability, and the changing composition of grid assets (e.g., solar and wind). There are many reasons why a generator may elect not to be committed as Resource Adequacy capacity (e.g., RAAIM, must offer obligations, bid insertion, overall lack of control and flexibility), so any sort of enhanced reporting that is focused on non-RA transparency must be carefully balanced with concerns regarded the release of commercially sensitive information, and how the release of such information may impact market behavior.  LSEs continue to have the ability to gauge the RA market by submitting requests for proposals and review responses.  NCPA believes that the current policy focus should be on the root causes of the current market scarcity that exists in the West, including focusing on removing barriers for new resource interconnection and development. 

NCPA acknowledges that evaluating an LSE’s PRM against a .1 LOLE could provide useful information that a LSE can use for planning purposes, but imposing that standard as the PRM introduces a number of other questions and concerns. The preliminary results from of this type of analysis have been varying, and do not appear to result in a material change in the PRM (as compared to the standards adopted today).  The level of additional detail sought via such a complicated study methodology will not ultimately address the underlying issues that the market is facing today.  Rather, the key focus should be finding ways to enable timelier and more efficient interconnect of new generating capacity to the system.  For so long as new resources remain trapped in a clogged interconnection queue, raising PRMs (if application of such a new standard would in fact raise PRMs) merely allows existing generation owners to charge higher prices for the megawatts they offer, further exacerbating high prices to ratepayers (but not improving reliability). NCPA understands that .1 LOLE is a common metric that is used in the industry, but there is no industrywide consensus or firm conclusion of its effectiveness. Calculating the .1 LOLE is subjective and requires various assumptions that could bias the result in one way or another.[1]  Rather than expending valuable time and resources simply looking for another method of calculating PRM that will likely not be significantly more effective that what is in place today, CAISO should focus it full effort and attention on addressing the underlying root cause of the current scarcity issues related to new generator citing, interconnection and development.

NCPA has never made a secret of the fact that its LRAs have voluntarily adopted CAISO’s default 15% planning reserve margin for operating reserves, forecast variability, and forced outages. This has served NCPA well for nearly 20 years with limited service interruptions despite several recent extreme heat events described as 1 in 20 or 1 in 30[2]. In any case, NCPA has other strong incentives to be resource adequate. As a load-following Metered Sub-system (MSS), NCPA is required to balance its load and resources in real time and faces severe financial penalties if it fails to do so. The LRAs governing NCPA members are monitoring the CEC proceeding under AB 209 and are continually made aware of any reliability issues. Individual LRAs are in the best position to assess what is needed to support reliability for customers within their jurisdiction.

For example, over time NCPA’s members have developed diverse resource portfolios consisting of multiple dispatchable and base load resources (e.g., large hydro, thermal, and geothermal) that when operated as a portfolio are well positioned to continue to reliably and economically serve its membership for years to come. NCPA continues to add additional dispatchable resources to its portfolio as a result of its LRA directed long-term procurement goals and objectives. NCPA’s LRA adopted Resource Adequacy counting rules do not differ from the CAISO default counting conventions in a way that affects reliability. Instead, they reflect the fact that NCPA has years of historical data on the performance of its resources and can accurately assess how they will perform and how much energy they will have available when required to support system reliability. In short, NCPA’s PRM and counting criteria are designed to support reliable electric service to its customers, and reflect any unique operating characteristics and capabilities of its resource portfolio. NCPA is extremely doubtful that adopting a uniform criteria will result in greater reliability for its customers, and cost allocation alone should not be the primary driver for changing rules that have worked well to support reliability in the past.

While questions about how CAISO will allocate the costs associated with CPM dispatches are legitimate, CPM dispatches for RA shortfalls remain rare and infrequent. NCPA requests that CAISO provide more transparency for its CPM process, decision making, and cost allocation by using August 2023 CPM as a case study in a future working group. Stakeholders need to better understand the CPM process in order to identify potential area of improvement, efficiency and cost effectiveness. It should be possible to honor jurisdictional rights while ensuring fair cost allocation.  Due to the historically rare occurrence of CPM procurement, the existing Resource Adequacy rules should not be changed or modified solely due to concerns with CPM cost allocation, this would be equivalent to a “tail wagging the dog” approach to the perceived issue or problem.

[1] See, e.g., Brattle Group, Resource Adequacy Requirements: Reliability and Economic Implications (2013), https://www.ferc.gov/sites/default/files/2020-05/02-07-14-consultant-report.pdf.

[2] https://www.caiso.com/Documents/Final-Root-Cause-Analysis-Mid-August-2020-Extreme-Heat-Wave.pdf p.4

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

PG&E appreciates the overview of the current RA program presented by CAISO in response to stakeholder requests from previous working group discussions. 

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

PG&E supports CAISO’s goal of improving transparency through analysis but seeks confirmation on the utilization of modeling results. It is important to confirm that increased transparency aims to provide timely information for stakeholders to assess the RA fleet without implying specific CAISO procurement actions. While supporting more analysis for added transparency, PG&E recommends that CAISO confirms the objectives of the RA only analysis and outlines how the assessment results should be used by stakeholders. 

PG&E is also concerned that the CAISO’s focus on only modeling RA resources in the year-ahead timeframe will almost certainly result in the CAISO identifying many more instances of supply shortages than would be expected if the modeling included a more complete set of resources.  Missing resources include the 10% of RA resources that are not required to be shown in the annual timeframe and resources that do not meet requirements to qualify as an RA resource but do participate in the CAISO’s market. The proposed analysis would provide a biased estimate of the actual LOLE and indicate a greater likelihood of supply shortages than reality. PG&E seeks clarification on CAISO’s safeguards to prevent the misuse of such an analysis for justifying unnecessary and costly procurement of resources.   

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

PG&E recommends that CAISO allocate time to address the high-priority issues outlined in problem statements two and three, as indicated by stakeholders from previous comments. These include: 

• Counting rules (UCAP) and RAAIM revisions. 

• Evaluation of processes and options for outages and substitute capacity. 

• Assessing the necessity of Flex RA. 

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

No comments at this time. 

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

No comments at this time. 

6. Please provide any feedback not already captured.

No comments at this time. 

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

The Six Cities found the CAISO’s overview to be helpful in ensuring that stakeholders participating in this initiative have a general understanding of the CAISO’s RA program processes and requirements.  It appears that stakeholders had a number of questions regarding the CAISO’s approach to exercising its Capacity Procurement Mechanism (“CPM”) authority.  It may be useful at an appropriate point within this stakeholder process to more fully document how the CAISO administers the CPM program and determines when backstop procurement, including procurement to address RA deficiencies, is appropriate. 

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

At this time, the Six Cities have comments regarding certain topics discussed during the December 6^th working group meeting.

Study scope – Preliminarily, the Six Cities are supportive of many aspects of the framework for analysis of RA sufficiency as discussed on December 6^th, although they continue to consider certain elements of the CAISO’s proposed analysis.  At this time, the Six Cities:

• Support consideration of a non-mandatory and non-binding, “soft” RA showing in the year ahead timeframe above minimum requirements to enable the CAISO to anticipate potential resources that may be available and included on monthly showings when they are due.  To be clear, the Six Cities are not definitively advocating for the development of such a showing at this stage; rather, the Six Cities support consideration of this approach as one way to enhance the accuracy of the CAISO’s short-term analyses.  If the CAISO and stakeholders ultimately determine that such a showing is appropriate, then the Six Cities suggest that the showing be permitted to include energy-only resources under long-term contracts with LSEs that, but for lack of deliverability status or maximum import capability (“MIC”) allocations, cannot be designated as RA, but could otherwise meet RA requirements (such as compliance with must-offer obligations).  Relatedly, the Six Cities concur with comments suggesting that there is likely a need to consider how to address storage resources with adjacent energy-only resources that will be used for battery charging.
• Support consideration of strategic reserve resources and reliability demand response resources (“RDRRs”) in the assessment.  As discussed during the December 6^th meeting, excluding these resources from the short-term assessment as though they do not exist implies that the CAISO must meet a heightened planning standard.  In response to this concern,  CAISO staff asserted that they did not want to “plan to get into emergency conditions.”  To be clear, the CAISO would not be evincing an intent to place itself into system emergencies by recognizing that it does have access to a suite of resources that are specifically intended to mitigate system emergencies and can be used for this purpose.  The Six Cities encourage the CAISO to continue to evaluate ways to acknowledge these resources in its short-term assessments so that CAISO LSEs are not exposed to increased procurement requirements and associated costs.  There may be a way to reflect these resources in the assessments that is reasonably indicative of their availability under specific conditions.

Backstop procurement – Several commenters addressed the CAISO’s backstop procurement authority under the Capacity Procurement Mechanism (“CPM”) program.  The Six Cities support the use of CPM in appropriate circumstances to address reliability needs, including where RA deficiencies pose risks to the CAISO grid.  As the Six Cities understand the CAISO’s implementation of its CPM authority, the CAISO’s practice is to engage in backstop procurement in situations where there is an identified trigger for the exercise of backstop authority, AND there is a discernable reliability need to be addressed by the procurement, based on the totality of circumstances, including updated predictions of system conditions and considerations of both actual shown RA in the aggregate and resources that are expected to be available during the relevant time period.  Proponents of automatic or non-discretionary backstopping appear to be motivated by a misplaced understanding that any and all RA deficiencies must be addressed to assure a certain level of procurement, irrespective of any other considerations, including actual reliability needs.  The Six Cities support retention of the CPM program as one that is discretionary and to be used in situations where there is a reliability issue to be addressed through incremental procurement, but they do not support the CAISO serving as the guarantor for specified levels of procurement.  While it is not known whether implementation of a “must-backstop” requirement would always result in significant additional CPM procurement, it would not help the currently tight bilateral capacity market conditions to have the CAISO competing with LSEs for additional procurement when backstopping may not actually be necessary.

Related to the topic of backstop procurement, the Six Cities note that some commenters have addressed the issue of whether compliance with the CPUC’s slice-of-day program requirements means that LSEs subject to the slice-of-day program should be automatically deemed compliant with CAISO RA requirements.  While the Six Cities understand the motivations of stakeholders in seeking assurances that they will not be subject to two different sets of RA rules, the Six Cities oppose any categorical exemptions from the cost of backstop of procurement for CPUC jurisdictional LSEs that meet the CPUC’s slice-of-day requirements.  It is not reasonable to simply impute any need for backstop procurement to non-CPUC jurisdictional LSEs, especially in situations involving collective deficiencies. 

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

Please refer to the Six Cities’ comments on Problem Statements 2 and 3, submitted on October 23, 2023, in response to the Discussion Paper and the October 5, 2023 stakeholder meeting. 

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

The Six Cities do not have comments on the current RA metrics published by the CAISO at this time. 

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

The Six Cities have no comments on the DMM discussion of sample RA metrics at this time.

6. Please provide any feedback not already captured.

The Six Cities have several comments regarding other topics of the working group process.

First, the Six Cities do not support the proposal regarding Problem Statement #4 from Middle River Power.  It appears to be duplicative of the concepts included in other problem statements, but with added prescriptive requirements.  In particular, the Six Cities do not agree that a problem statement needs to encompass the specific LOLE metric that is targeted, and it is inappropriate for the problem statement to specify that the CAISO will engage in backstop procurement if the target procurement level is not met.  These elements are more appropriately considered as possible solutions in the stakeholder process.

Second, the Six Cities are concerned by the use of polling during working group meetings, including in this initiative, to conclusively identify topics that are priorities.  It appears that the analysis of written stakeholder comments discussed during the December 6^th meeting did not include a consolidation of topics across the range of stakeholder input, which is why the CAISO consequently did not discern differences in priorities among the comments.  However, using the polling system to obtain any information other than non-binding, indicative preferences during the meeting is not appropriate.  In particular, it does not account for duplicative or multiple comments from the same organization, and there is no means to reconcile different responses from representatives for the same organization during the meeting.  Moreover, for parties such as the Six Cities that are comprised of multiple entities, responding via the polling tool in real time does not allow for sufficient time to coordinate a joint position.  The polling system does not have any way to discern a balancing or weighting among different stakeholder groups, including in situations where this would be appropriate.  Finally, anecdotally, the Six Cities observe that respondents to the poll are frequently a small fraction of participants in the meeting, which could suggest that other stakeholders may likewise view the polling process as less than useful for identifying stakeholder positions as compared with written comments.

With respect to the priorities listed on slide 38 during the stakeholder meeting, the Six Cities cite the following as their top priorities for this initiative:

1. Evaluate current RA – as the Six Cities have discussed previously, the current RA rules are unduly restrictive in several respects, and this initiative should encompass holistic evaluation and consideration of revisions to RA requirements and qualification rules that acknowledge the current capacity market conditions and the inability of LSEs to obtain additional incremental RA capacity resources at reasonable costs in the near term.  This topic includes counting rules (i.e., qualifying capacity requirements and the net qualifying capacity requirements) as well as rules relating to the existing showing requirements. 
2. RA substitute capacity and UCAP/RAAIM – recognizing that these are listed in the CAISO’s slide as two topics, the Six Cities support prioritizing consideration of improvements in how RA capacity resources are eligible to be counted, whether the RAAIM program is accomplishing its objectives and continues to be necessary or should undergo revision, and the CAISO’s substitution rules.  The Six Cities see these concepts as interrelated, and they are also tied to topic #1 immediately above (i.e., evaluation of current RA). 
3. LRA alignment with CAISO RA – The Six Cities acknowledge that CPUC-jurisdictional entities have an interest in ensuring appropriate coordination between the CPUC’s rules and the CAISO’s requirements.  At the same time, the CAISO’s RA program must continue to recognize and preserve the authority of non-CPUC LRAs with respect to RA policy and their LSEs to engage in resource planning and procurement consistent with the policies set by their LRAs. 

Finally, the Six Cities have concerns with the CAISO’s recent decision to consider consequences for EIM and EDAM resource sufficiency evaluation (“RSE”) failures as part of the RA initiative.  Under the current RA design, an RA deficiency is very different from an RSE deficiency, and it would be inappropriate to conflate the two potential types of deficiencies and related consequences.  While it may be that changes in the RA program may result in closer alignment of RA and RSE requirements, this initiative should not establish any a priori assumption that an RA deficiency should be equated with contribution to an RSE deficiency.

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

SCE appreciates the CAISO’s RA 101. It was extremely helpful to make sure that all stakeholders have a basic understanding of how RA currently works.

SCE believes it would be informative for the CAISO to provide information about when in the past the CAISO has backstopped RA, the analysis that was used to determine that it was necessary to backstop, and how the costs were allocated. This historical information should include both annual and monthly backstop, as well as system, local and flex RA.

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

SCE believes that the CAISO should perform its own analysis of the reliability of the RA resources because of the different requirements for different LSEs. The CAISO is responsible for the reliability of the grid and with different LRAs having different RA planning reserve margins, counting methodologies, and, as of 2025, different RA regimes, it is important that the CAISO provide an overall assessment of how well the disparate programs combine to achieve reliability for the CAISO grid.

SCE believes that a short-term RA study based on the shown RA in the annual showings, with reasonable assumptions to fill out the remaining 10% of RA makes sense. As the CAISO has suggested, this should likely include extra RA available on resources already shown on RA plans, new resources scheduled to come online, and resources that have been shown in previous years. The exact composition of the remaining 10% is likely not as important as ensuring that 100% of the RA needs are used. At this point, it does not seem as important to try to estimate the RA for the non-summer months. As we move forward in time this will likely become more important, but further discussions will be needed as to how to fill in the amount of system resources for those months that are not shown on local RA plans.

SCE also agrees that the CAISO should be studying resource adequacy in the 2-4 year and 5-10 year ahead time frames. These studies should use existing resources, authorized procurement, and IRP forecasts for the longer time frame. In performing these analyses, the CAISO should also examine the situation in the WECC to provide information on how much import RA the CAISO can count on, and how many resources within CAISO may be used to meet the RA needs of non-CAISO entities.

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

SCE has no new comments.

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

As discussed above, SCE would like to see the CAISO provide an annual short-term RA study based on the RA annual showings, as well as more longer term studies. SCE would also appreciate additional information about the annual and monthly determination of the need for backstop capacity.

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

6. Please provide any feedback not already captured.

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

No comment on this item.

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

Terra-Gen observes that CAISO’s discussion during the RAMPD WG has identified that it is hoping to develop additional, annual ex-ante RA assessments, beyond its current Summer Assessment timeframe. CAISO has also noted it is seeking input on addressing the significant challenges with the 90% annual RA showing requirement, and how to make appropriate assumptions to allow it to perform such further ex-ante RA assessments. Terra-Gen does not believe that CAISO should attempt to make assumptions or extrapolation of these 90% annual RA showings as an option to allow for an ex-ante annual RA assessment.  Such an approach would create an inefficient workload for CAISO staff to perform an assessment that will result in inherently inaccurate information since the analysis will always be based on the 90% annual RA showing requirement and require assumptions that would consistently be subject to change in subsequent monthly RA showings.  Further, due to these inherent inaccuracies, this approach will ultimately prove unhelpful for informing and understanding the true reliability and resource adequacy levels that will be provided in the pending compliance year due to these assumptions and subsequently changed resource mix included on monthly RA showings.

Terra-Gen recommends that CAISO should not try to extrapolate the 90% annual RA showings at all, nor make other related assumptions about the resource technologies that will ultimately be shown in the LSE’s 100% monthly RA showings in the annual advance timeframe.

CAISO should recognize the challenge related to this issue is rooted in broader structural challenges, specifically the monthly RA showings are subject to change and the final resource mix may change enough to invalidate any of the assumptions utilized to artificially create any 100% annual RA showing simply for analysis purposes. Terra-Gen suggests that rather than attempting to develop 100% annual RA showing assumptions or other half-solutions to these challenges, instead CAISO should acknowledge the intractability of 90% annual RA showings and the inherent inaccuracy in attempting any modeling efforts performed ex-ante in these advanced timeframes without broader structural changes.

Terra-Gen requests that CAISO carefully consider and acknowledge these structural challenges and make a fulsome effort, in close coordination with the California Public Utilities Commission (CPUC) and stakeholders, to explore significant structural changes to the overall RA program and showing process intended to address these timing-related reliability assessment challenges. For instance, Terra-Gen recommends that CAISO consider and discuss the potential for making a program change with the CPUC to move these full showing to a timeframe further in advance to provide adequate time for RA modeling and analysis to inform reliability levels and CPM needs. Terra-Gen suggests the CAISO and CPUC jointly consider the following potential RA showing timing changes:

1. Require 100% annual showing requirement (increase from 90%);
2. Require 100% seasonal or quarterly showing requirement with 60 or 90 day in-advance showing timeframe; or
3. Require 100% monthly showing with a deadline change to at least 90 days in advance.

Terra-Gen also believes that, until or unless more structural changes are made to the overall timing of the RA program that resolve the above noted challenges, CAISO should only perform an ex-post  Loss of Load Expectation (LOLE) analysis to validate the actual full monthly RA showings and its resulting ability to meet a 0.1 LOLE standard, i.e., "1-day-in-10-years" LOLE, as an ex-post review of the overall RA fleet and ability to provide a reliable system. Such an ex-post analysis would provide insights into the performance of the RA program and help identify issues that should be addressed going forward.

Terra-Gen requests CAISO pursue the suggestions above before attempting to develop other less effective, less accurate approaches, that may appear easier to develop and implement, but would not provide truly useful ex-ante annual RA assessment information.  If CAISO is unwilling or unable to pursue these suggestions or similar significant structural showing timing changes or is otherwise unwilling or unable to entertain such an approach, it should clearly explain its justification for avoiding such approaches for transparency to support a robust stakeholder process.

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

Terra-Gen requests CAISO’s Problem Statement 2 be prioritized over Problem Statement 1 and 3.   

Terra-Gen recommends CAISO focus the bulk of its energy and efforts in this initiative on Problem Statement 2 and related sub-issues. Specifically, Terra-Gen requests CAISO include and elevates the following items as higher priority than other issues that CAISO has previously identified:

1. Develop a proposal to reduce the Net Qualifying Capacity (NQC) for thermal units to properly reflect ambient temperature derates that are not accounted for in existing RA NQC accreditation rules, nor existing CPUC Qualifying Capacity (QC) methodologies;
2. Perform a comprehensive review of changes needed to the RA program structure and timing to allow reform of rules for substitution and planned outages, specifically with the goal to remove the previously developed 100% Planned Outage Substitution Obligation (POSO) requirement;
3. Consider a proposal to remove CAISO’s Resource Adequacy Availability Incentive Mechanism (RAAIM) and develop an Unforced Capacity (UCAP) performance reduction to better account for forced outage impacts in resource NQCs (Note: CAISO already has existing Tariff authority for NQC reductions for performance criteria that could be applied to account for forced outages in resource accreditation)[1];
4. Develop a proposal for the removal of CAISO’s Flexible RA product and requirements; and
5. Perform a comprehensive review of the impacts of the CPUC’s Slice-of-Day RA reform to identify related changes that are needed to coordinate the CAISO Tariff and Business Practice Manuals (BPM) with CPUC RA reform.

Terra-Gen believes that the CPUC and other Local Regulatory Authority (LRA) RA procurement directives and requirements should guide procurement and investment decisions in longer-run timeframes and that the CAISO’s single year RA program design is not sufficient nor appropriate to provide such signals to guide those long-run decisions for capital deployment and maintenance expenditures. Therefore, Terra-Gen suggests that CAISO should strike the reference to “Inefficient procurement and investment (e.g. maintenance and capital upgrade) decisions” from Problem Statement 2.

Terra-Gen also notes that CAISO should focus keenly on California RA program design and reforms to fix existing flaws and structural challenges prior to any efforts is made toward considering how to translate and transact with the Western Resource Adequacy Program (WRAP). Therefore, Terra-Gen recommends CAISO strike the reference to “… and the translatability and transactability of WRAP” included in the sub-issues for Problem Statement 2.

[1] See CAISO Tariff, Section 40.4.5 Reductions for Performance Criteria; Section40-RADemonstration-for-SchedulingCoordinatorsintheCAISOBalancingAuthorityArea-asof-Nov1-2023.pdf.

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

No comment on this item.

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

No comment on this item.

6. Please provide any feedback not already captured.

Terra-Gen supports further consideration and development of Problem Statement 4, submitted by Middle River Power, regarding LOLE metrics and ensuring the CAISO footprint meets the industry best practice generation adequacy standard of 0.1 LOLE. This recommendation is similar to aspects of Terra-Gen’s response to Question 2 noted above. To reiterate this position, Terra-Gen notes it supports the CAISO developing a process for regularly performing an ex-post analysis to review the actual monthly RA showings for prior annual periods following each year. Such an analysis would regularly provide needed information for interested parties to evaluate the true effectiveness of the RA program, and specifically provide transparency regarding the shown RA fleet and the ability of the RA program to provide adequate resources to achieve the 0.1 LOLE standard.

Terra-Gen also notes that the CAISO has not addressed outstanding RA issues regarding hybrid resources since its first and only initiative that developed rules and policies for hybrid resources, completed in 2021[1].  Several outstanding issues that were deferred have not been addressed since that effort concluded, including hybrid resources interactions with RA Must Offer Obligations (MOO), Ancillary Services (AS), Flexible RA RAAIM, and the use of outage cards and dynamic limits for signaling unavailability to the CAISO market and operators by hybrid resource Scheduling Coordinators. Terra-Gen requests CAISO scope an item in this initiative to conduct a comprehensive review of RA related issues impacting hybrid resources participation in CAISO’s energy and AS markets. If CAISO declines to scope hybrid resource RA issues into this RA initiative, then it should acknowledge there are outstanding issues stemming from items that were deferred from the initial hybrid resources initiative and commit to a review and consideration of hybrid resource policy updates through an upcoming storage and hybrid resource focused initiative.

[1] See CAISO Hybrid Resource Initiative; https://stakeholdercenter.caiso.com/StakeholderInitiatives/Hybrid-resources.

1. Please provide your organization’s feedback or any additional questions requesting clarification on the ISO’s RA 101 overview of current showing mechanics, data inputs, and the CPM processes.

WPTF appreciates the continued discussions provided at the latest working group meeting. We still believe additional clarification is warranted, and documented in the Business Practice Manuals, regarding what values the LSEs and Suppliers are supposed to provide the CAISO on monthly showings. We understand the CPUC will provide the CAISO with one value for each resource. Additionally, we comprehend what the LSEs will include in the CPUC’s SOD tool and showings for each resource. However, what remains unclear and undocumented is what value the LSEs will use from the SOD tool for each resource and show to the CAISO for the annual and monthly showings. Including this last piece of information in the CAISO documentation will be extremely useful in helping LSEs and suppliers submit their annual and monthly showings to the CAISO.

2. Please provide your feedback to the modeling discussion. The ISO is particularly interested in feedback on the current gaps or future objectives with modeling as it relates to the scope, models, methodology, assumptions, time frame, and suggestions on assumptions the ISO should make in the year ahead time frame (e.g., assessing if there is sufficient capacity if only 90% of system resources shown).

WPTF continues to appreciate the CAISO’s thought work on the modeling efforts undertaken to date. Like any modeling endeavors, certain assumptions must be made. Initially, the task involves determining what assumptions to apply for the remaining 10% in the year-ahead timeframe model. The concept of non-binding advisory showings is intriguing and warrants further consideration. WPTF expresses interest in exploring historical data to ascertain whether only 90% of requirements were historically shown or if the figure might be higher.

An additional consideration pertains to the discussion around the inclusion of energy-only resources. It is emphasized that the evaluation of the RA fleet aims to determine if the RA capacity suffices to meet the system's needs, and therefore, energy-only resources should generally not be included. However, exceptions may be considered, especially in cases where the CPUC includes charging requirements for storage resources that could potentially be fulfilled by energy-only resources.

There was also deliberation on how to treat SRR (Strategic Reliability Reserve) resources. While WPTF understands the perspective of some to include such resources, we do not believe they should be included in the RA year-ahead model, emphasizing that SRR resources are state-procured emergency resources and, it is our understanding, not solely dedicated to the CAISO. While it could be interesting to analyze potential shortfalls in comparison to SRR capacity post-evaluation, SRR resources should not be incorporated into the primary study.

Lastly, there is an outstanding question revolving around the actions CAISO should take once modeling results are complete and identify a potential capacity shortfall. This inquiry delves into the jurisdictional roles of each agency involved in the RA space. WPTF suggests that it would be beneficial for the CAISO to clarify its jurisdictional authority regarding potential actions based on LOLE (Loss of Load Expectation) study results. It is acknowledged that changes in jurisdictional authority might pose significant challenges, but at the very least, submitting the study results to CPUC proceedings could aid in informing CPUC decisions and actions to address any identified shortfalls.

3. Please provide any feedback that was not already captured in response to the Problem Statement 2 (program design) and Problem Statement 3 (cost causation).

The recent workshop included a brief discussion around prioritizing UCAP in the ongoing efforts. Notably, the CAISO had previously explored a similar path during the last round of RA enhancements. However, WPTF recalls that UCAP did not progress during that period, primarily due to jurisdictional roles. WPTF holds the belief that UCAP may fall within the jurisdictional authority of the CPUC rather than the CAISO. Seeking confirmation of this understanding, the question arises as to whether it is worth revisiting UCAP in the current effort, given the potential need for significant changes to RA jurisdictional authority. Recognizing the substantial challenge this may pose, it prompts consideration of the feasibility and implications of reintegrating UCAP into the ongoing discussions.

4. Please provide your organization’s feedback on current RA metrics published by the CAISO on CAISO’s Today’s Outlook, the Monthly Summer Performance Report, the Monthly Market Performance Report, or OASIS.

5. Please provide feedback to the DMM discussion of a sample of the RA metrics provided in the Annual Report.

6. Please provide any feedback not already captured.