Comments on Comments on Apr 7 meeting

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

The California Community Choice Association (CalCCA) appreciates the opportunity to comment on the Performance Evaluation Methodology (PEM) discussion at the April 7, 2025, meeting of the Demand and Distributed Energy Market Integration (DDEMI) Working Group (Working Group). The Working Group should prioritize problem statements 1, 2, 4, and 6. CalCCA does not yet have a position on problem statements 3 and 5, which address device-level participation in CAISO markets.

CalCCA further recommends that the California Independent System Operator (CAISO):

• Move the operational details of PEMs from the tariffs to the Business Practice Manual (BPM) to allow flexibility for modifying PEMs to be more effective and adding PEMs to support new technologies or processes for monitoring and evaluating performance;
• Include general descriptions of PEM categories in the tariffs to ensure compliance with Federal Energy Regulatory Commission (FERC) rules; and
• Work with the California Public Utilities Commission (CPUC) and stakeholders to explore opportunities to allow full export of behind-the-meter (BTM) energy storage and electric vehicles (EVs).

Responses to specific problem statements are included below:

1. The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options.

CalCCA agrees that PEMs other than the Day Matching Combined PEM are underutilized and that the CAISO should explore ways to make these other PEMs more useful and effective. The problem statement does not require further refinement.

2. Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices.

CalCCA agrees that existing PEMs do not support participation of BTM batteries and EVs in CAISO markets and supports exploring new or modified PEMs to include exported energy from these technologies. The problem statement does not require further refinement.

3. BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents.

CalCCA does not have a position on this problem statement at this time.

4. CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs.

Flexibility in modifying existing PEMs and adding new ones should be a high priority. The CAISO should move the operational details for individual PEMs from the tariffs to the BPM, while maintaining a general description of the PEM categories in the tariffs to comply with FERC rules. This will provide the necessary flexibility to improve the effectiveness and usefulness of existing PEMs and allow for new PEMs, within the categories included in the tariffs, to accommodate new technologies or processes for monitoring and evaluating BTM DER and demand response performance. The CAISO should not adopt the Department of Market Monitoring’s recommendation to preclude the adoption of any new PEMs. The Working Group should be encouraged to propose, develop, and vet new PEMs that could better reflect evolving market needs than the existing PEMs, especially given that many existing PEMs are underutilized, indicating some unsuitability.

The CAISO should also adopt a new stakeholder engagement process for changes to PEMs in the BPM to ensure all stakeholders have an opportunity to review and provide input. Many stakeholders may be used to engaging in the FERC tariff change process, but may not participate in the BPM update process.

5. Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level.

CalCCA does not have a position on this problem statement at this time.

6. Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules.

CalCCA supports improvements to the control group PEM and agrees that modifications to the registration requirements could increase the use of the control group PEM. This problem statement should be broadened, however, to allow the working group to address other potential challenges including those related to control group formation, pre-event day requirements, control group assessment periods, and error identification, and identify approaches to resolving them. CalCCA is concerned that the control group methodology may not be scalable without modifications or different approaches to the control group methodology that address potential challenges beyond those identified in the problem statement as drafted.

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

CalCCA has no further comments on the PEM discussion at the April 7, 2025, DDEMI WG meeting at this time.

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

Thank you for the opportunity to share input on behalf of the California Department of Water Resources – State Water Project (CDWR-SWP) through the DDEMI working group process. CDWR-SWP generally supports efforts to enhance the suite of Performance Evaluation Methodologies (PEMs) to better reflect evolving grid needs and market dynamics.

As technology continues to advance, CDWR-SWP supports updates to the BPM to accurately capture the capabilities of Distributed Energy Resources (DERs). We also generally support the problem statements presented by stakeholders, particularly those focused on improving PEM baselines and implementing enhancements that accommodate emerging technologies.

However, as new PEMs are introduced or existing ones refined, it is essential that they continue to fulfill their core purpose, capturing the accuracy and reliability of demand response (DR) resource performance to ensure continued support for market efficiency and provide.

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

CDWR-SWP has no additional comments.

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

The California Efficiency + Demand Management Council (“Council”) appreciates this opportunity to comment on the April 7 Demand and Distributed Energy Market Integration (“DDEMI”) working group (“Work Group”).  The Council addresses each of the draft PEM problem statements individually.

- Problem Statement 1: The Council does not dispute that a vast majority of the CAISO’s PEMs are currently not being utilized.  However, this outcome is not so much a problem in and of itself but, rather, one of the symptoms of Problem Statement 2.  The non-utilization of the PEMs as cited in this problem statement reflect design flaws which should be addressed in the context of Problem Statements 2-6.  Therefore, this problem statement should be discarded.

- Problem Statement 2: The Council supports this problem statement as written and recommends the Working Group attempt to address it. 

- Problem Statement 3: The Council supports this problem statement as written and recommends the Working Group attempt to address it.

- Problem Statement 4: The Council supports this problem statement as written and recommends the Working Group attempt to address it.

- Problem Statement 5: This problem statement is incomplete because it does not include the barrier that registering a customer at the service account level represents for customer enrollment by a DER provider and the customer’s registration in the Demand Response Registration System (“DRRS”).  Slide 34 in the Working Group slides identifies this issue, so this is not a new concept.  The Council recommends that this problem statement be revised as follows: “Requirement for registration of locations to be at the service account level causes high customer drop-out rates and prevents aggregators from developing resources at the customers device level.”

- Problem Statement 6: The Council generally supports this problem statement but cannot speak to potential conflicts with consumer data privacy rules.    

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

The Council has no additional comments.

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

Comments on Demand and Distributed Energy Market Integration

Department of Market Monitoring

May 1, 2025

Summary

The Department of Market Monitoring (DMM) appreciates the opportunity to comment on the Demand and Distributed Energy Market Integration Working Group dated April 7, 2025.[1] In these comments, DMM adds to our previous comments from the ISO’s Demand and Distributed Energy Market Integration (DDEMI) Working Group Meeting on February 5 and March 28, 2025.[2]^,[3] DMM includes additional comments on the issue of baseline methodologies, and asks stakeholders to clarify the need for any additional baselines.

Comments

Demand response baselines are a type of empirically estimated counterfactual, similar to those found in the economic literature to study the impacts of a rule or policy. The goal of such estimation procedures is to provide a “what-if” the policy (or schedule) was not in place, in order to estimate the impact of the policy (or demand response deployment). To estimate these impacts, empirical methods analyze differences in time and space.

The ISO currently offers more than 50 demand response baseline methodologies designed to estimate the performance of a wide range of demand response programs and technologies. Most of these baselines estimate load differences using control groups (space) and day matching (time). The ISO also offers methods with meter generation output and sub-metering arrangements to estimate the performance of specific types of demand response. All of these baseline methodologies were developed through the CAISO stakeholder process. 

In the current initiative, DMM understands that some stakeholders are proposing the creation of new baseline methodologies and enhancements to existing baseline methodologies. As DMM has raised previously, the goal of a baseline is to create a like-for-like estimate when the resources are scheduled, or not, and compare differences to appropriate analogous conditions.[4] DMM understands that existing baseline methodologies were developed with this intent, with extensive stakeholder input in past policy initiatives. Therefore, DMM asks stakeholders why the current baselines that utilize the current estimation procedures are insufficient to estimate the impacts of demand response events, and why the extensive number of existing baseline methodologies are so underutilized.[5]

DMM understands there may be improvements that could be made to the current baselines. However, DMM continues to ask stakeholders to review all existing methodologies, and clearly articulate the need for new baselines that are not already covered with the existing methodologies.[6] This is important, as the introduction of more baseline methodologies requires use of valuable ISO resources, introduces new potential for calculation errors, and creates new considerations for monitoring the behavior and performance of demand response resources.

[1] Demand and Distributed Energy Market Integration Working Group Discussion Paper, California ISO, March 3, 2025: https://stakeholdercenter.caiso.com/InitiativeDocuments/Discussion-Paper-Demand-Distributed-Energy-Market-Integration-Mar-03-2025.pdf

[2] Comments on Demand and Distributed Energy Market Integration, Department of Market Monitoring, February 21, 2025: https://www.caiso.com/documents/dmm-comments-on-demand-and-distributed-energy-market-integration-feb-05-2025-working-group-feb-21-2025.pdf

[3] Comments on Demand and Distributed Energy Market Integration, Department of Market Monitoring, March 28, 2025: https://www.caiso.com/documents/dmm-comments-on-demand-and-distributed-energy-market-integration-mar-28-2025.pdf

[4] Ibid.

[5] Comments on Demand and Distributed Energy Market Integration, Department of Market Monitoring, February 21, 2025, p 3: https://www.caiso.com/documents/dmm-comments-on-demand-and-distributed-energy-market-integration-feb-05-2025-working-group-feb-21-2025.pdf

[6] Comments on Demand and Distributed Energy Market Integration, Department of Market Monitoring, March 28, 2025: https://www.caiso.com/documents/dmm-comments-on-demand-and-distributed-energy-market-integration-mar-28-2025.pdf

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

Please see the PDF attached below the final question for DMM's fully formatted complete set of comments. For the reader's convenience, the complete text of the comments is pasted in response to #1, but there may be some formatting errors.

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

CALSSA appreciates the opportunity to provide feedback on the six proposed problem statements listed here. Our comments and proposed revisions follow each proposed problem statement, with new wording in bold italics and wording that would be removed shown with strikethrough.

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 

As discussed by stakeholders during the April 7 working group meeting, as framed, this problem statement points to a solution of reducing the number of methodologies, rather than addressing the reasons that many of the approved methodologies aren’t used. We suggest reframing the problem statement as follows: 

“The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns that there are barriers to the use of other methodologies that should be understood and should be eliminated or reduced to make the existing methodologies more usable or through developing new methodologies that are more usable on efficiency of the scale of options being offered and accuracy in performance evaluation options.”

2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 

CALSSA strongly supports amending the problem statement from the version in the April 7 working group presentation to include the issue that energy exports are excluded from PEMs. We suggest this problem statement be reworded as follows to clarify that frequent dispatch is useful to the grid and does not distort calculations, and because exported energy is not counted in any of the PEMs, rather than only the MGO option:

“Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, because whose frequent dispatching is valuable to the grid but is discouraged by existing distorts baseline calculations; or such as the MGO option, excludes, and because the existing PEMs exclude the export of energy from behind-the-meter devices.

3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 

CALSSA supports including a problem statement regarding device-level measurement. The current wording does not fully address important aspects of the issues around device-level measurement. We suggest the following rewording:

“CAISO PEMs provide limited opportunities to use BTM device-level measurement, which creates obstacles for eligibility, enrollment, and measurement and verification is not recognized for use in developing baselines for PEM options. Performance evaluations that depend on premises-level measurement and verification using utility-meter data reduce the ability for participants to use device-level data energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents.”

4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 

During the April 7 working group meeting, CAISO staff asked whether stakeholders would like to take up a problem statement related to whether PEMs should reside in BPMs only or in the tariff. If this problem statement is meant to address that concern, CALSSA agrees that it is worthwhile to limit the tariff to general language and have more specific language in BPMs. We also believe that it would be worthwhile to explore whether FERC has given similar guidance about PEMs needing to be in the tariff to other ISOs and RTOs more recently. We are not sure that a problem statement on the topic is needed; if so, the current wording appears sufficient.

5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level.

CALSSA supports a problem statement on the issue of customer enrollment being at the service account level. This problem statement should speak more directly to relevant enrollment barriers. We suggest rewording the statement as follows:

“Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level, including because of the existing “Click-Through”/“Share My Data” registration process is onerous, significantly reducing customer enrollment and participation.”

6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules.

CALSSA agrees that this is an important problem statement for working group attention. We suggest the following rewording:

“Requirement for control group end users to be registered in the Demand Response System limits the ability to create valid and useful control groups use of non-participating end users within a control group and is in conflict with consumer data privacy rules.”

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

The California Energy Storage Alliance (CESA) appreciates this opportunity to comment on the April 7th Working Group. Below are the individual problem statements followed by CESA's comments on how to refine them. 

Problem Statement #2: Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 

CESA’s Comment: Problem Statement #2 references the battery export issue specifically in the context of the MGO option. CESA requests that the specific problem regarding the exclusion of the export of energy from behind-the-meters not be limited to the MGO option. The export issue is also a problem within other PEMs, such as the commonly used 5-in-10 approach. 

Problem Statement #3:  BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 

CESA’s Comment: While CESA agrees on the importance of using device-level measurement for PEM options, the value of device-level measurement is not limited to developing baselines. Furthermore, CESA is not convinced that baselines are the optimal solution. We suggest this problem statement be revised to not prejudge the solution as improved baselines.  

Problem Statement #4: CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 

CESA’s Comment: CESA seeks clarity on Problem Statement #4. Is the problem statement aiming to convey that CAISO is too limited in adjusting the details of all things aside from the baseline methodologies framework? If so, CESA suggests the problem statement be restated clearly, specifying what is included in the problem, rather than specifying what is excluded from it. CESA does agree that tariff changes and FERC approvals do prohibit CAISO from implementing necessary improvements in a timely manner.  

Problem Statement #5: Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level. 

CESA’s Comment: Problem Statement #5 is overly narrow in scope. CESA agrees that the inadequacy of requiring enrollment only at the service account level is a problem. However, the statement should be broadened to include the significant enrollment barrier posed by the "Click-Through" and "Share My Data" processes. Some CESA members have participated in the CAISO markets through the existing PDR model in the past. When these members recruited their customers through the ShareMyData or Click-through process, only a very limited number of eligible customers (10 - 20%) completed the onerous process successfully, resulting in low participation rates from eligible DER assets. CESA suggests reframing the problem statement as follows: "The existing customer registration process—relying on 'Click-Through' and 'Share My Data' mechanisms—is overly burdensome, leading to low enrollment rates and limiting customer participation."  

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

No comment. 

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

Overall, Leap appreciates the diligence with which CAISO’s staff has reviewed stakeholder concerns, and we believe that the problem statements broadly do a good job identifying critical barriers to DR participation in California’s wholesale market. Most of the problem statements listed here are valuable to address, but adjustments could be made to make them clearer and more focused. Leap has identified several such changes below:

1) Leap understands the concern behind this problem statement, but ultimately this problem statement misses the underlying challenge. The reason many approved baseline methodologies are currently underutilized is because issues in the design and implementation of these baselines discourages DR providers from using them. These issues have been identified and discussed across several presentations to this working group, and are largely captured in problems statements #2-6.

Baseline underutilization would more accurately be described as a symptom of these other problems rather than a problem in itself, and it can be addressed by fixing the design issues that the other problem statements are focused on. As such, it’s not necessary to establish this as a stand-alone problem statement, and Leap recommends it be removed.

2)  This problem statement addresses several important issues with existing baseline methodologies, and Leap recommends it be maintained. However, while Leap understands the connecting thread between the different elements of this problem statement, we feel that the issues associated with the 5-in-10 and MGO baselines are different enough that it might be cleaner to separate this into two problem statements, as framed below:

1: "The commonly the commonly used 5-in-10 and 10-in-10 approaches are not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching complicates baseline calculations."

2: "The MGO and EVSE options contain several design elements that limit their attractiveness to DR providers, including overly strict meter accuracy requirements and the fact that they exclude the export of energy from behind-the-meter devices."

It’s appropriate to separate these issues because the solutions do not perfectly overlap. While making the MGO baseline more accessible would allow more frequent dispatches by batteries (due to MGO’s implementation of a zero baseline if a resource doesn’t have enough non-event days to reference for an X-in-10 baseline), it would not allow more frequent dispatching of EVs – or, for that matter, smart thermostats, which can also be impacted by this problem.

Similarly, creating or adjusting baselines that allow for more frequent dispatch (e.g. control group or prescriptive baselines) would not address other barriers preventing the MGO and EVSE baselines from being used more widely. Although there’s overlap between these two issues, addressing one does not fully address the other, so maintaining them as two problem statements is justified.

3)  Although CAISO does have some device-specific baselines right now (i.e. MGO, EVSE), there are opportunities to expand device-level measurements to other devices – or, potentially, develop a device-level measurement baseline that is not technology specific. That seems to be the issue this problem statement is addressing, and as such it should be retained. However, the language could be clarified, along the lines of the re-write copied below.

"BTM device-level measurement is not recognized for use in most PEM options. These performance evaluations depend solely on utility meter data and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents."

4) This problem statement is well-written and important. Although CAISO clarified in the last working group that FERC requirements necessitate keeping some version of the baseline methodologies in the CAISO tariff, there is potentially room to keep certain details of the baselines in the Business Practice Manual to allow for more rapid adjustments. This could be explored as part of this problem statement.

5) CAISO should also move forward with this problem statement, but it could be rephrased for clarity (see below)

"Requirement for all registration of DR customers to complete utility data sharing authorization process hinders enrollment and discourages aggregators from using baseline methodologies that measure developing resources at the customers device level (e.g. MGO, EVSE)."

6) This problem statement identifies a key challenge for investor-owned utilities, but it does not capture all the barriers for third-party providers. The below reframing would allow it to be more inclusive.

"The use of control-group baselines is limited by the requirement for control group end users to be registered in the Demand Response System (which conflicts with data privacy rules), and by the lack of access that third-party DR providers have to non-participant load data for control group comparisons. "

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

Leap would like to point out that some of the issues considered in these problem statements are also connected to separate issues set to be addressed in other discussion topics. Specifically, the problem statement looking at export credit in the MGO baseline has some overlap with the Modified Proxy Demand Resource proposal in the "Economic-based Demand Response Participation Model" discussion topic, which also provides a pathway for DER exports to be credited in RA. As the working group continues to develop these problem statements and eventually build out potential solutions, these linkages should be kept in mind so that any proposed solutions are not too limited in scope.

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

Marin Clean Energy (MCE) appreciates the opportunity to comment on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group meeting. As discussed below, MCE recommends that the Demand and Distributed Energy Market Integration (DDEMI) Working Group:

• Move forward with problem statements: one, two, and four;
• Move forward with problem statement six, subject to revision to reflect other recognized challenges with the control group methodology; and
• Not move forward with, or deprioritize, problem statements three and five at this time given their narrow focus on device-level measurement and registration.

MCE responses to individual problem statements are provided below:

1. The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options.

MCE agrees that tariff approved methodologies are currently underutilized. MCE supports the CAISO exploring the adoption of improvements to, and if necessary additional, PEMs. Reducing barriers for emerging Distributed Energy Resources (DERs) is critical for improving efficiency and accuracy, and for unlocking broader DER participation in CAISO markets. To the extent feasible, MCE encourages the CAISO to consider soliciting proposals for, and authorizing, pilot programs to test proposed enhancements or alternatives to methodologies before adopting changes to tariffs or Business Practice Manuals (BPMs).

2. Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices.

MCE agrees that existing PEMs are in need of refinement to support emerging DER deployment and strategies. MCE supports CAISO exploring and adopting improvements to, and if necessary, additional PEMs. Reducing barriers for emerging DERs is critical for improving efficiency and accuracy, and for unlocking broader DER participation in CAISO markets. To the extent feasible, MCE encourages the CAISO to consider soliciting proposals for, and authorizing, pilot programs to test proposed enhancements or alternatives to methodologies before adopting changes to tariffs or BPMs.

3. BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents.

MCE does not support the Working Group moving forward with problem statements three and five at this time given the narrow focus on device-level measurement and registration. As a load-serving entity currently operating as a demand response provider, MCE does not see the current requirements as a barrier to scaling enrollment. However, a focus on the device-level approach could fracture the fundamental economic premise of DER aggregation at scale by introducing excessive complexity for the CAISO, market participants, and individual customers.

For example, moving to a device-level registration framework may lead to a lack of optimization at a single site, introducing confusion and the potential erosion of economic value of market participation, particularly at the account and individual customer level. If Asset 1 is being operated by one aggregator and Asset 2 is being operated by a different aggregator, and both assets are located at the same site, there could be a scenario where the assets work against each other. Device-level participation may also discourage the use of open-source communication protocols. In a site-level participation model, different devices must communicate with each other, or a central hub, to ensure co-optimization rather than rely on proprietary protocols. Account level participation ensures that all assets at an individual site are working together to achieve a specific objective.

A device-level framework would also require the CAISO to manage, verify, and maintain vast amounts of highly granular data, requiring significant administrative overhead and infrastructure that could increase barriers for participation and slow market growth. In contrast, the current account-level model offers a more practical balance between operational feasibility and costs, allowing for efficient aggregation while ensuring performance accountability.

Relatedly, during the April 7th meeting, the CAISO presented the following draft problem statements regarding demand response (DR) registration challenges: “DR registration is too complex, which limits DR participation or inaccurately reflects DR program capabilities; – DR participation in CAISO markets is constrained by reliance on utility meter data, cumbersome registration processes, and strict metering accuracy requirements that exclude many DERs; – While CAISO has PEM options for device-level performance measurement, current requirements—such as the IOU “click-through” registration process and ANSI C12 accuracy standards—undermine their accessibility and effectiveness.”

MCE agrees that DR participation in CAISO markets is currently limited by strict metering requirements, utility meter data processes, and high latency. MCE also agrees that timely access to utility meter data is a barrier to utilizing performance evaluation options other than Day Matching Combined. Rather than bypassing existing data sharing mechanisms — which already involve established processes and associated costs — priority should be placed on improving access to meter data and other scalable solutions that do not require the CAISO to directly manage individual devices. Efforts to enhance existing data sharing mechanisms are currently underway in California Public Utilities Commission (CPUC) proceedings. Accordingly, MCE would be in support of a problem statement that captures the performance evaluation barriers involving utility meter data,      while recognizing and connecting to the improvement efforts active in other venues in place of problem statements three and five.

4. CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs.

MCE agrees with this problem statement and recommends that the Working Group move forward with seeking additional flexibility in modifying existing methodologies without the need for Federal Energy Regulatory Committee (FERC) approved tariff changes. Specifically, MCE recommends that, to the extent reasonable and practicable, operational details be moved from the tariffs to the BPMs. Doing so will allow for the necessary flexibility to keep pace with evolving market needs as stakeholders expand their offerings and knowledge in this emerging space. If operational details are moved from the tariffs to BPMs, the CAISO should ensure that all stakeholders are aware of proposed changes and the stakeholder review process, by adopting a new stakeholder engagement process for changes to PEMs in the BPM.

5. Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level.

See response to problem statement three.

6. Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules.

MCE generally agrees that the Working Group should move forward with problem statement six, but suggests that the CAISO revise the current statement to reflect other recognized challenges with the control group methodology.

The control group methodology can yield more accurate outputs and reduce exogenous distortions to customer baseline calculations, but its use is constrained by current rules, including the requirement to register non-participant accounts for “matched” control groups.

MCE agrees that the Working Group should address the registration requirement discussed in problem statement six, but MCE would also highlight that this is not the only challenge with scaling the control group methodology. When the Working Group shifts to identifying proposed actions to address the developed problem statements. MCE encourages the CAISO to consider soliciting proposals for, and authorizing, pilot programs to test proposed enhancements or alternatives to the control group methodology before adopting any changes to tariffs or BPMs. This will ensure that there is time to test and learn from solutions in this early phase of DER adoption and market transformation.

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

In the April 7th meeting, MCE asked about the difference between a modification or enhancement to one of the three existing baseline types and the creation of a new methodology. MCE recommends that the CAISO provide clarification on any distinctions between the two and the associated paths forward as it relates to the Working Group’s efforts for forming stakeholder recommendations for policy development. To this end, MCE recommends that the CAISO not preclude the adoption of any new PEMs.

MCE also supports those that have highlighted that the current accuracy standard is too prohibitive and recommends that the Working Group include a problem statement that would allow for amending the Metering BPM, and recommends the working group target a 1% accuracy requirement.

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

Nostromo Energy appreciates the CAISO soliciting stakeholder comments and presentations for their April 7th, 2025, Demand and Distributed Energy Market Integration Initiative Working Group.?And offers the following comments on the proposed problem statements: 

Regarding Problem Statement 1, Nostromo Energy does not believe that having a vast majority of the CAISO’s PEMs being underutilized diminishes market efficiency or PEM accuracy in any way.  The existing PEMs were created in good faith based on the assumptions at that time and the current lack of use may be because those assumptions did not match the actual needs of the developing Demand Response space.  As stakeholders have expressed through the Working Groups, the current lack of use of the existing PEMs does not mean that new or modified PEMs should not be explored. The appropriate path forward is for the working group to seek to understand why the existing PEMs are underutilized and develop solutions that work for DR providers. 

Regarding Problem Statement 2, Nostromo Energy supports this problem statement and believes that the heavy support already communicated by stakeholders such as LEAP, PG&E, and the California Efficiency + Demand Management Council for refining the DR methodologies to be more responsive and applicable to new technologies identifies it as a key priority for the Working Group’s efforts.   

Regarding Problem Statement 3, Nostromo Energy supports this problem statement as written and looks forward to working with the Working Group to address this issue. 

Regarding Problem Statement 4, Nostromo Energy agrees but understands that moving the DR methodology details to a more flexible medium would require FERC approval. 

Nostromo Energy offers no comment on Problem Statements 5 and 6 at this time. 

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

Nostromo has no additional comments 

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

PG&E appreciates the opportunity to comment on the PEM problem statements discussed during the April 7th WG.

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options.  

PG&E believes this problem statement needs refinement before moving ahead with trying to address it.  Much stakeholder discussion in the DDEMI WG has centered around the barriers preventing widespread use of the full suite of PEMs. PG&E requests clarification of the concerns about efficiency and accuracy, which could be in conflict with problem statements regarding modification of existing PEMs or creation of new PEMs. 

Proposed revision: The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns regarding the existence of shortcomings and barriers related to the currently approved PEMs. 

2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices.  

PG&E agrees the WG should move forward with addressing this problem statement. 

3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents.  

PG&E agrees the WG should move forward with addressing this problem statement. 

4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs.  

PG&E does not fully align with this problem statement and suggests it could be reframed to explore the problem of how tariffs, regulations and market rules keep up with a dynamic landscape and evolving market needs for demand and distributed energy resources. 

5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level  

PG&E agrees the WG should move forward with addressing this problem statement. 

6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules 

? ?PG&E agrees the WG should move forward with addressing this problem statement. 

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

PG&E offers the following comments in response to the potential issues raised by CAISO regarding PG&E’s Proposal to Revise Demand Response Control Group Settlement Methodology. 

ISO Issue1: Verifying the control customers haven’t also been registered in another DR program may be challenging without registration requirements for matched control groups. 

1. Prevent selection of control customers enrolled in another DR program 

The matching process, which involves comparing control and treatment customers' loads on proxy days using methods like propensity score or Euclidean distance, can help prevent the selection of customers registered in another DR program.   Should a candidate control customer perform for another DR program during a proxy day, they would not be selected for the control group. To ensure the control group excludes customers enrolled in another DR program, the control group requires frequent updates to capture customers who may enroll mid-event season.    

Additionally, the control group needs to be validated after the matching process to ensure that it accurately represents participants.  The validation process can assist in identifying customers enrolled in another DR program. Should a selected control customer perform for another DR program, the control group would not appear to be valid.  

2. Prevent double counting of impacts 

When control group customers are also enrolled in a DR program, their participation in DR events can affect their load profiles, which might bias the baseline used for comparison. The proposed baseline adjustment would convert the control group methodology into one more similar to a difference-in-difference approach, which can help mitigate this bias by subtracting out the effect of the other DR program. 

ISO Issue 2: Verifying the existence of control group customers may be challenging without registration requirements for matched control groups. 

Verifying the existence of control group customers should occur outside of DRRS, privately by CAISO, through a data request supported by an NDA. 

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

PacifiCorp agrees that the draft problem statements capture the conversations of the working group thus far and doesn’t have anything further to add at this time. PacifiCorp underscores the need for demand response and distributed energy resources to be included in the CAISO markets as they provide flexible demand.  

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

PacifiCorp would like the CAISO to discuss how PEMs may be structured for reliability demand response programs and specifically if there are any differences in how those PEMs are structured.  

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

Peterson Power Systems applauds and supports the CAISO’s leadership in assembling the Demand and Distributed Energy Market Integration (DDEMI) Working Group.

As a leading California based Caterpillar Power Dealership, Peterson Power Systems offers a variety of power producing products including energy storage, spark ignition engines utilizing natural gas, LPG,  bio gas and renewable hydrogen, fuel cells, compression ignition engines utilizing both renewable and conventional liquid fuels, hybrid systems and microgrids of multiple technologies tied together. A majority of the power generation equipment we supply is installed behind the meter.  As a technologically agnostic equipment supplier, Peterson Power Systems has an intimate knowledge of a multitude of technologies’ strengths and weaknesses in various applications.

Due to a number of factors including but not limited to, cost, reliability, availability, capabilities and significant operational limitations of other technology  options, compression ignition engines (also known as diesel generators) remain the gold standard in underwriting the “insurance” necessary to maintain the electrical system and mitigate the potentially disastrous effects of outages at hospitals, wastewater treatment plants, police and fire stations and other critical societal infrastructure.  In fact, certain codes and standards (ie: NFPA 110) require that diesel gensets be included in the design of and construction of mission critical facilities such as hospitals and wastewater treatment centers.  When people, a business or institution purchases insurance, they rarely do so with the intent of using it.  The same is true with compression ignition engines.

California has a sizeable portfolio of emergency stand-by generation exceeding 15,000 Mw’s.  In fact, Governor Newsome has called upon this fleet a few times during his tenure as governor to provide emergency capacity.   

Peterson believes that as part of the DDEMI effort, the CAISO and its stakeholders should study Portland General Electric’s Dispatchable Standby Generation (DSG) program and evaluate implementation of a similar program in CA to address emergency situations, grid interruptions and capacity shortfalls.

Not only has PGE’s DSG program created a significant amount of emergency capacity within their system, but the program incentivizes owners of older generation to upgrade the emissions reduction systems creating a huge environmental win for all in the process.

Here is a link to PGE’s DSG: https://portlandgeneral.com/save-money/save-money-business/dispatchable-standby-generation

Here is a Link to PGE’s DSG FAQ Sheet:

Chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://assets.ctfassets.net/416ywc1laqmd/6xPDM0LVfZrHyuzUbQAMeD/04f107a741a0107a3a51bc821f62891e/dispatchable-standby-generation-faq.pdf

The CAISO has publicly stated that it remains committed to addressing challenges with pathways for advancing demand flexibility and underscores the need to consider further enhancements or new participation models for demand, demand response, and distributed energy resources. 

Portland General Electric’s DSG program is mature, successful and addresses a number of similar concerns shared by the CAISO and other California stakeholders.  Peterson Power Systems would be pleased to participate in the DDEMI to help the group explore the DSG program and assess its potential benefit to California if adopted through a policy initiative.

Peterson Power Systems appreciates the opportunity to submit these comments and looks forward to productive participation in the DDEMI Working Group.

Respectfully,

Greg Lamberg

Director, Transitional Energy Technologies and Regulatory Policy

Peterson Power Systems

galamberg@petersonpower.com

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

1. PGE asks for more information regarding the number of currently approved methodologies. For examples, section 4.13.4 of the current CAISO tariff outlines six methodologies: (1) Ten-in-Ten, (2) Five-in-Ten, (3) Metering Generator Output Methodology, (4) Control Group Methodology, (5) Weather Matching Methodology, (6) Load-Shift Methodology (for BTM storage).
   • PGE is supportive of this problem statement if analysis demonstrates an opportunity for improvement to existing methodologies and their categorizations.
2. PGE supports moving forward with an evaluation of the existing PEMs and believes that the methodologies reviewed and approved by other approved regulators should be allowable for DR market participation – especially for participating EIM BAAs, who are ultimately responsible for any performance shortfalls.
3. No comment
4. PGE supports further exploration of this problem statement, including further review of the 2010 FERC decision cited by CAISO as the basis for requiring tariff changes when adjusting program details. Given the significant market evolution since then, particularly the introduction of the EIM, PGE would like to reconsider this approach.
   For example, the EIM has created a multi-BAA market structure, fundamentally altering the landscape at the time of the 2010 ruling. Because external BAAs are responsible for their own load deviations, PGE is interested in exploring a separate path for EIM entities to employ new baseline methodologies with an appropriate approval process. This could better reflect the distinct responsibilities of non-CAISO BAAs within the EIM footprint and could be modeled after existing tariff sections that outline different rules for various participation models, such as SQMEs and CAISO LSEs.
5. No comment
6. No comment

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

PGE believes that EIM entities should have the flexibility to develop their own approved baseline methodologies, separate from CAISO LSEs. While such methodologies would require approval and follow an established process to prevent arbitrary changes, this process should not require integration into the CAISO tariff, as required for CAISO LSEs.

This approach could reflect existing practices for participating resources, particularly in terms of metering setup. For instance, PGE has distinct internal processes for meter requirements, SQMD acquisition, analysis, and adjustments. These procedures, while specific to PGE, may align with existing data provision best practices. Any such model development would be available for all EIM entities.

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

1) The full suite of tariff approved performance evaluation methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options.  

2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices.  

3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Grid-level performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents.  

4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs.

5) While requirements for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level, current processes by the UDC to validate these registrations would be difficult and would require additional costs to upgrade and modify the Demand Response Registration System (DRRS), which could be mitigated or avoided by adding a separate utility meter to these DERs.     

6) Requirement for control group end users to be registered in the Demand Response Registration System (DRRS) limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules.

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

BTM device-level registrations and performance measurement could improve performance payments for certain DERs but may not reflect actual grid impacts. Performance payments should be commensurate with actual grid impacts.

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

General Feedback:
 

Sunrun appreciates CAISO’s leadership in addressing the existing barriers to enable distributed energy resources to participate in the wholesale market effectively. Overall, we think the draft problem statements do a good job highlighting the challenges around enrollment and performance measurement. That said, a few of them could benefit from further framing to fully capture the issues at hand.

Problem Statement #1:

Sunrun agrees that the existing tariff approved methodologies are underutilized. The existing barriers lead to many participants using the Day Matching option to measure performance. The Day Matching is also not an ideal PEM option with barriers described in the following problem statement. We encourage CAISO to reframe the problem statement and address all challenges why each PEM is not utilized by market participants. 

Problem Statement #2:

Sunrun agrees with the problem statement that existing PEMs are not well-suited for assets like behind-the-meter batteries, and we share the concern that excluding exported energy is particularly problematic. We recommend framing the issues related to “battery export” more broadly across all PEM options, rather than linking them to a single option such as MGO. This approach would provide the necessary flexibility to address performance measurement with battery exports across various PEM structures as appropriate.

Problem Statement #3:
Sunrun agrees that device-level measurement is critical to fairly valuing the performance of distributed energy resources, such as behind-the-meter batteries. We recommend removing the word "baseline" from the problem statement. At this stage, it is important to keep PEM options open and encourage creativity in developing solutions, rather than tying the issue to a specific approach like baselining.

Problem Statement #4:

Sunrun agrees that wholesale market participation rules must evolve to keep pace with changing market needs. We encourage CAISO to pursue the most flexible policy pathways available to adapt to ongoing market developments.

Problem Statement #5:

It’s a positive step to highlight the inadequacy of requiring enrollment only at the service account level. However, the broader enrollment challenges created by the "Click-Through" and "Share My Data" processes are not fully addressed. Based on Sunrun’s experience, nearly 90% of eligible customers fail to complete the current enrollment process, leading to low DER participation overall. We recommend reframing the issue as:

"The existing customer registration process—relying on 'Click-Through' and 'Share My Data' mechanisms—is overly burdensome, resulting in low enrollment rates and limiting customer participation. In addition, requiring enrollment only at the service account level is inadequate, as it fails to accommodate multiple eligible devices under the same account seeking to register with CAISO."

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

Tesla supports the PEM problem statements discussed and appreciates the Working Group’s focus on improving DER participation in the CAISO wholesale market. We also recognize and appreciate the incorporation of prior stakeholder feedback, including Tesla’s comments regarding device-level participation and enhanced performance measurement for behind-the-meter DERs, particularly storage resources.

However, Tesla suggests further clarification for a few of the Draft Problem Statements:

• Problem Statement #2: Tesla agrees that CAISO should enable DER exports to be measured within performance evaluation methodologies. However, CAISO should not constrain DER exports to evaluation solely under the MGO methodology. As discussed during the April 7 Working Group call, traditional baselines such as 5-in-10 and 10-in-10 are not well-suited to BTM resources like batteries and EVs, and do not adequately account for DER exports. While modifying the MGO methodology to support device-level performance measurement and potential exports would be beneficial, Tesla recommends that CAISO also consider other PEM frameworks that can appropriately measure and evaluate DER exports beyond the MGO methodology. This would provide needed flexibility to support a range of DER configurations and participation models.
   
• Problem Statement #3: Tesla supports the Working Group’s identification of barriers related to device-level measurement. Tesla recommends clarifying this Problem Statement to ensure that the Working Group considers not only baseline-based PEMs but also performance methodologies that do not rely on baselines. Device-level performance measurement should be enabled whether the methodology includes baselines or is based on other evaluation frameworks. The Problem Statement should remain neutral at this time to allow for potential innovation in PEM design.

Tesla also recommends that the Working Group consider adding a new Problem Statement (or modifying an existing statement) to address barriers to customer registration caused by complex data-sharing requirements. As Tesla noted in prior comments, processes such as the Customer Information Service Request (CISR) form and Share My Data agreements have proven burdensome and confusing for customers, significantly impeding DER participation. The Working Group should explicitly seek to identify enrollment barriers and propose streamlined solutions that respect customer privacy while enabling broader DER aggregation, including exploring how device-level enrollment and M&V could make such utility data sharing unnecessary for some customers. A Problem Statement on this issue should therefore be included for the Working Group to address.

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion

1. Please provide your organization’s comments on the Performance Evaluation Methodology (PEM) problem statements discussed during the April 7th Working Group. Do you agree the Working Group should move forward with trying to address the problem statement? Does the problem statement need further refinement?

1) The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options. 2) Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices. 3) BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents. 4) CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs. 5) Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level 6) Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules

Voltus appreciates the CAISO’s work in developing these PEM problem statements, which are effective and with which we are generally aligned. A few short comments and refinements are provided below. 
 

1. The full suite of tariff approved methodologies is currently underutilized, leading to heavy centralization for using Day Matching Combined, which raises concerns on efficiency of the scale of options being offered and accuracy in performance evaluation options.

Voltus is fully aligned with this problem statement.

2. Existing PEMs, such as the commonly used 5-in-10 and 10-in-10 approaches, are either not well-suited for emerging DERs like behind-the-meter batteries and EVs, whose frequent dispatching distorts baseline calculations; or such as the MGO option, excludes the export of energy from behind-the-meter devices.

The challenges to counting battery export are not limited to the MGO option. There is a general need to creatively develop a new PEM or new PEMs for net export. In Voltus’s opinion, it is reasonable to consider net export as positive performance (incremental capacity) regardless of how the asset operated in the days prior—i.e., it is reasonable to count net export without reference to a baseline. The modified PDR participation model could utilize a new net export PEM on this principle. (The California Energy Commission's Demand-Side Grid Support program uses such a methodology in Option 3 today.) 

3. BTM device-level measurement is not recognized for use in developing baselines for PEM options. Performance evaluations depend on energy measurement (load and generation) and don’t recognize non-energy metered technologies contributions to load reduction calculation equivalents.

Voltus supports and reiterates the points made by CEDMC in its Device Level Measurement presentation at the April 7 DDEMI Working Group meeting. 

However, Voltus notes that device-level measurement has benefits beyond baseline development, for instance, by allowing direct performance evaluation of net export as incremental capacity, as discussed in the comments to Problem Statement #2 above. Therefore, Voltus recommends editing the first sentence of this problem statement to, “BTM device-level measurement is not recognized for use in PEM.”

4. CAISO’s limited flexibility in adjusting details beyond its core baseline methodologies framework without tariff changes, restricts ability to keep pace with evolving market needs.

Voltus agrees that PEMs’ placement in the tariff rather than in BPMs can delay the development and implementation of market improvements.

5. Requirement for registration of locations to be at the service account level prevents aggregators from developing resources at the customers device level.

Voltus reiterates and supports CEDMC’s comments related to this issue in its April 7 presentation (see, e.g., p. 18 of the consolidated 4/7 slide deck). Accordingly, the problem statement should be slightly broadened to contemplate related registration barriers. Voltus suggests the following addition, either as an additional problem statement or as the first sentence of the current problem statement: “The existing customer registration process—relying on 'Click-Through' and 'Share My Data' mechanisms—can be overly burdensome, lowering enrollment rates and limiting customer participation.”

6. Requirement for control group end users to be registered in the Demand Response System limits use of non-participating end users within a control group and is in conflict with consumer data privacy rules.

No comment on Problem Statement #6.

Thank you for this opportunity to provide feedback on the PEM problem statements.

2. Please provide any further comments related to the Performance Evaluation Methodology discussion from the April 7th Working Group discussion