Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=uesb20
Energy Sources, Part B: Economics, Planning, and Policy
ISSN: 1556-7249 (Print) 1556-7257 (Online) Journal homepage: https://www.tandfonline.com/loi/uesb20
Agent-based Optimization to Estimate Nash
Equilibrium in Power Markets
A. Yucekaya & J. Valenzuela
To cite this article: A. Yucekaya & J. Valenzuela (2013) Agent-based Optimization to Estimate Nash Equilibrium in Power Markets, Energy Sources, Part B: Economics, Planning, and Policy, 8:2, 209-216, DOI: 10.1080/15567249.2011.578103
To link to this article: https://doi.org/10.1080/15567249.2011.578103
Published online: 10 Jan 2013.
Submit your article to this journal
Article views: 118
View related articles
ISSN: 1556-7249 print/1556-7257 online DOI: 10.1080/15567249.2011.578103
Agent-based Optimization to Estimate
Nash Equilibrium in Power Markets
A. Yucekaya
1and J. Valenzuela
21Industrial Engineering Department, College of Engineering, Cibali Campus,
Kadir Has University, Istanbul, Turkey
2Department of Industrial & Systems Engineering, Auburn University,
Auburn, Alabama, USA
In most deregulated power markets, firms bid daily into a day-ahead power market. The auction mechanism, supply and demand, determine the equilibrium at each hour. In this environment, firms aim to maximize their revenues by carefully determining their bids. This requires the development of effective computational methods that help them estimate their competitors’ behaviors under incomplete information. In this article an agent-based method that uses particle swarm optimization is described to simulate the behavior of market participants. Particle swarm optimization is used in the bidding process and an agent-based model is applied to find a Nash equilibrium. Different stopping conditions are used to determine the equilibrium. Experimental results are presented for two power systems. Keywords: agent-based modeling and simulation, electricity price, equilibrium, particle swarm opti-mization (PSO), power markets, strategic bidding
1. INTRODUCTION
The deregulation of power markets aims to increase the competition and hence lower the price of electricity. In a deregulated power market suppliers and consumers meet under the supervision of an independent system operator (ISO) or regional transmission organization (RTO). After generation companies submit their sell offers and load serving entities (LSE) offer their buy bids to the RTO, the next step is to find an equilibrium where supply and demand meet. The equilibrium not only sets the market clearing price (MCP) but also sets the market clearing quantity that makes social welfare an optimum (Bouffard and Galiaba, 2006). The Pennsylvania New Jersey Maryland (PJM) ISO forecasts and publishes the demand for each hour of the next day (PJM, 2011). These forecasts are used by power suppliers to determine their generation positions for the next day and to develop the power component of their sell offers. In the day-ahead market, the sell offers submitted by suppliers and the buy bids offered by demand companies set the MCP for each hour.
Address correspondence to Ahmet Yucekaya, Industrial Engineering Department, Kadir Has University, Kadir Has Street, Cibali, 34083, Fatih, Istanbul, Turkey. E-mail: ahmety@khas.edu.tr
210 A. YUCEKAYA AND J. VALENZUELA
A sell offer consists of pairs of power quantities and prices that the supplier is willing to sell for. A supplier tries to maximize its return by determining an offer that maximizes its revenues (David and Wen, 2000).
When a supplier cannot affect the MCP (price-taker) the supplier can determine its sell offers by considering its generating units separately (Valenzuela and Mazumdar, 2003). In this case, the strategic bidding problem (SBP) consists of finding the proper sizes of the bid quantities and prices. A particle swarm optimization (PSO) algorithm that finds a sell bid for a price-taker is described in Yucekaya et al. (2009). On the other hand, some firms’ offers may occupy an important part in the aggregated supply stack of the RTO and their move on quantities and prices may change the MCP. However, if a supplier decides to bid higher prices to increase the MCP, the RTO could not accept the offer and replace it by the next cheapest offer of another firm. Thus, under an oligopoly, the SBP of a firm becomes more complex and aims to determine the structure of the sell offer while considering the strategic behavior of its competitors. The problem is subject to uncertainties since the bidding is usually a blind auction in which each player has one chance to submit its offer before knowing other firms’ offers. In the PJM market, the SBP needs to be solved before the bidding ends at noon of each day (PJM, 2011).
The SBP has been modeled using different approaches. In Ventosa et al. (2005), models that consider firms’ strategic behaviors are classified into equilibrium models and agent-based models. Game theory is the common approach for equilibrium models. The Cournot, Bertrand, and Supply Function Equilibrium (SFE) game models have been shown as appropriate for representing different power markets. Under the Cournot game, strategic decisions of sellers relate to quantity choices. Customers, who are not active in the game, are represented by a demand function that depends on the price. The sellers make simultaneous optimal choices of quantities to respond to their competitors’ optimal choices. In the Bertrand game model, firms choose price as a strategic decision variable instead of quantity. The model introduces consumers as an active participant of the game with the assumption that customers search the market to find the lowest price. Thus, in the Bertrand model, firms simultaneously choose prices and consumers choose where to buy. This dynamic interaction of the game leads to a market-clearing price equal to the marginal costs of sellers. The SFE is a much newer model and it was proposed by Klemperer and Meyer (1989). It consists of firms expressing their bids in terms of a price-quantity curve in the absence of certainty. The SFE model is considered more accurate to represent the short-term power markets since it reflects the bidding rules in several markets where players submit price-quantity offers.
Several studies that use the SFE model have been reported in the literature. In Baldick et al. (2004), the SFE model assumes that the supply functions are linear and the strategic players in the market have a price and capacity cap. It is shown that the SFE is more effective than other approaches in terms of representing and reaching equilibrium. In a similar study, the market players are modeled as non-degreasing supply function providers (Rudkevich, 2003). According to Baldick (2006), equilibrium models are analyzed in terms of transmission network, generator cost function and operating characteristics, bidding, demand, and uncertainty.
There is also a vast literature on the use of agent-based models to represent power markets. An agent-based simulation approach for modeling the day-ahead power trading in the United States wholesale power market is described in Sueyoshi and Tadiparthi (2005, 2007). Agents bid into the market and they update their bidding strategies at each run of the simulation based on a learning factor until equilibrium is reached. The effectiveness of the approach is shown by using data from the PJM west. Nicolaisen et al. (2000) developed agent-based model for the wholesale electricity market operating in a short-term environment under capacity conditions and double auction rules. Another simulation model that uses a multi-adaptive agent model for generators bidding in the United Kingdom power market was developed by Bagnall (2000). It is shown that agents learn bidding strategies in a manner similar to their real world behaviors. Botterud et al. (2006) used
an agent-based simulation model to show how companies use market power during the electricity market bidding process.
In this article, we propose an agent-based model to represent the competition among power suppliers. We assume that the demand for the day-ahead is known by all power suppliers who compete to obtain a chunk of that demand. The MCP is determined based on accepting the offers that provide lowest-cost power to the market. The simulation of the interactions between players is used to determine the price-quantity pairs that will lead to equilibrium. The objective of the model is to find an equilibrium that maximizes the profit of each competing supplier while meeting the given demand. The outputs are price-quantity offers for each firm and corresponding profits at equilibrium. A sell offer can represent one generating unit or a fleet of units which cost function is obtained by combining the individual cost functions of the units. This approach is used by firms that prefer to bid into the market using portfolio-based cost functions (Baldick, 2006).
2. MARKET OPERATION
The model assumes that there are M participants in a power pool who are referred to as power suppliers. These power suppliers may either have individual generators or even a portfolio of generators. All these participants bid into the day-ahead market and aim to maximize their profit by using bidding strategies that best represent their expectations. The RTO collects the sell offers simultaneously and it starts the security-constrained economic dispatch algorithm to set the equilibrium for the market. The RTO sorts the sell offers starting from the minimum price offers to more expensive ones and sorts the buy bids starting from the maximum price offers to less expensive ones. The RTO sets the equilibrium market price and power quantity for each hour of next day where the supply and demand meet.
3. MODEL FORMULATION AND ASSUMPTIONS
We are particularly interested in modeling the strategic behavior of the generation side rather than the demand side. In most power markets, LSEs do not bid strategically and are typically price-takers. Therefore, the demand at each hour of the next day is assumed to be known to all suppliers. Although the RTO considers transmission limits when running the security-constrained economic dispatch model, we do not include them in our model. The reason is that each power supplier plans its own power transmission in its zone and the transfer limits between zones are predetermined by the RTO. Consequently, we assume that transmission congestion and power loss costs are already included in the cost function of the firm.
The model is formulated under the assumptions:
1. A sell offer consists of N or less price-quantity blocks.
2. Each supplier determines its offer based on its portfolio of power resources.
3. Sell offers for each day are determined before the market closes at noon and are valid for the next 24 hours, starting at midnight of the same day.
4. Demand can be forecasted for all suppliers before the sell offers are submitted. 5. Transmission constraints are not included in the model.
6. Equilibrium of interest occurs in a single-round auction market.
In addition, a solution (set of sell offers) for the SBP is Nash equilibrium if no supplier can obtain higher profit by changing its offer while the other suppliers keep their offers unchanged and
212 A. YUCEKAYA AND J. VALENZUELA
the supply equals the demand. The objective of the problem is to find the day-ahead settlement levels that will satisfy these equilibrium conditions. In finding an equilibrium, there are N M pairs of decision variables bj i and qj i .j D 1; : : : ; M / .i D 1; : : : ; N / that need to be determined.
The variable qj idenotes the amount of energy increase in block i of firm j , offered at the price
bj i for delivery at any hour.
4. AGENT-BASED PARTICLE SWARM OPTIMIZATION MODEL
Agent-based modeling and simulation (ABMS) is a computational approach to model economic systems which have interacting components or dynamic agents (Macal and North, 2006). Agents usually interact among themselves and between environments by updating themselves sequentially rather than simultaneously. Yucekaya et al. (2009) proposed a PSO algorithm that is called decomposition-based particle swarm optimization (dBPSO) for the bidding of a single price taker unit. In this article the ABMS and dBPSO are combined to obtain the method agent-based particle swarm optimization (ABPSO). In our model, each firm participating in the day-ahead market is modeled as an agent.
Each agent has a cost function, capacity, and pairs of price-quantity bids as attributes. The agents’ interaction occurs in the pool where the offered quantities and corresponding prices are submitted. The model assumes that the final objective is to reach equilibrium for the next day. Each agent’s objective is to maximize its profit by bidding into the market using dBPSO and making sure not to violate the conditions defined in ABMS when they allocate their price-quantity offers. Figure 1 shows the flowchart of the ABPSO. Notice that at a particular iteration the agents behave as price-takers.
The hourly cleared prices used at that iteration are the results of the bidding strategies submitted by each agent in the previous iteration. Agents update their price and quantity bids to define their next strategy based on the previous cleared prices. This iterative process ensures that each generator minimizes its risk of cost recovery and infeasibility of its offer. The stopping criterion determines when the equilibrium is reached. To determine whether the equilibrium has been reached at a particular iteration, the ABPSO looks at the clearing prices computed at the previous iteration. If the clearing prices at these two consecutive iterations are “similar” and all conditions are met, the ABPSO declares equilibrium and stops. Notice that the submission of similar bids at two consecutive iterations may lead to similar cleared prices at these two iterations. This might indicate that the market clearing price is converging and the bids submitted by the agents are reaching equilibrium. The major problem with the stopping criterion is to quantify how “similar” the clearing prices are. We study three stopping rules with the aim to choose one. In the first stopping rule (SP), for each hour we calculate the percentage of difference between the clearing prices of two consecutive iterations. The ABPSO stops when the absolute value of the percentage difference is less than a value "1 for all hours. In the second stopping rule (SA), we calculate
the average of differences of clearing prices over all hours. When the absolute average of the differences is less than a value "2the ABPSO stops. In the third stopping rule (SW), we calculate
the average of the demand weighted price difference of two consecutive iterations over all hours. The process stops when this value is less than a predetermined "3.
Notice that agents select a strategy based on interactions with other agents and the environment. The interaction with the environment occurs in a way that total supply of all agents should be equal to total demand. The amount of power allocated to an agent affects other agents’ strategies. The interaction between agents occurs based on the offer prices and offer quantities. If the offer of an agent is selected, the agent can either maintain this strategy or update it in the next iteration in order to obtain higher profits. This process continues until the price difference in two consecutive iterations is low enough so equilibrium is declared.
FIGURE 1 The flowchart of the ABPSO.
5. NUMERICAL EXAMPLE AND ANALYSIS
The ABPSO model was coded using the programming language C. We have used the same set of parameters for the dBPSO as in Yucekaya et al. (2009).1 The forecasted demand for the next
day is used for the two cases: the duopoly and 5-firm cases.
5.1. Duopoly Case
In this experiment, the market consists of two firms .M D 2/ which have capacities of 8,085 MW and 6,281 MW, respectively. We run the ABPSO under three stopping rules to determine 1A detailed version of the paper that includes the details of the data and solution approach is available at: https://
docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B64KvEJz9AkrNmM1MDUwYjAtYTU2Ni00Nzkx LTk2ZmItODI0YWU3NGRlZWQ1&hl=en&authkey=CKmv-8MH.
214 A. YUCEKAYA AND J. VALENZUELA
TABLE 1 Firm’s Profit for Duopoly
Total Profit, $ Profit Increase, % Stopping
Rule Firm 1 Firm 2 Firm 1 Firm 2 SP 1,302,689 1,043,974 .08 .00 SA 1,302,689 1,043,974 .08 .00 SW 1,355,414 1,094,031 .15 .01
which rule provides better results. We choose the values "1 D 1%, "2 D $0:25, and "3 D 1%. These values were chosen so the equilibrium condition was reached in a reasonable time. The clearing prices under rule SP and SA are equal while the prices are slightly different under rule SW. In terms of computational time, the ABPSO using rules SA and SW required around 15 min to reach equilibrium in an Intel core duo computer with 2 Ghz CPU and 4GB RAM (Dell, Round Rock, TX). The rule SP required 33 min.
The total profits of the firms are given in Table 1. Notice that the profits are equal under the stopping rules SP and SA and slightly higher under SW for both firms. To check the Nash equilibrium, the dBPSO is run separately for each firm to obtain the best bid assuming that the clearing prices are the ones obtained by the ABPSO. The process is repeated for each stopping rule. The resulting profit of each firm is used to compute a percentage profit increase, which is also given in Table 1. It can be observed that the potential for profit increase is minimal (less than 0.2%) for all rules. Therefore, there is no significant incentive for either supplier for changing its offer. This means that either stopping rule provides Nash equilibrium. Nevertheless, rule SA should be preferred because of its lower computational time and less incentive to change the offer.
5.2. Five-firm Case
In this experiment, the market consists of five firms .M D 5/ with 3,450 MW, 1,600 MW, 2,935 MW, 4,950 MW and 2,281 MW capacities, respectively. The coefficients are obtained based on the given heat-input rates in Valenzuela and Mazumdar (2003).
As in the duopoly experiment, the ABPSO is run under the same three stopping rules. The same values for "1, "2, and "3 are used. The clearing prices under the three rules came up slightly
differently. In terms of computational time, the ABPSO using rules SA and SW required around 35 min, for SP it required 67 min to reach equilibrium.
To check the Nash equilibrium, the dBPSO is run separately for each firm to obtain the best bid assuming that the clearing prices are the ones obtained by the ABPSO (Figure 1). This is repeated for each stopping rule. As in the duopoly case, the resulting profit of each firm is used to compute a percentage profit increase, which is given in Table 2. It can be observed that under the rule SP Firm 5 has an economic incentive to change its offer, while under rule SW Firm 1 has an incentive to change its offer. However, under SA the potential for profit increase is insignificant for all five firms (less than 1%). Therefore, we accept that SA provides the Nash equilibrium.
We have aggregated the sell offers of the firms under the three stopping rules and plot them in Figure 2. Although the curves are different within the range 0 to 15,216 MW (system capacity), they are very similar in the range of the demand 3,115 to 6,561 MW.
TABLE 2 Firm’s Profit Increase
Profit Increase (%) Stopping
Rule Firm 1 Firm 2 Firm 3 Firm 4 Firm 5 SP 0.46 0.17 0.51 2.87 13.51 SA 0.14 0.40 0.85 0.72 0.06 SW 4.28 0.11 2.20 2.30 1.54
FIGURE 2 Aggregated offer curve under different stopping rules. (color figure available online)
6. CONCLUSIONS
In this article, we showed that ABPSO can be used to simulate the bidding process in the PJM power market and help find equilibrium. The defined stopping rules and their results confirm the defined equilibrium constraints for duopoly, but the second stopping rule also satisfies the constraints for other cases and hence should be selected. In terms of computational time, relatively not much time is required to reach the results. The time is especially important as the bidding is a daily process and the decisions are made in limited time. The cost data and firm strategies in the market are dynamic. Thus, the data change frequently in the day-ahead market. ABPSO can be designed as an effective tool to run the updated data anytime. The model can further be developed for the real market if anyhow real operational cost and demand data is provided.
REFERENCES
Bagnall, A. J. 2000. A multi-adaptive agent model of generator bidding in the UK market in electricity. Proceedings of the Genetic and Evolutionary Computation Conference, Las Vegas, NV, July 8–12, pp. 605–612.
Baldick, R. 2006. Computing the electricity market equilibrium: Uses of market equilibrium models. Power Systems Conference and Exposition, Atlanta, GA, Oct. 29–Nov. 1, pp. 66–73.
Baldick, R., Grant, R., and Kahn, D. 2004. Theory and application of linear supply function equilibrium in electricity markets. J. Regul. Econ. 25:143–167.
Botterud, A., Thimmapuram, P. R., and Yamakado, M. 2006. Simulating GenCo Bidding Strategies in Electricity Markets With and Agent-based Model. Available at: http://www.dis.anl.gov/publications/articles/ceeesa_EMCAS_IAEE72005 NorwayPaper.pdf.
216 A. YUCEKAYA AND J. VALENZUELA
Bouffard, F., and Galiaba, F. 2006. Power system security and short-term electricity market equilibrium. Power Systems Conference and Exposition, Atlanta, GA, Oct. 29–Nov. 1, pp. 90–95.
David, A. K., and Wen, F. 2000. Strategic bidding in competitive electricity markets: A literature survey. IEEE Power Engineering Society Conference Proceedings, Seattle, WA, July 16–20, pp. 2168–2173.
Klemperer, P., and Meyer, M. 1989. Supply function equilibrium. Econometrica 57:1243–1277.
Macal, C. M., and North, M. J. 2006. Tutorial on agent-based modeling and simulation Part 2: How to model with agents. Proceedings of the 2006 Winter Simulation Conference, Orlando, FL, Dec. 3–6, pp. 2–15.
Nicolaisen, J., Petrov, V., and Tesfatsion, L. 2000. Market power and efficiency in a computational electricity market with discriminatory double-auction pricing. IEEE T. Evolut. Comput. 5:504–523.
PJM Interconnections. 2011. PJM. Available at: http://www.pjm.com.
Rudkevich, A. 2003. Supply function equilibrium: theory and applications. Proceedings of the 36th Hawaii International Conference on System Sciences, Big Island, HI, Jan. 6–9, p. 10.
Sueyoshi, T., and Tadiparthi, G. R. 2005. A wholesale power trading simulator with learning capabilities. IEEE T. Power Syst. 20:1330–1340.
Sueyoshi, T., and Tadiparthi, G. R. 2007. Agent-based approach to handle business complexity in U.S. wholesale power trading. IEEE T. Power Syst. 22:532–543.
Valenzuela, J., and Mazumdar, M. 2003. Commitment of electric power generators under stochastic market prices. Oper. Res. 51:880–893.
Ventosa, M., Baillo, A., Ramos, A., and Rivier, M. 2005. Electricity market modeling trends. Energ. Policy 33:897–913. Yucekaya, A. D., Valenzuela, J., and Dozier, G. 2009. Strategic bidding in electricity markets using particle swarm
