1980
Views
549
Downloads
0
Crossref
N/A
WoS
N/A
Scopus
N/A
CSCD
The work gives a review on the distributed Nash equilibrium seeking of noncooperative games in multi-agent networks, which emerges as one of the frontier research topics in the area of systems and control community. Firstly, we give the basic formulation and analysis of noncooperative games with continuous action spaces, and provide the motivation and basic setting for distributed Nash equilibrium seeking. Then we introduce both the gradient-based algorithms and best-response based algorithms for various type of games, including zero-sum games, aggregative games, potential games, monotone games, and multi-cluster games. In addition, we provide some applications of noncooperative games.
The work gives a review on the distributed Nash equilibrium seeking of noncooperative games in multi-agent networks, which emerges as one of the frontier research topics in the area of systems and control community. Firstly, we give the basic formulation and analysis of noncooperative games with continuous action spaces, and provide the motivation and basic setting for distributed Nash equilibrium seeking. Then we introduce both the gradient-based algorithms and best-response based algorithms for various type of games, including zero-sum games, aggregative games, potential games, monotone games, and multi-cluster games. In addition, we provide some applications of noncooperative games.
A. Nedić and J. Liu, Distributed optimization for control, Annu. Rev. Control Robot. Auton. Syst., vol. 1, pp. 77–103, 2018.
T. Yang, X. Yi, J. Wu, Y. Yuan, D. Wu, Z. Meng, Y. Hong, H. Wang, Z. Lin, and K. H. Johansson, A survey of distributed optimization, Annu. Rev. Control, vol. 47, pp. 278–305, 2019.
G. Notarstefano, I. Notarnicola, and A. Camisa, Distributed optimization for smart cyber-physical networks, Found. Trends Syst. Control, vol. 7, no. 3, pp. 253–383, 2019.
R. Xin, S. Pu, A. Nedić, and U. A. Khan, A general framework for decentralized optimization with first-order methods, Proc. IEEE, vol. 108, no. 11, pp. 1869–1889, 2020.
P. Yi and Y. G. Hong, Distributed cooperative optimization and its applications, (in Chinese), Sci. Sinica Math., vol. 46, no. 10, pp. 1547–1564, 2016.
Y. Chen, S. Mei, F. Zhou, S. H. Low, W. Wei, and F. Liu, An energy sharing game with generalized demand bidding: Model and properties, IEEE Trans. Smart Grid, vol. 11, no. 3, pp. 2055–2066, 2020.
W. Wei, L. Wu, J. Wang, and S. Mei, Network equilibrium of coupled transportation and power distribution systems, IEEE Trans. Smart Grid, vol. 9, no. 6, pp. 6764–6779, 2018.
A. Zanardi, G. Zardini, S. Srinivasan, S. Bolognani, A. Censi, F. Dörfler, and E. Frazzoli, Posetal games: Efficiency. existence.and refinement of equilibria in games with prioritized metrics, IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 1292–1299, 2022.
M. H. Manshaei, Q. Zhu, T. Alpcan, T. Bacşar, and J. P. Hubaux, Game theory meets network security and privacy, ACM Comput. Surv., vol. 45, no. 3, p. 25, 2013.
Q. Zhu and T. Basar, Game-theoretic methods for robustness, security, and resilience of cyberphysical control systems: Games-in-games principle for optimal cross-layer resilient control systems, IEEE Control Syst., vol. 35, no. 1, pp. 46–65, 2015.
S. Nardi, F. Mazzitelli, and L. Pallottino, A game theoretic robotic team coordination protocol for intruder herding, IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 4124–4131, 2018.
D. Shishika, J. Paulos, and V. Kumar, Cooperative team strategies for multi-player perimeter-defense games, IEEE Robot. Autom. Lett., vol. 5, no. 2, pp. 2738–2745, 2020.
I. Menache and A. Ozdaglar, Network games: Theory, models, and dynamics, Synth. Lect. Commun. Networks, vol. 4, no. 1, pp. 1–159, 2011.
G. Chalkiadakis, E. Elkind, and M. Wooldridge, Computational aspects of cooperative game theory, Synth. Lect. Artif. Intell. Mach. Learn., vol. 5, no. 6, pp. 1–168, 2011.
F. Fele, J. M. Maestre, and E. F. Camacho, Coalitional control: Cooperative game theory and control, IEEE Control Syst. Mag., vol. 37, no. 1, pp. 53–69, 2017.
A. Nedić and D. Bauso, Dynamic coalitional TU games: Distributed bargaining among players' neighbors, IEEE Trans. Autom. Control, vol. 58, no. 6, pp. 1363–1376, 2013.
I. Alvarez, V. Alexander, and S. Poznyak, Urban traffic control via Stackelber-Nash equilibria, IFAC Proc. Vol., vol. 42, no. 15, pp. 582–587, 2009.
N. B. Mandayam, S. B. Wicker, J. Walrand, T. Basar, J. W. Huang, and D. P. Palomar, Game theory in communication systems [guest editorial], IEEE J. Sel. Areas Commun., vol. 26, no. 7, pp. 1042–1046, 2008.
A. Attar, T. Basar, M. Debbah, H. V. Poor, and Q. Zhao, Guest editorial game theory in wireless communications, IEEE J. Sel. Areas Commun., vol. 30, no. 1, pp. 1–3, 2012.
H. Sandberg, S. Amin, and K. H. Johansson, Cyberphysical security in networked control systems: An introduction to the issue, IEEE Control Syst. Mag., vol. 35, no. 1, pp. 20–23, 2015.
J. F. Nash Jr, Equilibrium points in n-person games, Proc. Natl Acad. Sci. USA, vol. 36, no. 1, pp. 48–49, 1950.
F. Facchinei and C. Kanzow, Generalized Nash equilibrium problems, Ann. Oper. Res., vol. 175, no. 1, pp. 177–211, 2010.
Y. Nesterov, Stable traffic equilibria: Properties and applications, Optim. Eng., vol. 1, no. 1, pp. 29–50, 2000.
A. Kannan, U. V. Shanbhag, and H. M. Kim, Addressing supply-side risk in uncertain power markets: Stochastic Nash models, scalable algorithms and error analysis, Optim. Methods Software, vol. 28, no. 5, pp. 1095–1138, 2013.
J. S. Pang, G. Scutari, D. P. Palomar, and F. Facchinei, Design of cognitive radio systems under temperature-interference constraints: A variational inequality approach, IEEE Trans. Signal Process., vol. 58, no. 6, pp. 3251–3271, 2010.
D. Monderer and L. S. Shapley, Fictitious play property for games with identical interests, J. Econ. Theory, vol. 68, no. 1, pp. 258–265, 1996.
Y. Lou, Y. Hong, L. Xie, G. Shi, and K. H. Johansson, Nash equilibrium computation in subnetwork zero-sum games with switching communications, IEEE Trans. Autom. Control, vol. 61, no. 10, pp. 2920–2935, 2016.
S. Grammatico, F. Parise, M. Colombino, and J. Lygeros, Decentralized convergence to Nash equilibria in constrained deterministic mean field control, IEEE Trans. Autom. Control, vol. 61, no. 11, pp. 3315–3329, 2016.
J. Koshal, A. Nedić, and U. V. Shanbhag, Distributed algorithms for aggregative games on graphs, Oper. Res., vol. 64, no. 3, pp. 680–704, 2016.
F. Salehisadaghiani and L. Pavel, Distributed Nash equilibrium seeking: A gossip-based algorithm, Automatica, vol. 72, pp. 209–216, 2016.
M. Ye and G. Hu, Game design and analysis for price-based demand response: An aggregate game approach, IEEE Trans. Cybern., vol. 47, no. 3, pp. 720–730, 2017.
M. Ye and G. Hu, Distributed Nash equilibrium seeking by a consensus based approach, IEEE Trans. Autom. Control, vol. 62, no. 9, pp. 4811–4818, 2017.
S. Liang, P. Yi, and Y. Hong, Distributed Nash equilibrium seeking for aggregative games with coupled constraints, Automatica, vol. 85, pp. 179–185, 2017.
P. Yi and L. Pavel, Distributed generalized Nash equilibria computation of monotone games via double-layer preconditioned proximal-point algorithms, IEEE Trans. Control Network Syst., vol. 6, no. 1, pp. 299–311, 2019.
K. Lu, G. Jing, and L. Wang, Distributed algorithms for searching generalized Nash equilibrium of noncooperative games, IEEE Trans. Cybern., vol. 49, no. 6, pp. 2362–2371, 2019.
J. Lei and U. V. Shanbhag, Distributed variable sample-size gradient-response and best-response schemes for stochastic Nash equilibrium problems, SIAM J. Optim., vol. 32, no. 2, pp. 573–603, 2022.
W. Liu, W. Gu, J. Wang, W. Yu, and X. Xi, Game theoretic non-cooperative distributed coordination control for multi-microgrids, IEEE Trans. Smart Grid, vol. 9, no. 6, pp. 6986–6997, 2018.
T. Tatarenko, W. Shi, and A. Nedić, Geometric convergence of gradient play algorithms for distributed Nash equilibrium seeking, IEEE Trans. Autom. Control, vol. 66, no. 11, pp. 5342–5353, 2021.
D. Paccagnan, B. Gentile, F. Parise, M. Kamgarpour, and J. Lygeros, Nash and wardrop equilibria in aggregative games with coupling constraints, IEEE Trans. Autom. Control, vol. 64, no. 4, pp. 1373–1388, 2019.
P. Yi and L. Pavel, An operator splitting approach for distributed generalized Nash equilibria computation, Automatica, vol. 102, pp. 111–121, 2019.
F. Salehisadaghiani, W. Shi, and L. Pavel, Distributed Nash equilibrium seeking under partial-decision information via the alternating direction method of multipliers, Automatica, vol. 103, pp. 27–35, 2019.
C. X. Shi and G. H. Yang, Distributed Nash equilibrium computation in aggregative games: An event-triggered algorithm, Inf. Sci., vol. 489, pp. 289–302, 2019.
Z. Deng and S. Liang, Distributed algorithms for aggregative games of multiple heterogeneous Euler-Lagrange systems, Automatica, vol. 99, pp. 246–252, 2019.
T. Basar, Control and game-theoretic tools for communication networks, Appl. Comput. Math., vol. 6, no. 2, pp. 104–125, 2007.
H. Yin, U. V. Shanbhag, and P. G. Mehta, Nash equilibrium problems with scaled congestion costs and shared constraints, IEEE Trans. Autom. Control, vol. 56, no. 7, pp. 1702–1708, 2011.
D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, and T. Graepel, et al., A general reinforcement learning algorithm that masters chess, shogi, and go through self-play, Science, vol. 362, no. 6419, pp. 1140–1144, 2018.
N. Brown and T. Sandholm, Superhuman AI for multiplayer poker, Science, vol. 365, no. 6456, pp. 885–890, 2019.
A. Fischer, M. Herrich, and K. Schönefeld, Generalized Nash equilibrium problems-recent advances and challenges, Pesq. Oper., vol. 34, no. 3, pp. 521–558, 2014.
B. Swenson, R. Murray, and S. Kar, On best-response dynamics in potential games, SIAM J. Control Optim., vol. 56, no. 4, pp. 2734–2767, 2018.
G. Scutari, D. P. Palomar, F. Facchinei, and J. S. Pang, Convex optimization, game theory, and variational inequality theory, IEEE Signal Process. Mag., vol. 27, no. 3, pp. 35–49, 2010.
F. Facchinei, A. Fischer, and V. Piccialli, On generalized Nash games and variational inequalities, Oper. Res. Lett., vol. 35, no. 2, pp. 159–164, 2007.
A. Dreves, F. Facchinei, C. Kanzow, and S. Sagratella, On the solution of the KKT conditions of generalized Nash equilibrium problems, SIAM J. Optim., vol. 21, no. 3, pp. 1082–1108, 2011.
J. S. Pang and M. Fukushima, Quasi-variational inequalities, generalized Nash equilibria, and multi-leader-follower games, Comput, Comput. Manage. Sci., vol. 2, no. 1, pp. 21–56, 2005.
F. Parise and A. Ozdaglar, A variational inequality framework for network games: Existence, uniqueness, convergence and sensitivity analysis, Games Econ. Behav., vol. 114, pp. 47–82, 2019.
G. Gürkan and J. S. Pang, Approximations of Nash equilibria, Math. Program, vol. 223, no. 1−2, pp. 117–253, 2009.
S. Li and T. Başar, Distributed algorithms for the computation of noncooperative equilibria, Automatica, vol. 23, no. 4, pp. 523–533, 1987.
M. Zhu and E. Frazzoli, Distributed robust adaptive equilibrium computation for generalized convex games, Automatica, vol. 63, pp. 82–91, 2016.
G. Belgioioso and S. Grammatico, Semi-decentralized Nash equilibrium seeking in aggregative games with separable coupling constraints and non-differentiable cost functions, IEEE Control Syst. Lett., vol. 1, no. 2, pp. 400–405, 2017.
F. Facchinei, V. Piccialli, and M. Sciandrone, Decomposition algorithms for generalized potential games, Comput. Optim. Appl., vol. 50, no. 2, pp. 237–262, 2011.
B. Gharesifard and J. Cortés, Distributed convergence to Nash equilibria in two-network zero-sum games, Automatica, vol. 49, no. 6, pp. 1683–1692, 2013.
J. Hofbauer and S. Sorin, Best response dynamics for continuous zero-sum games, Discrete Contin. Dyn. Syst. Ser. B, vol. 6, no. 1, pp. 215–224, 2006.
J. Ghaderi and R. Srikant, Opinion dynamics in social networks with stubborn agents: Equilibrium and convergence rate, Automatica, vol. 50, no. 12, pp. 3209–3215, 2014.
M. Y. Huang, P. E. Caines, and R. P. Malhamé, Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized ε-Nash equilibria, IEEE Trans. Autom. Control, vol. 52, no. 9, pp. 1560–1571, 2007.
R. Zhu, J. Zhang, K. You, and T. Basar, Asynchronous networked aggregative games, Automatica, vol. 136, p. 110054, 2022.
M. Ye, G. Hu, L. Xie, and S. Xu, Differentially private distributed Nash equilibrium seeking for aggregative games, IEEE Trans. Autom. Control, vol. 67, no. 5, pp. 2451–2458, 2022.
H. Kebriaei, S. J. Sadati-Savadkoohi, M. Shokri, and S. Grammatico, Multipopulation aggregative games: Equilibrium seeking via mean-field control and consensus, IEEE Trans. Autom. Control, vol. 66, no. 12, pp. 6011–6016, 2021.
Z. Deng and X. Nian, Distributed generalized Nash equilibrium seeking algorithm design for aggregative games over weight-balanced digraphs, IEEE Trans. Neural Networks Learn. Syst., vol. 30, no. 3, pp. 695–706, 2019.
Y. Zhang, S. Liang, X. Wang, and H. Ji, Distributed Nash equilibrium seeking for aggregative games with nonlinear dynamics under external disturbances, IEEE Trans. Cybern., vol. 50, no. 12, pp. 4876–4885, 2020.
D. Monderer and L. S. Shapley, Potential games, Games Econ. Behav., vol. 14, no. 1, pp. 124–143, 1996.
O. Candogan, A. Ozdaglar, and P. A. Parrilo, Near-potential games: Geometry and dynamics, ACM Trans. Econ. Comput., vol. 1, no. 2, p. 11, 2013.
N. Li and J. R. Marden, Designing games for distributed optimization, IEEE J. Sel. Top. Signal Process., vol. 7, no. 2, pp. 230–242, 2013.
F. P. Kelly, A. K. Maulloo, and D. K. H. Tan, Rate control for communication networks: Shadow prices, proportional fairness and stability, J. Oper. Res. Soc., vol. 49, no. 3, pp. 237–252, 1998.
Y. Yang, F. Rubio, G. Scutari, and D. P. Palomar, Multi-portfolio optimization: A potential game approach, IEEE Trans. Signal Process., vol. 61, no. 22, pp. 5590–5602, 2013.
P. Yi, Y. Zhang, and Y. Hong, Potential game design for a class of distributed optimisation problems, J. Control Decis., vol. 1, no. 2, pp. 166–179, 2014.
J. Lei and U. V. Shanbhag, Asynchronous schemes for stochastic and misspecified potential games and nonconvex optimization, Oper. Res., vol. 68, no. 6, pp. 1742–1766, 2020.
M. Ye and G. Hu, Solving potential games with dynamical constraint, IEEE Trans. Cybern., vol. 46, no. 5, pp. 1156–1164, 2016.
J. Zhang, W. Liang, B. Yang, H. Shi, K. Wang, and Q. Wang, A potential game approach for decentralized resource coordination in coexisting IWNs, IEEE Trans. Cognit. Commun. Netw., vol. 8, no. 2, pp. 1118–1130, 2022.
A. Kannan and U. V. Shanbhag, Distributed computation of equilibria in monotone Nash games via iterative regularization techniques, SIAM J. Optim., vol. 22, no. 4, pp. 1177–1205, 2012.
G. Scutari, F. Facchinei, J. S. Pang, and D. P. Palomar, Real and complex monotone communication games, IEEE Trans. Inf. Theory, vol. 60, no. 7, pp. 4197–4231, 2014.
J. Koshal, A. Nedic, and U. V. Shanbhag, Regularized iterative stochastic approximation methods for stochastic variational inequality problems, IEEE Trans. Autom. Control, vol. 58, no. 3, pp. 594–609, 2013.
F. Yousefian, A. Nedić, and U. V. Shanbhag, Self-tuned stochastic approximation schemes for non-Lipschitzian stochastic multi-user optimization and Nash games, IEEE Trans. Autom. Control, vol. 61, no. 7, pp. 1753–1766, 2016.
J. Lei, U. V. Shanbhag, J. S. Pang, and S. Sen, On synchronous, asynchronous, and randomized best-response schemes for stochastic Nash games, Math. Oper. Res., vol. 45, no. 1, pp. 157–190, 2019.
D. Gadjov and L. Pavel, A passivity-based approach to Nash equilibrium seeking over networks, IEEE Trans. Autom. Control, vol. 64, no. 3, pp. 1077–1092, 2019.
S. Huang and P. Yi, Distributed best response dynamics for Nash equilibrium seeking in potential games, Control Theory Technol., vol. 18, no. 3, pp. 324–332, 2020.
Y. Zhu, W. Yu, G. Wen, and G. Chen, Distributed Nash equilibrium seeking in an aggregative game on a directed graph, IEEE Trans. Autom. Control, vol. 66, no. 6, pp. 2746–2753, 2021.
M. Ye, L. Yin, G. Wen, and Y. Zheng, On distributed Nash equilibrium computation: Hybrid games and a novel consensus-tracking perspective, IEEE Trans. Cybern., vol. 51, no. 10, pp. 5021–5031, 2021.
P. Yi and L. Pavel, Asynchronous distributed algorithms for seeking generalized Nash equilibria under full and partial-decision information, IEEE Trans. Cybern., vol. 50, no. 6, pp. 2514–2526, 2020.
M. Bianchi, G. Belgioioso, and S. Grammatico, Fast generalized Nash equilibrium seeking under partial-decision information, Automatica, vol. 136, p. 110080, 2022.
B. Franci and S. Grammatico, Stochastic generalized Nash equilibrium seeking under partial-decision information, Automatica, vol. 137, p. 110101, 2022.
D. Gadjov and L. Pavel, Single-timescale distributed GNE seeking for aggregative games over networks via forward-backward operator splitting, IEEE Trans. Autom. Control, vol. 66, no. 7, pp. 3259–3266, 2021.
M. Bianchi and S. Grammatico, Continuous-time fully distributed generalized Nash equilibrium seeking for multi-integrator agents, Automatica, vol. 129, p. 109660, 2021.
Y. Zhu, W. Yu, W. Ren, G. Wen, and J. Gu, Generalized Nash equilibrium seeking via continuous-time coordination dynamics over digraphs, IEEE Trans. Control Netw. Syst., vol. 8, no. 2, pp. 1023–1033, 2021.
Y. Zou, B. Huang, Z. Meng, and W. Ren, Continuous-time distributed Nash equilibrium seeking algorithms for non-cooperative constrained games, Automatica, vol. 127, p. 109535, 2021.
G. Chen, Y. Ming, Y. Hong, and P. Yi, Distributed algorithm for ε-generalized Nash equilibria with uncertain coupled constraints, Automatica, vol. 123, p. 109313, 2021.
Z. Deng, Distributed Nash equilibrium seeking for aggregative games with second-order nonlinear players, Automatica, vol. 135, p. 109980, 2022.
S. Krilašević and S. Grammatico, Learning generalized Nash equilibria in multi-agent dynamical systems via extremum seeking control, Automatica, vol. 133, p. 109846, 2021.
X. Xu and Q. Zhao, Distributed no-regret learning in multiagent systems: Challenges and recent developments, IEEE Signal Process. Mag., vol. 37, no. 3, pp. 84–91, 2020.
M. Ye, G. Hu, and F. L. Lewis, Nash equilibrium seeking for N-coalition noncooperative games, Automatica, vol. 95, pp. 266–272, 2018.
X. Zeng, J. Chen, S. Liang, and Y. Hong, Generalized Nash equilibrium seeking strategy for distributed nonsmooth multi-cluster game, Automatica, vol. 103, pp. 20–26, 2019.
M. Ye, G. Hu, F. L. Lewis, and L. Xie, A unified strategy for solution seeking in graphical N-coalition noncooperative games, IEEE Trans. Autom. Control, vol. 64, no. 11, pp. 4645–4652, 2019.
Y. Pang and G. Hu, Nash equilibrium seeking in N-coalition games via a gradient-free method, Automatica, vol. 136, p. 110013, 2022.
N. S. Kukushkin, Best response dynamics in finite games with additive aggregation, Games EcoBehav., vol. 48, no. 1, pp. 94–110, 2004.
B. Swenson, S. Kar, and J. Xavier, Empirical centroid fictitious play: An approach for distributed learning in multi-agent games, IEEE Trans. Signal Process., vol. 63, no. 15, pp. 3888–3901, 2015.
B. Swenson, C. Eksin, S. Kar, and A. Ribeiro, Distributed inertial best-response dynamics, IEEE Trans. Autom. Control, vol. 63, no. 12, pp. 4294–4300, 2018.
B. Wang, Y. Wu, and K. J. R. Liu, Game theory for cognitive radio networks: An overview, Comput. Netw., vol. 54, no. 14, pp. 2537–2561, 2010.
L. Duan, J. Huang, and B. Shou, Investment and pricing with spectrum uncertainty: A cognitive operator's perspective, IEEE Trans. Mobile Comput., vol. 10, no. 11, pp. 1590–1604, 2011.
X. Liu, R. Zhu, B. Jalaian, and Y. Sun, Dynamic spectrum access algorithm based on game theory in cognitive radio networks, Mobile Netw. Appl., vol. 20, no. 6, pp. 817–827, 2015.
X. Cao, Y. Chen, and K. J. R. Liu, Cognitive radio networks with heterogeneous users: How to procure and price the spectrum? IEEE Trans. Wireless Commun., vol. 14, no. 3, pp. 1676–1688, 2015.
G. S. Kasbekar and S. Sarkar, Spectrum white space trade in cognitive radio networks, IEEE Trans. Autom. Control, vol. 61, no. 3, pp. 585–600, 2016.
W. Tushar, T. K. Saha, C. Yuen, D. Smith, and H. V. Poor, Peer-to-peer trading in electricity networks: An overview, IEEE Trans. Smart Grid, vol. 11, no. 4, pp. 3185–3200, 2020.
Y. Chen, C. Zhao, S. H. Low, and S. Mei, Approaching prosumer social optimum via energy sharing with proof of convergence, IEEE Trans. Smart Grid, vol. 12, no. 3, pp. 2484–2495, 2021.
Z. Wang, F. Liu, Z. Ma, Y. Chen, M. Jia, W. Wei, and Q. Wu, Distributed generalized Nash equilibrium seeking for energy sharing games in prosumers, IEEE Trans. Power Syst., vol. 36, no. 5, pp. 3973–3986, 2021.
The authors would like to thank Xiaoyu Ma and Wenting Liu for their help in preparing the figures and tables. This work was supperted by Shanghai Sailing Program (Nos. 20YF1453000 and 20YF1452800), the National Science Foundation of China (Nos. 62003239, 62003240, 62003243, and 61903027), Shanghai Municipal Science and Technology Major Project (No. 2021SHZDZX0100), and Shanghai Municipal Commission of Science and Technology (No. 19511132101).
The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).