| Reading |
| CS 420/594: Flake, ch. 18 (Natural & Artificial Computation) | |
| CS 594: Bar-Yam, ch. 2 (Neural Networks I), sections 2.1-2.2 (pp. 295-322) |
| Tit-for-Two-Tats |
| More forgiving than TFT | |
| Wait for two successive defections before punishing | |
| Beats TFT in a noisy environment | |
| E.g., an unintentional defection will lead TFTs into endless cycle of retaliation | |
| May be exploited by feigning accidental defection |
| Effects of Many Kinds of Noise Have Been Studied |
| Misimplementation noise | ||
| Misperception noise | ||
| noisy channels | ||
| Stochastic effects on payoffs | ||
| General conclusions: | ||
| sufficiently little noise Þ generosity is best | ||
| greater noise Þ generosity avoids unnecessary conflict but invites exploitation | ||
| More Characteristics of Successful Strategies |
| Should be a generalist (robust) | ||
| i.e. do sufficiently well in wide variety of environments | ||
| Should do well with its own kind | ||
| since successful strategies will propagate | ||
| Should be cognitively simple | ||
| Should be evolutionary stable strategy | ||
| i.e. resistant to invasion by other strategies | ||
| KantÕs Categorical Imperative |
| ÒAct on maxims that can at the same time have for their object themselves as universal laws of nature.Ó |
| Ecological & Spatial Models |
| Ecological Model |
| What if more successful strategies spread in population at expense of less successful? | |
| Models success of programs as fraction of total population | |
| Fraction of strategy = probability random program obeys this strategy |
| Variables |
| Pi(t) = probability = proportional population of strategy i at time t | ||
| Si(t) = score achieved by strategy i | ||
| Rij(t) = relative score achieved by strategy i playing against strategy j over many rounds | ||
| fixed (not time-varying) for now | ||
| Computing Score of a Strategy |
| Let n = number of strategies in ecosystem | |
| Compute score achieved by strategy i: |
| Updating Proportional Population |
| Some Simulations |
| Usual Axelrod payoff matrix | |
| 200 rounds per step |
| Demonstration Simulation |
| 60% ALL-C | |
| 20% RAND | |
| 10% ALL-D, TFT |
| Collectively Stable Strategy |
| Let w = probability of future interactions | |
| Suppose cooperation based on reciprocity established | |
| Then no one can do better than TFT provided: |
| ÒWin-Stay, Lose-ShiftÓ Strategy |
| Win-stay, lose-shift strategy: | ||
| begin cooperating | ||
| if other cooperates, continue current behavior | ||
| if other defects, switch to opposite behavior | ||
| Called PAV (because suggests Pavlovian learning) | ||
| Simulation without Noise |
| 20% each | |
| no noise |
| Effects of Noise |
| Consider effects of noise or other sources of error in response | ||
| TFT: | ||
| cycle of alternating defections (CD, DC) | ||
| broken only by another error | ||
| PAV: | ||
| eventually self-corrects (CD, DC, DD, CC) | ||
| can exploit ALL-C in noisy environment | ||
| Noise added into computation of Rij(t) | ||
| Simulation with Noise |
| 20% each | |
| 0.5% noise |
| Spatial Effects |
| Previous simulation assumes that each agent is equally likely to interact with each other | ||
| So strategy interactions are proportional to fractions in population | ||
| More realistically, interactions with ÒneighborsÓ are more likely | ||
| ÒNeighborÓ can be defined in many ways | ||
| Neighbors are more likely to use the same strategy | ||
| Spatial Simulation |
| Toroidal grid | |
| Agent interacts only with eight neighbors | |
| Agent adopts strategy of most successful neighbor | |
| Ties favor current strategy |
| Typical Simulation (t = 1) |
| Typical Simulation (t = 5) |
| Typical Simulation (t = 10) |
| Typical Simulation (t =
10) Zooming In |
| Typical Simulation (t = 20) |
| Typical Simulation (t = 50) |
| Typical Simulation (t =
50) Zoom In |
| Simulation of Spatial Iterated Prisoners Dilemma |
| Get sipd simulator |
| SIPD Without Noise |
| Conclusions: Spatial IPD |
| Small clusters of cooperators can exist in hostile environment | |
| Parasitic agents can exist only in limited numbers | |
| Stability of cooperation depends on expectation of future interaction | |
| Adaptive cooperation/defection beats unilateral cooperation or defection |
| Additional Bibliography |
| von Neumann, J., & Morgenstern, O. Theory of Games and Economic Behavior, Princeton, 1944. | |
| Morgenstern, O. ÒGame Theory,Ó in Dictionary of the History of Ideas, Charles Scribners, 1973, vol. 2, pp. 263-75. | |
| Axelrod, R. The Evolution of Cooperation. Basic Books, 1984. | |
| Axelrod, R., & Dion, D. ÒThe Further Evolution of Cooperation,Ó Science 242 (1988): 1385-90. |