Correlated Equilibria in Game Theory: Existence, Computation, and Succinct Representations, Study notes of Computer Science

Download Correlated Equilibria in Game Theory: Existence, Computation, and Succinct Representations and more Study notes Computer Science in PDF only on Docsity!

Tools of the Trade The Exponential Curse

Existence and Computation of Correlated

Equilibria

Andreas Hoenselaar

CS 244

January 11, 2006

Tools of the Trade The Exponential Curse

Outline

(^1) Introduction Basic definitions The Mission

(^2) Tools of the Trade Linear Programming Markov Chains

(^3) The Exponential Curse Succinct Games

Tools of the Trade The Exponential Curse

Basic definitions The Mission

Definition of correlated equilibria

Definition A distribution x ∈ ∆ is a correlated equilibrium if the following is true for all players p and pairs of strategies (i, j): After drawing a strategy profile from x where player p’s component is i, there is no incentive to play another strategy j and it holds that ∑

s∈S−p

(upis − upjs)xis ≥ 0

Tools of the Trade The Exponential Curse

Basic definitions The Mission

Correlated equilibria vs. Nash equilibria

Nash equilibria are special cases of correlated equilibria.

Nash equilibria Distribution x on S is a product distribution: xs =

∏n p= 1 x

p sp ∀s^ ∈^ S ⇒ x is completely defined by its marginal distributions xp

Correlated Equilibria As long as x is a valid distribution, it can have any form All |S| entries are required for a full description

Tools of the Trade The Exponential Curse

Linear Programming Markov Chains

Primal and dual problem

Primal

min c>x s.t. Ax ≥ b x ≥ 0

Dual

max π>b s.t. A>π ≤ c π ≥ 0

Strong duality For optimal x∗^ and π∗^ it holds that c>x∗^ = π∗>b.

Tools of the Trade The Exponential Curse

Linear Programming Markov Chains

Solutions to the primal and dual problem

Optimization results for linear programs An LP can have an optimal feasible solution x∗ be infeasible (contradicting constraints) be unbounded (ie there are feasible points with arbitrarily large objective value)

Tools of the Trade The Exponential Curse

Linear Programming Markov Chains

The Relationship between (CE) and (P)

(CE)

Find x = (x 1 ,... , x|S|) s.t. Ux ≥ 0 ∑

s∈S

xs = 1

0 ≤ x ≤ 1

(P)

max

s∈S

xs

s.t. Ux ≥ 0 x ≥ 0

Lemma Problem (P) is unbounded iff (CE) has a solution.

Tools of the Trade The Exponential Curse

Linear Programming Markov Chains

(D) is always infeasible

Lemma For every y ≥ 0 there is a product distribution x such that xU>y = 0.

∑

s∈S−p

(upis − upjs)xis ≥ 0

q 6 =p

xqsq

xpi

j∈Sp

ypji −

j∈Sp

xpj ypij

Tools of the Trade The Exponential Curse

Linear Programming Markov Chains

Definition of a Markov Chain (2)

The Markov property For every state sequence (Xt)t= 0 , 1 ,... (Xt^ ∈ Q) it holds that

P(Xt+^1 = qm|X^0 ,... , Xt) = P(Xt+^1 = qm|Xt) ∀t ∈ N^0 , qm ∈ Q.

This implies that the state at any given point in time only depends on the previous state and the transition matrix.

Example

Q = {q 1 , q 2 , q 3 }, T =

 (^) , p^0 =

3

1 3

1 3

Tools of the Trade The Exponential Curse

Linear Programming Markov Chains

Steady-state of a Markov Chain

Motivation What can we say about the probabilty to be in a specific state qm after n steps?

Computation of the steady-state distribution Given an initial state distribution p^0 we can calculate the state distribution after n steps: pn^ = p^0 · (T)n

Surprisingly, this sequence converges towards an equilbirium distribution p∗^ in the limit of infinitely many steps: lim n→∞

p^0 · (T)n

= p∗

Tools of the Trade The Exponential Curse Succinct Games

The need for a succinct representation

Exponential size of distributions A distribution x over the set of strategy profiles S has an exponential number of entries:

|x| = |S| =

∏^ n

i= 1

|Si| ∈ O(sn) where s = max (|S 1 |,... , |Sn|)

A first insight If a correlated equilibrium is to be calculated in polynomial time, the game and thus the set of strategy profiles must have a more succinct representation.

Tools of the Trade The Exponential Curse Succinct Games

Definitions for Succinct Games

Definition A succinct game G = (I, T, U) is defined by a set of inputs I an algorithm T that returns the number of players n the cardinalities of the strategy sets (t 1 ,... , tn) in polynomial time an algorithm U that returns the utility ui(s) for player i under a distribution s = (s 1 ,... , sn) The game is of polynomial type if n and all ti’s are polynomially bounded in their argument.

Tools of the Trade The Exponential Curse Succinct Games

Types of succinct games (1)

Symmetric Games In symmetric games, players cannot be distinguished and have the same set of strategies: The payoff depends on a player’s choice and the distribution of the other players’ choices over the strategy set Requires O(ns) space instead of O(sn) for its description

Tools of the Trade The Exponential Curse Succinct Games

Graphical games

in graphical games, players are identified with nodes in a graph and games are played only along the (undirected) edges. Description length: O(n · sk) for maximum node degree k plus explicit representation of gp, the game with all players in the neighborhood Correlated equilibria can be calculated in polynomial time in graphical games

Polynomial expectation property Calculate expectation by iterating over all stratagy profiles in gp, ignore players not in the neighborhood