Using artificial markets to teach computer science through trading robots
Download
1 / 59

Using Artificial Markets to Teach Computer Science Through Trading Robots - PowerPoint PPT Presentation


  • 106 Views
  • Uploaded on

Using Artificial Markets to Teach Computer Science Through Trading Robots. How to get students interested in algorithms, combinatorial optimization and software development. Karl Lieberherr, Northeastern University, Boston. Outline. Specker Derivative Game (SDG) history example, bottom up

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using Artificial Markets to Teach Computer Science Through Trading Robots' - shira


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Using artificial markets to teach computer science through trading robots

Using Artificial Markets to Teach Computer Science Through Trading Robots

How to get students interested in

algorithms, combinatorial optimization and software development

Karl Lieberherr, Northeastern University, Boston

SDG Mittagsseminar


Outline
Outline Trading Robots

  • Specker Derivative Game (SDG)

    • history

    • example, bottom up

    • top-down

      • derivatives, raw materials, finished products

  • Risk analysis for a derivative

  • Problem reductions – noise elimination

  • SDG(MAX-SAT): risk analysis using polynomials

  • Conclusions

SDG Mittagsseminar


History
History Trading Robots

  • Around 1975: working on non-chronological backtracking for MAX-SAT for my PhD with Erwin Engeler.

  • Ernst Specker analyzed MAX-SAT which lead to the Golden Ratio Result: joint FOCS 79 and JACM 1981 paper. Ideas applicable to MAX-CSP.

  • 2006: sabbatical at Novartis reactivated my interest in MAX-SAT.

  • 2007: Turned Golden Ratio Result into a game SDG(Max) parameterized by a maximization problem Max.

  • 2007/2008: Taught SDG to students who had a lot of fun trying to produce a winning robot (class size 30).

SDG Mittagsseminar


Sdg example
SDG Example Trading Robots

  • derivative(CNF{(2,0),(1,1)}, 0.70)

  • 4 variables maximum

  • would you like to buy it?

  • you will get two rights

    • you will receive a CNF R of the given type.

    • if you can satisfy fraction q of clauses in R, I will pay back q to you.

CNF = SEQUENCE of clauses

SDG Mittagsseminar


Cnf 2 0 1 1 raw materials
CNF{(2,0),(1,1)} raw materials Trading Robots

2: a

2: b

2: c

1: !a !b

1: !b !c

2: a

2: b

3: c

1: d

3: !a !b

9: !a !c

7: !b !c

1: !a !d

6: !b !d

6: !c !d

2: a

3: b

1: c

8: !a !b

6: !b !c

Find best

satisfaction

ratio

1: a

1: b

8: !a !b

SDG Mittagsseminar


Cnf 2 0 1 1 raw materials finished products
CNF{(2,0),(1,1)} raw materials, Trading Robots finished products

2: a

2: b

2: c

1: !a !b

1: !b !c

2: a

2: b

3: c

1: d

3: !a !b

9: !a !c

7: !b !c

1: !a !d

6: !b !d

6: !c !d

6/8=0.75

2: a

3: b

1: c

8: !a !b

6: !b !c

17/20=0.85

price of 0.7

seems fair!?

1: a

1: b

8: !a !b

9/10=0.9

35/40 = 0.875

SDG Mittagsseminar


Using artificial markets to teach computer science through trading robots
Oops Trading Robots

  • Our analysis was not thorough enough!

  • 2 kinds of uncertainty:

    • worst formula?

    • best assignment?

SDG Mittagsseminar


Playing with the weights
Playing with the weights Trading Robots

x: a

x: b

x: c

x: d

y: !a !b

y: !a !c

y: !b !c

y: !a !d

y: !b !d

y: !c !d

x =1, y=1

best assignment

a=1, b=0, c=0, d=0: (1+6)/10=7/10=0.7

x = 2, y=1

best assignment

a=1, b=1, c=0, d=0: (4+5)/14=9/14=0.64

derivative(CNF{(2,0),(1,1)}, 0.70)

LOSS: 0.06

SDG Mittagsseminar


The sdg game two high level views
The SDG Game: Trading Robots Two high level views

  • Financial: Implement trading robots that survive in an artificial derivative market through offering derivatives, and buying and processing derivatives produced by other trading robots.

  • Biological: Implement organisms that survive in an artificial world through offering outsourced services, and consuming and processing outsourced services produced by other organisms.

SDG Mittagsseminar


Derivative pred p s bought by b
Derivative: (pred, p, s) bought by b Trading Robots

  • Max is an NP-hard combinatorial maximization problem with objective function range [0,1].

  • Buyer b buys derivative at price p.

  • Seller s delivers raw material R (instance of Max) satisfying predicate pred.

  • Raw material R is finished by buyer with outcome O of quality q and seller pays q to buyer.

  • Buyer only buys if she thinks q > p.

  • Uncertainty for buyer: which raw material R will I get? Only know the predicate! What is the quality of the solution of Max I can achieve for R?

SDG Mittagsseminar


Using artificial markets to teach computer science through trading robots

Buyer b Trading Robots

Seller s

(pi,p,s)

SDG Mittagsseminar


Using artificial markets to teach computer science through trading robots

Buyer makes profit of 0.8 - 0.7 = 0.1 Trading Robots

0.8

(p2,0.9,s)

Buyer b

Seller s

O

0.8

O

0.8

R

R

(p1,0.7,s)

sold

Derivatives

R satisfies pi

0.7

SDG Mittagsseminar


Artificial markets
Artificial markets Trading Robots

  • Trading Robots that survive in a virtual world of an artificial market of financial derivatives.

    • Trading Robots that don’t follow the world rules don’t survive.

    • Trading Robots are ranked based on their bank account.

    • Teaches students about problem solving, software development, analyzing and approximating combinatorial maximization problems, game design and financial derivatives.

SDG Mittagsseminar


Survive in an artificial market
Survive in an artificial market Trading Robots

  • Each robot contains a:

    • Derivative buying agent

    • Derivative offering agent

    • Raw material production agent

    • Finished product agent (solves Max)

  • Winning in robot competitions strongly influences the final grade.

  • Game is interesting even if robots are far from perfect.

  • Focus today: how to play the game perfectly (never losing)

SDG Mittagsseminar


To play well solve min max
To play well: solve min max Trading Robots

maximum

solutions

instances

selected

by predicate

(an infinite set)

0.91

0.62

0.8

0.618

0.619

minimum

Analysis for one Derivative

SDG Mittagsseminar


To play well solve min max1
To play well: solve min max Trading Robots

maximum

solutions

instances

selected

by predicate

(an infinite set)

0.91

0.62

0.8

0.618

0.62

0.619

minimum

small subset of

raw materials

guaranteed to contain

minimum of maxima

Analysis for one Derivative

Noise

SDG Mittagsseminar


Raw material selected
Raw material selected Trading Robots

all possible finished products

0.7

0.7

0.7

small subset of finished products

guaranteed to contain maximum

Noise

SDG Mittagsseminar


Risk analysis
Risk analysis Trading Robots

  • Life cycle of a derivative (pred,p)

    • offer

      • risk high if I can find rm and fp with q(fp) > p

    • buy

      • risk high if I can find rm and fp with q(fp) < p

    • raw material (rm)

    • finished product (fp ,quality q(fp))

  • Two uncertainties

    • raw material is not the worst (uncertainty_rm)

    • finished product is not the best (uncertainty_fp)

SDG Mittagsseminar


To play sdg perfectly eliminate risk
To play SDG perfectly Trading Robots eliminate risk

  • buy

    • break-even price

  • sell

    • break-even price

  • produce

    • efficiently find worst case example

  • process

    • efficiently achieve break-even quality

SDG Mittagsseminar


Goal never lose with offer buy
Goal: never lose with offer/buy Trading Robots

  • Choose algorithms RM and FP

SDG Mittagsseminar


Concerns for sdg max
Concerns for SDG(Max) Trading Robots

  • Robot communication

    • exchange language for derivatives, raw material, finished product

    • centralized game with administrator managing store of derivatives

    • decentralized game

  • Winning in the market

    • clever algorithm design

SDG Mittagsseminar


Analysis of sdg max
Analysis of SDG(Max) Trading Robots

tpred =

inf

all raw materials rm

satisfying predicate pred

max

all finished products fp

produced for rm

q(fp)

SDG Mittagsseminar


Analysis of sdg max1
Analysis of SDG(Max) Trading Robots

tpred = lim

n -> ∞

min

all raw materials rm of size n

satisfying predicate pred

max

all finished products fp

produced for rm

q(fp)

SDG Mittagsseminar


Spec for rm and fp
Spec for RM and FP Trading Robots

tpred = lim

n -> ∞

min

all raw materials rm of size n

satisfying predicate pred and

having property WORST(rm)

max

small subset of all finished products fp

produced for rm

q(fp)

SDG Mittagsseminar


Analysis of sdg max2
Analysis of SDG(Max) Trading Robots

tpred = lim

n -> ∞

min

all raw materials rm of size n produced by RM

satisfying predicate pred

max

all finished products fp

produced for rm by FP

q(fp)

SDG Mittagsseminar


Using artificial markets to teach computer science through trading robots
Hope Trading Robots

  • Max is NP-hard

  • SDG(Max) simplifies Max if our goal is to never lose.

SDG Mittagsseminar


Sdg max sat
SDG(MAX-SAT) Trading Robots

  • Predicates using clause types.

  • Example predicate PairSat = All CNFs with clauses of any length but clauses of length 1 must contain one positive literal.

  • What is the right price p for derivative (PairSat, p, Specker)

SDG Mittagsseminar


Sdg max sat1
SDG(MAX-SAT) Trading Robots

  • Predicate space: any subset of clause types of PairSat

  • t all PairSat = (√5 -1)/2

  • t {(2,0),(1,1)} = (√5 -1)/2

  • t {(100,50), (3,2), (2,0),(1,1)} = (√5 -1)/2

Noise for the purpose of constructing raw material.

SDG Mittagsseminar


Sdg max sat2
SDG(MAX-SAT) Trading Robots

  • t {(2,0),(1,1)} = t SYM{(2,0),(1,1)} = (√5 -1)/2

  • SYM stands for Symmetrization: Idea: if you give me a CNF with a satisfaction ratio f, I give you a symmetric CNF with a satisfaction ratio <= f.

  • For a CNF in SYM{(2,0),(1,1)}, the MAX-SAT problem reduces to maximizing a polynomial.

SDG Mittagsseminar


Students implement trading robots
Students implement trading robots Trading Robots

  • Fall semester (undergraduates): SDG(MAX-SAT)

  • Spring semester (graduates): SDG(MAX-CSP)

    • Predicate space: Any subset of Boolean relations of rank 3

      • CSP({22},4/9,robot1)

      • CSP({17,22},1/2,robot2)

SDG Mittagsseminar


Opportunities for learning example sdg max sat
Opportunities for learning Trading Robots Example SDG(MAX-SAT)

  • Abstraction: What is important to play the game well.

    • Game reductions: To play game SDG(MAX-SAT) well, it is sufficient to play game SDG(X) well, where X is simpler than MAX-SAT.

SDG Mittagsseminar


Complexity theory connection
Complexity theory connection Trading Robots

  • Break-even prices are not only interesting for the SDG game.

  • They also have complexity-theoretic significance: they are critical transition points separating P from NP (for “most” predicates).

SDG Mittagsseminar


General dichotomy theorem
General Dichotomy Theorem Trading Robots

MAX-CSP(G,f): For each finite set G of relations

there exists an algebraic number tG

For f ≤ tG: MAX-CSP(G,f) has polynomial solution

For f≥ tG+ e: MAX-CSP(G,f) is NP-complete, e>0.

1

hard (solid)

NP-complete

polynomial solution:

Use optimally biased coin.

Derandomize.

P-Optimal.

tG = critical transition point

easy (fluid)

Polynomial

0

due to Lieberherr/Specker (1979, 1982)

SDG Mittagsseminar


Other break even prices lieberherr specker 1982
Other break-even prices Trading Robots (Lieberherr/Specker (1982))

  • G = {R0,R1,R2,R3}; Rj : rank 3, exactly j of 3 variables are true. tG= ¼

SDG Mittagsseminar


Other break even prices lieberherr specker 19821
Other break-even prices Trading Robots (Lieberherr/Specker (1982))

  • G(p,q) = {Rp,q = disjunctions containing at least p positive or q negative literals (p,q≥1)}

    • Let a be the solution of (1-x)p=xq in (0,1). tG(p,q)=1-aq

SDG Mittagsseminar


Lessons learned from sdg
Lessons learned from SDG Trading Robots

  • Developing trading robots and make them survive in an artificial market is very motivating to students

  • Students learn experientially about many important topics driven by the single goal of making their robots competitive

    • software development

    • problem solving by reduction (noise reduction)

    • combinatorial optimization

    • game design

    • sub-optimal playing is very educational too!

SDG Mittagsseminar


Noise reduction important topic
Noise reduction: important topic Trading Robots

  • seen in solving minimization and maximization problems

  • To implement trading robots, we use a tool called DemeterF which is good at noise reduction during programming process: focus on important classes and eliminate noise classes

SDG Mittagsseminar


Conclusions
Conclusions Trading Robots

  • SDG(Max) is an interesting tool for teaching a wide variety of topics.

  • It helps if you give your students a robot that knows the basic rules. Then the students can focus on improving the robots rather than getting all robots to communicate properly.

SDG Mittagsseminar


Conclusions1
Conclusions Trading Robots

  • SDG(Max) is an interesting tool for research.

  • Does it always turn an NP-hard maximization problem into a polynomial time approximation algorithm?

SDG Mittagsseminar


References
References Trading Robots

  • Lieberherr/Specker (1979, 1981) FOCS and Journal of the ACM

  • Lieberherr (1982) Journal of Algorithms

  • Recent Workshop paper:

    • Christine D. Hang and Ahmed Abdelmeged and Daniel Rinehart and Karl J. Lieberherr, The Promise of Polynomial-based Local Search to Boost Boolean MAX-CSP Solvers, 2007, Proceedings of Fourth International Workshop on Local Search Techniques in Constraint Satisfaction, CP2007, Providence, Rhode Island.

  • DemeterF home page: http://www.ccs.neu.edu/research/demeter/DemeterF/

  • SDG home page: http://www.ccs.neu.edu/home/lieber/evergreen/specker/sdg-home.html

SDG Mittagsseminar


Obstacles to finding p
Obstacles to finding p Trading Robots

  • Try to find a CNF satisfying PairSat in which only a small percentage of the clauses can be satisfied.

    • Challenge of finding the worst case.

    • Even if we find the worst case, we might not find the maximum assignment for that case.

SDG Mittagsseminar


Cnf 2 0 1 1 raw materials finished products1
CNF{(2,0),(1,1)} raw materials, Trading Robots finished products

2: a

2: b

2: c

1: !a !b

1: !b !c

2: a

2: b

3: c

1: d

3: !a !b

9: !a !c

7: !b !c

1: !a !d

6: !b !d

6: !c !d

6/8=0.75

2: a

3: b

1: c

8: !a !b

6: !b !c

17/20=0.85

price of 0.7

seems fair!?

1: a

1: b

8: !a !b

9/10=0.9

35/40 = 0.875

SDG Mittagsseminar


Cnf 2 0 1 1 raw materials finished products2
CNF{(2,0),(1,1)} raw materials, Trading Robots finished products

s

2: a

2: b

3: c

1: d

3: !a !b

9: !a !c

7: !b !c

1: !a !d

6: !b !d

6: !c !d

35/40 = 0.875

SDG Mittagsseminar


Cnf 2 0 1 1 reduction
CNF{(2,0),(1,1)} reduction Trading Robots

T(s)

no new variables

2: a !b !c

2: !a b !d

3: a c

1: c d

3: !a !b !c

9: !a !c !d

7: !b !c

1: !a !b !d

6: !b !d

6: !c !d

To each interpretation I1 of

T(s) corresponds an interpretation

I of s which satisfies at least

as many clauses in s as I1 in T(s).

35/40 = 0.875

SDG Mittagsseminar


Cnf 2 0 1 1 reduction1
CNF{(2,0),(1,1)} reduction Trading Robots

no new variables

2: a !b !c

2: !a b !d

3: a c

1: c d

3: !a !b !c

9: !a !c !d

7: !b !c

1: !a !b !d

6: !b !d

6: !c !d

To each interpretation I1 of

T(s) corresponds an interpretation

I of s which satisfies at least

as many clauses in s as I1 in T(s).

35/40 = 0.875

SDG Mittagsseminar


Cnf 2 0 1 1 reduction2
CNF{(2,0),(1,1)} reduction Trading Robots

with new

variables!

2: a !e

2: b !f g

3: c e h i

1: d !e !g

3: !a !b !h !i

9: !a !c !f !g

7: !b !c !g

1: !a !d !f !i

6: !b !d !g !h

6: !c !d

35/40 = 0.875

SDG Mittagsseminar


Sdg max
SDG(Max) Trading Robots

  • Derivative = (Predicate, Price in [0,1], Player).

  • Players offer and buy derivatives.

  • Buying a derivative gives you the rights:

    • to receive raw material R satisfying the predicate.

    • upon finishing the raw material R at quality q (trying to find the maximum solution), you receive q in [0,1].

SDG Mittagsseminar


After talk
After talk Trading Robots

  • Discussion with Emo

  • break-even price as Nash equilibrium

  • 2 person zero sum game

    • distribution on clauses

    • distribution on assignments

SDG Mittagsseminar


Sdg max1
SDG(Max) Trading Robots

  • not percentage of all but percentage of maximum.

  • approximation: seller knows a good solution for raw material. If buyer achieves

SDG Mittagsseminar


Using artificial markets to teach computer science through trading robots

  • derivative = (predicate,p,robot) Trading Robots

  • p = price = estimate how close I can get to the maximum

  • problem = knowing exact maximum is hard

  • approximation: p = price = estimate how close I can get to the outcome the seller knows for an instance satisfying the predicate.

SDG Mittagsseminar


Using artificial markets to teach computer science through trading robots

SDG Mittagsseminar


Using artificial markets to teach computer science through trading robots

  • still flavor of derivatives Trading Robots

  • (pred,p,r) p is estimate how well I can solve an instance satisfying pred compared to how well r can solve it. r has more time available. Here don’t need assumption that objective function in [0,1].

  • now: p is an estimate how well I can solve an instance satisfying pred compared to 1.

SDG Mittagsseminar


Cryptography
cryptography Trading Robots

  • Emo’s satisfiability class: 4 algorithms for 2-Sat. For my algorithms class?

  • r knows good solution. If I would know this good solution, could break system.

  • any approximation would not be useful. Price would have to be 1

SDG Mittagsseminar


Using artificial markets to teach computer science through trading robots

  • SDG(CNF-ALL) Trading Robots

  • SDG(CNF-MAX) uninteresting: eval is exponential

  • SDG(CNF-SECRET)

    • owner delivers with rm a number f.

    • seller, after receiving fp, delivers outcome satisfying f

    • if buyer achieves f * price or higher, buyer receives 1, otherwise 0. all-or-nothing. Can buyer discover the secret that was put into instance by seller?

    • problem: price of 1 not interesting. Discount? Buyer only pays 0.9 * f.

SDG Mittagsseminar


Discount idea
discount idea Trading Robots

  • works also with SDG(CNF-ALL)

  • price is p, but pay only 0.9*p. More realistic. Pay only 0.9, 0.1 is for effort.

  • Use all or nothing. discount = d

SDG Mittagsseminar


Using artificial markets to teach computer science through trading robots
ABB Trading Robots

  • Trading Robot development for artificial markets (to illustrate good separation of concerns from design to implementation using a traversal-based, functional implicit invocation architecture)

    • why trading robot: industrial IT

    • why good separation of concerns: always helps.

    • why traversal-based, functional implicit invocation architecture: built-in variability (class graph generic), support for parallelism: exploiting multi-core architecture

  • Example: Shannon decomposition, testing history of generic SDG game.

SDG Mittagsseminar


Derivative csp pred approx all max secret robot price 1 d approx
derivative[CSP] = (pred, approx (all, max, secret), robot) Trading Robots price = (1-d)*approx

  • global discount d: pay only (1-d)*approx.

  • if achieve approx of (all, max, secret): receive 1 otherwise 0.

  • secret requires extra protocol step.

SDG Mittagsseminar


Derivative csp pred approx all secret robot price 1 d approx
derivative[CSP] = (pred, approx (all, secret), robot) Trading Robots price = (1-d)*approx

  • global discount d: pay only (1-d)*approx.

  • if achieve approx of (all, secret): receive 1 otherwise 0.

  • secret requires extra protocol step.

  • (delete max, because evaluation is costly. Game becomes less interesting.)

  • good for variant of SDG game

SDG Mittagsseminar


Derivative booleancsp pred approx robot price 1 d approx
derivative[BooleanCSP] = (pred, approx, robot) Trading Robots price = (1-d)*approx

  • global discount d: pay only (1-d)*approx.

  • d is compensation for effort to produce finished product

  • if achieve approx of all or more: receive 1 otherwise 0.

  • good for talk: quick intro.

  • Use BooleanCSP because it is clear what all means.

SDG Mittagsseminar