- 142 Views
- Uploaded on
- Presentation posted in: General

Agent Technology for e-Commerce

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Agent Technology for e-Commerce

Appendix A: Introduction to Decision Theory

Maria Fasli

http://cswww.essex.ac.uk/staff/mfasli/ATe-Commerce.htm

Decision theory is the study of making decisions that have a significant impact

Decision-making is distinguished into:

- Decision-making under certainty
- Decision-making under noncertainty
- Decision-making under risk
- Decision-making under uncertainty

- Most decisions have to be taken in the presence of uncertainty
- Probability theory quantifies uncertainty regarding the occurrence of events or states of the world
- Basic elements of probability theory:
- Random variables describe aspects of the world whose state is initially unknown
- Each random variable has a domain of values that it can take on (discrete, boolean, continuous)
- An atomic event is a complete specification of the state of the world, i.e. an assignment of values to variables of which the world is composed

Probability space

- The sample space S={e1,e2,â€¦,en} which is a set of atomic events
- The probability measure P which assigns a real number between 0 and 1 to the members of the sample space
Axioms

- All probabilities are between 0 and 1
- The sum of probabilities for the atomic events of a probability space must sum up to 1
- The certain event S (the sample space itself) has probability 1, and the impossible event which never occurs, probability 0

- In the absence of any other information, a random variable is assigned a degree of belief called unconditional or prior probability
- P(X) denotes the vector consisting of the probabilities of all possible values that a random variable can take
- If more than one variable is considered, then we have joint probability distributions
- Lottery: a probability distribution over a set of outcomes
L=[p1,o1;p2,o2;â€¦;pn,on]

- When we have information concerning previously unknown random variables then we use posterior or conditional probabilities: P(a|b) the probability of a given that we know b
- Alternatively this can be written (the product rule):
P(aïƒ™b)=P(a|b)P(b)

- Independence
P(a|b)=P(a) and P(b|a)=P(b) or P(aïƒ™b)=P(a)P(b)

The product rule can be written as:

P(aïƒ™b)=P(a|b)P(b)

P(aïƒ™b)=P(b|a)P(a)

By equating the right-hand sides:

This is known as Bayesâ€™ rule

Simple example: to take or not my umbrella on my way out

The consequences of decisions can be expressed in terms of payoffs

Payoff table

Loss table

An alternative representation of payoffs â€“ tree diagram

- An action is said to dominate another, if for each possible state of the world the first action leads to at least as high a payoff (or at least as small a loss) as the second one, and there is at least one state of the world in which the first action leads to a higher payoff (or smaller loss) than the second one
- If one action dominates another, then the latter should never be selected and it is called inadmissible

Payoff table

- The maximin rule
- The maximax rule
- The minimax loss

Payoff table

Loss table

- The Expected Payoff (ER) rule dictates that the action with the highest expected payoff should be chosen
- The Expected Loss (EL) rule dictates that the action with the smallest expected loss should be chosen
If P(rain)=0.7 and P(not rain)=0.3 then:

ER(carry umbrella) = 0.7(-Â£1)+0.3(-Â£1)=-Â£1

ER(not carry umbrella) = 0.7(-Â£50)+0.3(-Â£0)=-Â£35

EL(carry umbrella) = 0.7(Â£0)+0.3(Â£1)=Â£0.3

EL(not carry umbrella) = 0.7(Â£49)+0.3(Â£0)=Â£34.3

- Usually the consequences of decisions are expressed in monetary terms
- Additional factors such are reputation, time, etc. are also usually translated into money
- Issue with the use of money to describe the consequences of actions:
- If a fair coin comes up heads you win Â£1, otherwise you loose Â£0.75, would you take this bet?
- If a fair coin comes up heads you win Â£1000, otherwise you loose Â£750, would you take this bet?

- The value of a currency, differs from person to person

- The concept of preference is used to indicate that we would like/desire/prefer one thing over another
- ooâ€™ indicates that o is (strictly) preferred to oâ€™
- o ~ oâ€™ indicates that an agent is indifferent between o and oâ€™
- o oâ€™ indicates that o is (weakly) preferred to oâ€™
- Given any o and oâ€™, then ooâ€™, or oâ€™o, or o ~ oâ€™
- Given any o, oâ€™ and oâ€™â€™, then if ooâ€™ and oâ€™oâ€™â€™, then oâ€™oâ€™â€™
- If ooâ€™oâ€™â€™, then there is a p such that [p,o;1-p,oâ€™â€™] ~ oâ€™
- If o ~ oâ€™, then [p,o; 1-p,oâ€™â€™] ~ [p,oâ€™; 1-p, oâ€™â€™]
- If ooâ€™, then (pï€ ï‚³qï€ ïƒ› [p,o;1-p,oâ€™]ï€ [q,o;1-q,oâ€™] )

- A utility function provides a convenient way of conveying information about preferences
- If ooâ€™, then u(o)>u(oâ€™) and if o ~ oâ€™ then u(o)=u(oâ€™)
- If an agent is indifferent between:
(a) outcome o for certain and

(b) taking a bet or lottery in which it receives oâ€™ with probability p and oâ€™â€™ with probability 1-p

then u(o)=(p)u(oâ€™)+(1-p)u(oâ€™â€™)

- Ordinal utilities
- Cardinal utilities
- Monotonic transformation

How can an agent assess a utility function?

- Suppose most and least preferable payoffs are R+ and R- and
u(R+)=1 and u(R-)=0

- For any other payoff R, it should be:
u(R+) ï‚³ u(R) ï‚³ u(R-) or 1 ï‚³u(R) ï‚³ 0

- To determine the value of u(R) consider:
- L1: Receive R for certain
- L2: Receive R+ with probability p and R- with probability 1-p

- Expected utilities:
- EU(L1)=u(R)
- EU(L2)=(p)u(R+ )+(1-p)u(R- )=(p)(1)+(1-p)(0)=p

- If u(R)>p, L1 should be selected, whereas if u(R)<p, L2 should be selected, and if u(R)=p then the agent is indifferent between the two lotteries

- The value, i.e. utility, of money may differ from person to person
- Consider the lottery
- L1: receive Â£0 for certain
- L2: receive Â£100 with probability p and -Â£100 with (1-p)

- Suppose an agent decides that for p=0.75 is indifferent between the two lotteries, i.e. p>0.75 prefers lottery L2
- The agent also assess u(-Â£50)=0.4 and u(Â£50)=0.9

- If p is fixed, the amount of money that an agent would need to receive for certain in L1 to make it indifferent between two lotteries can be determined. Consider:
- L1: receive Â£x for certain
- L2: receive Â£100 with probability 0.5 and -Â£100 with 0.5

- Suppose x=-Â£30, then u(-Â£30)=0.5 and -Â£30 is considered to be the cash equivalent of the gamble involved in L2
- The amount of Â£30 is called the risk premium â€“ the basis of insurance industry

- The utility of an action may depend on a number of factors
- Multi-dimensional or multi-attribute utility theory deals with expressing such utilities
- Example: you are made a set of job offers, how do you decide?
u(job-offer) = u(salary) + u(location) +

u(pension package) + u(career opportunities)

u(job-offer) = 0.4u(salary) + 0.1u(location) +

0.3u(pension package) + 0.2u(career opportunities)

But if there are interdependencies between attributes, then additive utility functions do not suffice. Multi-linear expressions:

u(x,y)=wxu(x)+wyu(y)+(1-wx-wy)u(x)u(y)