Bull-Paper Review1. Holland (1986): “ Classifier systems … rule-based systems with general mechanisms to process rules in parallel, for the adaptive generation of rules, and for testing the effectiveness of existing rules.”
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Holland (1986): “Classifier systems … rule-based systems with general mechanisms to process rules in parallel, for the adaptive generation of rules, and for testing the effectiveness of existing rules.”
LCS= Reinforcement Learning Evolutionary Computing heuristics to produce adaptive systems
Bull on EC: “the population of candidate solutions is seen to adapt to the problem”
Reinforcement learning attempts to map state action combinations to their utility, with the aim to maximize future rewards.
EC is used to search the space of possible rules, while reinforcement learning techniques are used to assign utilities to existing rules, thereby guiding the search for better rules.
Holland’s LCS: bids of successfully rules are placed in a bucket and reinforcement learning redistributes these bids between subsequent chosen rules. As an example how this redistribution is done for ZCS read pages 20-21 of the textbook.
For understanding ZCS read the textbook and not Bull’s paper who fails to explain its mechanisms clearly.
Bull on the difference between ZCS and XCS: The most significant difference between XCS and other systems it its intention to form an accurate mapping of the problem space.
ZCS/Holland employ TD(0) whereas XCS employs Q-learning to be explained next week when
The second paragraph of the textbook on page 124 assumes a more complex model of XCS in which a state space will be explored to be explained next week; as of now, just assume that the payoff of an action is the reward it receives: P=R