1 / 52

A Core Course on Modeling

A Core Course on Modeling. Week 7-A working model – and then?.      Contents     . The Need for Interpretation Approach Criteria for Modeling Genericity Scalability Specialization Audience Convincingness Distinctiveness Surprise Impact Criteria for modeling and purposes

bruis
Download Presentation

A Core Course on Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Core Course on Modeling Week 7-A working model – and then?    Contents     • The Need for Interpretation • Approach • Criteria for Modeling • Genericity • Scalability • Specialization • Audience • Convincingness • Distinctiveness • Surprise • Impact • Criteria for modeling and purposes • Summary • References to lecture notes + book • References to quiz-questions and homework assignments (lecture notes)

  2. define conceptualize initial problem  conceptual model formalize conceptual model  formal model execute formal model  result conclude result  resolve initial problem? A Core Course on Modeling Week 7-A working model – and then?    The Need for Interpretation     2 context  initial problem

  3. A Core Course on Modeling Week 7-A working model – and then?    The Need for Interpretation     3 A model outcome in itself cannot solve a problem. Interpretation is necessary final step. How to achieve a good-quality interpretation? Model makers in 19th century London predicted 3 meter rising of horse manure levels by 1950 Model makers in 21st century predict 0.5 meter rising of sea levels by 2050

  4. A Core Course on Modeling Week 7-A working model – and then?    Approach     4 We want to say that one model (including the interpretation of the outcomes) is better than another. ‘Better’ means: with respect to fulfilling the model’s purpose. Therefore we need criteria to compare. The first step to quality assessment is the ability to compare things that are utterly different. Comparison invariably needs criteria.

  5. A Core Course on Modeling Week 7-A working model – and then?    Approach     5 So: we will try to formulate criteria for comparing problem solutions. Problem solution = model outcome + interpretation As yet, we have no means to compare ‘quality’ of interpretations and assumptions can be assessed (compared) with the tools of Chapter 6 (sensitivity analysis, uncertainty analysis)

  6. A Core Course on Modeling Week 7-A working model – and then?    Criteria for Modeling     6 When formulating criteria for comparing problem solutions, the main problem is the immense variety of problems asking for modeling. Our criteria need to apply to all of them. The large variety of possible modeling processes is a complication for finding a single generic set of criteria.

  7. A Core Course on Modeling Week 7-A working model – and then?    Criteria for Modeling     7 The standard approach to finding a complete set of ‘things’, is to construct a complete taxonomy. A complete taxonomy requires enumerable properties: properties where all possible values can be enumerated. The quatro-game is based on 4 enumerable properties: color(black,white), size(tall,short), shape(round,square), interior(hollow,solid). Rows of 4 must be formed of pieces which all have a common value for one property

  8. A Core Course on Modeling Week 7-A working model – and then?    Criteria for Modeling     8 Property 1: Does the criterion regard the begin (definition stage of the modeling process) or the end (conclusion stage)? • Example of criterion related to definition: what is the scale of the problem? (scalability) • Example of criterion related to conclusion: how much depends on it? (impact) The input-side of a problem relates to stage 1 (problem definition); the output-side relates to stage 5 (conclusion: presentation and interpretation). This photo seems to suggest that input and output are relative notions …

  9. A Core Course on Modeling the modeled system: part of the world, including the model and some stakeholders the model: the present painting Week 7-A working model – and then?    Criteria for Modeling     9 stakeholder2: the modeler Property 2: Does the criterion regard the ‘inside’ (model, modeled system) or the ‘outside’ (stakeholders + context) ? stakeholder1: the customer • Example of criterion related to the inside (modeled system): what is the scale of the problem? (scalability) • Example of criterion related to the outside (stakeholders): how much knowledge do the stakeholders need? (specialization) In this famous Velasquez painting, the stakeholders and the modeled system are deliberately confused. In modeling practice, the distinction between subject (the modeled system) and object (the stakeholder(s)) are clearly distinct. Hopefully.

  10. A Core Course on Modeling Week 7-A working model – and then?    Criteria for Modeling     10 Property 3: Does the criterion regard qualitative or quantitative aspects ? • Example of criterion related to quantitative aspect: what is the scale of the problem? (scalability) • Example of criterion related to qualitative aspect: how different can modeled systems be? (genericity) The distinction quantitative – qualitative is sometimes associated to the closed – open distinction. ‘Closed’ means that all possible outcomes can be foreseen or enumerated; ‘open’ means that this is not possible.

  11. define inside qualitative define inside quantitative define outside qualitative define outside quantitative conclude inside qualitative conclude inside quantitative conclude outside qualitative conclude outside quantitative A Core Course on Modeling Week 7-A working model – and then?    Criteria for Modeling     11 Combinations scalability define define/ conclude inside / outside qualitative / quantitative criterion genericity audience genericity specialization scalability distinctiveness conclude speciali-zation convincingness impact quantitative audience inside surprise convin-cingness qualitative outside distinctive-ness The 3 distinctions together form an imaginary cube, each corner corresponds to one of the criteria. surprise impact

  12. A Core Course on Modeling Week 7-A working model – and then?    Genericity (define,inside,qualitative)     12 Genericity: to which extent is the approach capable to handle various types of modeled systems / purposes? Examples: • Dynamical systems: buffers & flows • Process modeling: state charts • Data modeling: statistics • Modeling from scratch: 4 categories

  13. A Core Course on Modeling Week 7-A working model – and then?    Genericity (define,inside,qualitative)     13 Genericity: example Given an object of some shape and density ; we want to know its mass. • If it is a cube with side p: m=a3 • if it is a rectangular block with height h: m=h*Areatop • if it is a truncated pyramid: m=  h*(Areatop + (Areatop*Areabottom) + Areabottom)/3 (works for all these shapes!) Above models are increasingly generic; each can deal with the simpler cases as well. How heavy is the Borobodur? (E.g., in the context of risks for landslides etc.: the Borobodur is one of the heaviest temples in the world, restauration in the 1980-ies required partial reinforcement of the hill on which it stands to prevent landslides). Only formula (‘model’) 3 is sufficiently generic

  14. A Core Course on Modeling Week 7-A working model – and then?    Scalability (define,inside,quantitative)     14 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) Performance: O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2n), O(n!), … – SE worse than polynomial Many problems contain some characteristic dimension (e.g., n=number of boxes). If n grows, there will come a point where the model fails when trying to solving the problem. The larger n can be, the better the scalability of the solution.

  15. A Core Course on Modeling Week 7-A working model – and then?    Scalability (define,inside,quantitative)     15 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) Performance: O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2n), O(n!), … – SE worse than polynomial O(1) n The size of Google’s data repository, according to http://www. worldwide websize.com/ , varied between 15 and 55 billion webpages in less than 4 months. Still, the response time stays near constant due to sophisticated indexing strategies

  16. A Core Course on Modeling Week 7-A working model – and then?    Scalability (define,inside,quantitative)     16 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) Performance: O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2n), O(n!), … – SE worse than polynomial O(1log n) n To tell if there is a needle in the haystack is easy, provided the hay is sorted: check the halfway element. If this is larger than the needle, discard the upper half, otherwise discard the lower half – and checking the halfway element in the remaining part until either found or not found.

  17. A Core Course on Modeling Week 7-A working model – and then?    Scalability (define,inside,quantitative)     17 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) Performance: O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2n), O(n!), … – SE worse than polynomial O(n) n For many tasks, the effort of doing n things is proportional to n. If one chicken lays three eggs per week, 5 chicken lay 15 eggs per week. This does not hold for everything, though … climbing n flights of stairs may take more effort than n  the effort of one. But computers don’t get tired, and many computer tasks scale similarly to chicken laying eggs.

  18. A Core Course on Modeling Week 7-A working model – and then?    Scalability (define,inside,quantitative)     18 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) Performance: O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2n), O(n!), … – SE worse than polynomial O(nlogn) n Sorting an array of unsorted things can be done in O(n log n). Military go faster, but they work in parallel.

  19. A Core Course on Modeling Week 7-A working model – and then?    Scalability (define,inside,quantitative)     19 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) Performance: O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2n), O(n!), … – SE worse than polynomial O(np) n Straightfowardly sweeping a matrix takes O(n3), but faster methods exist that bring this down to something like O(n2.5). The difference is noticable only for n sufficiently large.

  20. A Core Course on Modeling Week 7-A working model – and then?    Scalability (define,inside,quantitative)     20 Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions? Scale: n (=number of elements, states, records, time steps, …) Performance: O(1) – solving effort (SE) does not depend on n O(log n) – SE hardly depends on n O(n) – SE proportional to n O(n log n) – SE slightly worse than proportional O(np) – SE polynomial O(2n), O(n!), … – SE worse than polynomial O(n!) n The travelling salesman problem asks for the shortest route connecting n places. Exact solution is believed impossible in O(np) but good approximations sometimes take only O(np)

  21. A Core Course on Modeling Week 7-A working model – and then?    Specialization (define,outside,qualitative)     21 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? How much control should the modeler give the problem owner over • validity of assumptions, • regime of application, • use of results? A model, intended for specialists has to face the challenge of withstanding expert scrutiny. On the other hand, a model intended for laymen has to be robust against unintended misuse.

  22. A Core Course on Modeling Week 7-A working model – and then?    Specialization (define,outside,qualitative)     22 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? • Invest in presentation • graph style (line plot, scatterplot, polar plot, barchart, pie chart, 3D surface, …) • labels, captions, scales (zero, lin/log, units, major / minor scale lines …) • how to represent uncertainty? • which plots to combine in a single graph? The appropriate visualization style is the answer to the question: what message should this graph convey?

  23. A Core Course on Modeling Week 7-A working model – and then?    Specialization (define,outside,qualitative)     23 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? • When in doubt, be conservative A modeler resembles a blindfolded torreador: without full knowledge, consequences of the modeled system being mischievous may be quite harmful. For good reasons, the jackass adagium reads: ‘don’t try this at home!’

  24. A Core Course on Modeling Week 7-A working model – and then?    Specialization (define,outside,qualitative)     24 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? • Be warned for biased users • give a full and detailed account of as much as possible assumptions and estimates • insist on the report being indivisible • refrain from easy-to-misapprehend results • Get a second opinion Users having personal interests in model outcomes may not always take a balanced stand.

  25. A Core Course on Modeling Week 7-A working model – and then?    Specialization (define,outside,qualitative)     25 Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner? • Take care for self-fulfilling and self-denying prophecies • wicked problems: the problem solution post-hoc modifies the problem statement Both legendary prophet Michael Nostradamus (glass bowl?) and present-day opinion investigators (black ballot box?) relate to the working principle of self-fulfilling and self-denying prophecies.

  26. A Core Course on Modeling Week 7-A working model – and then?    Audience (define,outside,quantiative)    26 Audience: what size of intended audience does the model address? • Large size: low level of specialization • Large size: no option for bi-directional communication • Large size: consider interactive model (example: ‘stemwijzer’) In the 1999, Wachowski-brothers’ movie ‘the Matrix’, a simulation’s intended audience size is stretched so as to encompass the entire world’s population – with the extra that nearly everybody is ignorant of the fact.

  27. A Core Course on Modeling Week 7-A working model – and then?    Convincingness (conclude,inside,qualitative)     27 Convincingness: a model is more convincing if it contains fewer and/or less implausible assumptions. • Are assumptions logically deducible from other, less problematic assumptions? • If not, are there any first-principle ‘laws’ to back up assumptions (example: if this component behaves as a lever, we can use the physical laws of torque in a lever)? • If not, can we construct a simplified formal model system(example: tidal motions as a lumped system)with outcomes that can be empirically verified? • If not, can we get support from comparison to observations on an empirical model system(example: behavior of visitors in shopping malls to predict buying behavior)? • If not, is there at least any argument for consistency of the assumptions? (example: price elasticity, ‘fun’ in the taxi model) In the early days of photography, a photo (as prototypical empirical evidence) counted as proof. As this no longer holds, we need to re-think what constitutes ‘plausible evidence’.

  28. A Core Course on Modeling Week 7-A working model – and then?    Convincingness (conclude,inside,qualitative)     28 Convincingness: a model is more convincing if it contains fewer implausible assumptions. • Convincingness in the absence of ‘laws’: • conceive a plausible formal model system to which laws do apply (e.g., replace the world + oceans by a homogenous ball covered by a sheet of water). The measure of correspondence between these two is left out of the discussion; • convincingness relies on the agreement with empirical data. Tides are caused by subtle interplay between gravity, water (viscosity vs. inertia), and the complex geometry of the earth’s sea floors. A (glass box) model to predict tidal motions contains none of these, but a hypothetical ‘sheet of water on a perfect sphere’ instead.

  29. A Core Course on Modeling Week 7-A working model – and then?    Convincingness (conclude,inside,qualitative)     29 Convincingness: a model is more convincing if it contains fewer implausible assumptions. • Convincingness in the absence of a formal model system and/or no empirical validation: • look for an empirical model system • physical: water tank, wind tunnel • economical, social: comparative populations, historical survey data • biological: the guinea pig! Scale models are physical replica of the modeled system, usually at a smaller scale. Applications include windtunnel (flow, drag) and hydromechanical investigations, but also scale models as used in architecture and urban studies. These are examples of empirical model systems. BTW: J. Swift’s story seems to relate to a scale model; it is actually a social model.

  30. A Core Course on Modeling Week 7-A working model – and then?    Distinctiveness (conclude,inside,quantitative)     30 Distinctiveness: a model has higher distinctiveness if it allows distinction between alternatives that are more similar. • Most purposes ask for some form of distinctiveness. Examples: • prediction 1: something is correctly predicted to happen at time T1 rather than T2; how close can T1 and T2 be? • explanation: Q1 causes P1 and not P2; Q2 causes P2 and not P1. How close can Q1 vs. Q2, or P1 vs. P2 be so that the explanation holds? • optimization: input P1 yields output Q1, and input P2 yields output Q2, where Q1 correctly is found to dominate Q2. How close can P1 and P2 be? Many modeling purposes boil down to the ability of distinguishing between two alternatives that are as similar as possible. This relates to accuracy and precision, but also to non-numerical purposes.

  31. A Core Course on Modeling Week 7-A working model – and then?    Distinctiveness (conclude,inside,quantitative)     31 Distinctiveness: a model has higher distinctiveness if it allows distinction between alternatives that are more similar. This hinges on the notion of ‘distance’, explained in week 6. The quantitative measure for distinctiveness is usually a similarity measure: the more similar A and A’ whereas the model can successfully distinguish them, the better its distinctiveness.

  32. A Core Course on Modeling Week 7-A working model – and then?    Distinctiveness (conclude,inside,quantitative)     32 Distinctiveness: a model has higher distinctiveness if it allows distinction between alternatives that are more similar. Distinctiveness relates to two common types of errors: • false positive (conclude X where there is no X) • false negative (don’t conclude X where there is X) Successful spreading of false money is an example of false negative: the counterfeit goes unnoticed. Coin operated vending machines have to safeguard against both false positives and false negatives; the consequences of the two types of errors being very different. This is true for most models where distinctiveness plays a role.

  33. A Core Course on Modeling Week 7-A working model – and then?    Surprise (conclude,outside,qualitative)     33 The potential for surprise of a model outcome is the extent to which it may bring unforeseen new ideas. There are Open and closed spaces of outcomes. Examples of closed outcomes: • A model computing the probability of X can only produce a number between 0 and 1. • A model verifying Y can only produce ´true´ or ´false´. A widespread opinion is, that computations cannot ‘output more than you put in’. This underestimates computers’ scope of computation, or overestimates humans’ scope of anticipation.

  34. A Core Course on Modeling Week 7-A working model – and then?    Surprise (conclude,outside,qualitative)     34 The potential for surprise of a model outcome is the extent to which it may bring unforeseen new ideas. Examples of open outcomes: • result of using ontologies • outcome of evolutionary algorithms • outcome of PCA, abstraction, … Common feature: output space has size O(2n) whereas input space has size O(n) Evolutionary programming, applied to shape design. Given a comprehensible genotype representation, the space of phenotypes is virtually infinity. Xai Xu et. al., Special Issue of SIGGRAPH, Vol. 31, No. 4, pp. 57:1-57:10, 2012

  35. A Core Course on Modeling Week 7-A working model – and then?    Impact (conclusion,outside,quantitative)     35 The potential for impact of a model is the extent to which the model outcome can affect the stakeholders (either beneficially or adversely). Two perspectives: • prestige and profit: the more impact the better • risk and responsability: the less impact the better ‘Impact’ is a double-edged sword: some modelers attempt to increase the impact of their models whereas other modellers prefer to play safe and seek modeling with less impact.

  36. A Core Course on Modeling Week 7-A working model – and then?    Impact (conclusion,outside,quantitative)     36 The potential for impact of a model is the extent to which the model outcome can affect the stakeholders. Intended impact can be capitalized: r1=revenues in absence of model outcome; r2=revenues with model outcome present; c1=cost of ownership in absence of model outcome; c2=cost of ownership with model outcome present; =((r2-r1)-(c2-c1))/(|r2-r1|+|c2-c1|) • 0  ||1: magnitude of impact • If 0   1: positive impact (e.g., models for optimization); • if -1    0 negative impact (e.g., risk prediction or analysis) To assign a quantitative value to a model outcome we must assume some market mechanism where we identify revenues and costs of ownership, both in the absence of the model outcomes and in their presence.

  37. A Core Course on Modeling Week 7-A working model – and then?    Impact (conclusion,outside,quantitative)     37 The potential for impact of a model is the extent to which the model outcome can affect the stakeholders. Adverted impact can be quantified or capitalized: C=the estimated chance (per time interval) of an incident (=an erroneous model outcome, e.g. a type-I or type-II error) V=the estimated value loss per incident Both C and V can serve to quantify the negative impact of a model. Their product CV has dimension of money / time and can be added to c2 in the formula for .

  38. A Core Course on Modeling Week 7-A working model – and then?    Impact (conclusion,outside,quantitative)     38 The potential for impact of a model is the extent to which the model outcome can affect the stakeholders. The impact of a model should balance with its reliability.

  39. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     39 Not each of the eight criteria is equally relevant for all purposes. Given the purpose of the model, the modeler should identify the most relevant criteria to seek improvement when necessary. For instance: … (see next sheets)

  40. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     40 Not each of the eight criteria is equally relevant for all purposes. prediction (1, 2): convincingness, distinctiveness, impact An almanac is a model to serve the purpose of predictions (both 1 and 2). Its impact is rather large (given the long historical record of almanac-use), but the distinctiveness is small, and its convincingness decreases over time.

  41. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     41 Not each of the eight criteria is equally relevant for all purposes. compression: scalability, audience, distinctiveness Printing is a form of compression (transferring written text into printed text) with huge population and distinctiveness; by present day’s standards, the scalability is not impressive

  42. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     42 Not each of the eight criteria is equally relevant for all purposes. inspiration: surprise All the world’s religious traditions acknowledge holy scriptures of some sort as primary source of inspiration. Their potential for surprise is sometimes rather limited.

  43. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     43 Not each of the eight criteria is equally relevant for all purposes. unification: genericity, convincingness, surprise In alchemy, the pre-scientific forerunner of both chemistry and psychology, unification of conflicting properties was one of the leading purposes. In modern times, scores on all three criteria are low – when seen through modern eyes.

  44. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     44 Not each of the eight criteria is equally relevant for all purposes. abstraction: distinctiveness, convincingness, surprise The term ‘abstraction’ in modern art no longer defines a relation between a modeled system and a model; rather, the absence of any modeled system is the defining feature for true abstract art. The potential for surprise, however, sometimes still applies.

  45. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     45 Not each of the eight criteria is equally relevant for all purposes. verification: scalability, convincingness, impact Biting a golden coin is the proverbial way to verify its authenticity. Scalability and convincingness are probably low, but if the coin contains chocolate, or if the biter has metal inlays, the impact may be quite considerable.

  46. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     46 Not each of the eight criteria is equally relevant for all purposes. exploration: genericity, surprise Few models for exploration will have a larger potential of surprise than the Mars exploration robot.

  47. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     47 Not each of the eight criteria is equally relevant for all purposes. decision: convincingness, distinctiveness, impact The Delphi oracle was a decision-making model that derived its convincingness from its deliberately low distinctiveness – thereby being known for its major impact.

  48. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     48 Not each of the eight criteria is equally relevant for all purposes. optimization: genericity, scalability, convincingness, impact Of all generic optimization models for continuous systems, hill climbing is perhaps one of the most generic ones. It often suffers from scalability, though – even when implemented by modern, automated devices.

  49. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     49 Not each of the eight criteria is equally relevant for all purposes. specification: genericity, distinctiveness Sometimes the distinction between a specification model (blueprint) and a realization is somewhat blurred.

  50. A Core Course on Modeling Week 7-A working model – and then?    Criteria for modeling and purposes     50 Not each of the eight criteria is equally relevant for all purposes. realization: genericity, distinctiveness, impact Realization …

More Related