Thanks Terry for being the greatest advisor ever and importantly for making me take 6.001.

2. Thanks Terry for being the greatest advisor ever and importantly for making me take 6.001. Thanks George for giving me that intersecting rectangle assignment two years back. It sparked this thesis. Thanks Patrick for being such an optimist and taking 6.034 and 6.xxx. A lot of my learning comes from you. Thanks to Takehiko, Axel, Jimmy, Neri, Onur, Sergio, Daniel for patiently enduring my explanations. Your reviews were invaluable.

3. Computational Model of Visual Interpretation Sketches are Powerful From the very early phases of design conception, designers uses sketches as a design tool. Sketches however are ambiguous. Meanings are associated on fly as the designer ‘comes up’ with certain ideas while working with it. There is no hierarchy in a sketch. In fact ‘structure’ is established only after meanings are applied to the sketch. However even in such structurally and conceptually fluid territory, the designer solves most of his design problems and very often comes to quick resolutions.

4. Computational Model of Visual Interpretation Thesis In order to understand the nature of such ‘design exploration’ and augment it with computational support, we must understand the nature of these assertions, how we interact with them and how we visually interpret them. That is the motivation of this thesis.

5. Computational Model of Visual Interpretation The Sketch Environment • Fluid and Extensible • no apriori declaration of intent • Ambiguity and Multiplicity of Semantics • parts assume different meanings with interpretation • Context • realms of thought (site-plan) • - Interpretation • real-world concepts – buildings-courtyard-streets • Reflection • opportunism in design (emphasize courtyard)

6. Current Trends of Design Computation Systems Current Models: Hierarchical Rigidity A sketch has no structural hierarchy. The designer leaps from context to context changing his structural interpretation on the fly. However computation systems have strict hierarchies allowing them to look at the world in a monotonous way. Also the overheads are so high in developing the structural abstraction that the fluidity of a sketch vanishes and the abstraction itself becomes a design issue. “Descriptions fix things in computation, and nothing is ever more than its description anticipates explicitly.”[Stiny 1994]

7. Current Trends of Design Computation Systems Hierarchical Rigidity Current Models: No Reflection Generative systems have no layers of reflection. Hence there is no way the generation can be guided by global principles. Design process as a SEE – DO – SEE cycle.

8. Computational Model of Visual Interpretation Society of Mind Minsky No Structure Computation Stiny Near-Miss and Learning Winston Reflection in Design Schon Knowledge in Learning Papert Visual Routines / Spatial Reasoning Ullman Principle of Opportunism In Design Papazian

10. Concept1 Concept1 Shape A Concept2 Shape A Concept2 Concept3 Theoretical basis for the Model Abstraction and Conception Spaces Sketches are abstract representations. However, designers use them to explore real world concepts. The actual shapes that form the sketch physically comprises the abstraction space, while concepts that the abstractions trigger in the designers mind make up the conception space. Interpretation 1 Interpretation 2 (Notice the structural / topological difference)

11. ENCLOSURE Shape Concepts Theoretical basis for the Model Shapes and Concepts • Shapes are the real visual assertions, while concepts are imagined. • Hence concepts might not have physical existence in the geometry. In fact Concepts need not have any geometric descriptions at all. • Shapes are a flat collection of parts, while concepts have hierarchy. Interpretation gives concepts their hierarchy. • Concepts have multiple descriptions, strengths, and even actions-rules associated with them.

12. Theoretical basis for the Model Schemas – Procedural Knowledge Units Assertions Trigger Trigger Schemas Generate Concepts Schemas are reasoning modules independent of structure and can perform reasoning using geometric, functional, relational or logical predicates. (If ‘assertion IS ‘(gathering-place) IS ‘(next-to-buildings) Then MAKE-COURTYARD-CONCEPT)

13. Theoretical basis for the Model Schemas – Types GEOMETRIC (Description) (courtyard) -> (enclosure) -> ‘(unsorted (edge,edge,edge,*)) -> ‘(unsorted (edge,edge,*,*)) (alley) -> (or (and (aspect, (more-than 2)) (sorted (gap, edge,gap,edge)) (and (aspect, (less-than .5) (sorted (edge,gap,gap,edge))) RELATIONAL (Constraints) (courtyard) -> (not (inside (figures))) (courtyard) -> (not (next-to (courtyard))) REASONING (Consistency) (courtyard) -> (not (inside (room))) FUNCTIONAL (Higher Level Design Concepts) (courtyard) -> (gathering-place) -> (adjacent-to (many (building))) (courtyard) -> (has-a fountain)

14. Theoretical basis for the Model Schemas – Learning Schemas are learnt from real world interaction and experience. They are used to schematically store our knowledge about the world. Schemas can trigger other schemas within the context as well. For example, a courtyard-schema might trigger entrances-schema or fountain-schema. Therefore, Schemas also gives a structure to the entire conception space as well.

15. Theoretical basis for the Model Active + Passive = Imagined World Active Assertions Passive Assertions Passive Assertions Passive Assertions The ‘Imagined world’ which contain many ways to look at the world. The shape in the figure is neither resolved as a collection of three line concept nor as a single planar concept, but all the concepts are simultaneously present. Subsequent selection of sub-sets of active agents creates a partial-mental state and allows us to imagine the assertion as either three lines or a planar concept.

17. Computational Model of Visual Interpretation PASSIVE ACTIVE

18. Computational Model of Visual Interpretation Focus In this framework, the perceptual focus manager selects a subset of the shape assertions in the ‘world’. To simulate gaze in a simple manner the lisp LISP machine creates a boundary of preset dimensions, randomly places it in the sketchpad and selects the visual assertions, which are completely or partially inside the imposed boundary

19. Computational Model of Visual Interpretation Computational Model of Visual Interpretation Passive Assertions Multi-representation creates are strong knowledge-base. Direction-ideas Figure-ideas Position-ideas Edge-ideas

20. Computational Model of Visual Interpretation Computational Model of Visual Interpretation Schema generates the concept The courtyard concepts are generated by the following schema, (courtyard-schema) -> (concept which-is-in ‘world which-is-not-inside ‘figures which-is-not ‘too-wide ‘too-long which-has ‘(edge edge edge edge) which-has ‘(edge edge edge) which-is-not ‘too-small ) These concepts are added to the ‘Imagined World’, for the next cycle of interpretation. By default, the concepts remain as ‘passive’ assertions.

21. Computational Model of Visual Interpretation Computational Model of Visual Interpretation Action-Rules Action-rules embedded in the courtyard-concepts can be used to add ‘active assertions’ to the abstraction space.

23. Computational Model of Visual Interpretation Computational Model of Visual Interpretation Example Runs

25. Computational Model of Visual Interpretation Computational Model of Visual Interpretation TRIGGER as an exploration tool It makes explicit a large set of possibilities, without any bias. There can surprising opportunities that the human designer might have missed.

26. Computational Model of Visual Interpretation Computational Model of Visual Interpretation Sketches with Performance Criteria Evaluation modules can directly process the sketches and suggest high-performance alternatives. TED I FOUND A NARROW ALLEY…

27. Computational Model of Visual Interpretation Computational Model of Visual Interpretation Shape Grammar Implementation Exact structural match would not be necessary for triggering rules. This rule can match the assertions below