1 / 56

An Electronic Field Guide: Plant Exploration in the 21 st Century

An Electronic Field Guide: Plant Exploration in the 21 st Century. Peter Belhumeur, Columbia University David Jacobs, University of Maryland John Kress, Smithsonian Institution Steven Fiener, Columbia University Ravi Ramamoorthi, Columbia University. Collaborators. Smithsonian Institute

tave
Download Presentation

An Electronic Field Guide: Plant Exploration in the 21 st Century

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Electronic Field Guide:Plant Exploration in the 21st Century Peter Belhumeur, Columbia University David Jacobs, University of Maryland John Kress, Smithsonian Institution Steven Fiener, Columbia University Ravi Ramamoorthi, Columbia University

  2. Collaborators Smithsonian Institute • John Kress • Rusty Russell • Ida Lopez • Norm Bourg University of Maryland • Haibin Ling • David Jacobs • Columbia University • Sean White • Ravi Ramamoorthi • Steve Feiner

  3. Dioscorides: “De Materia Medica” (77 A.D.) “Codex Vindobonensis” (512 A.D.)

  4. Abu Hanifa Al-Dinawari: “Kitab Nabat” (895)Ibn Wahshiyya: “Al Filaha Nabatiya” (930) كتابالفلاحةالنبطية المقدّمة ترجمهآبوبكربنوحشيّة بســماﷲالرحمنالرحيموصلىاﷲعلىسيّدنامحمدوالهوصحبهوسلم هذاكتابالفلاحةالنبطيّةنقلهمنلسانالكسدانيينالىالعربيّةآبوبكراحمدبنعليبنقيسالكسدانيالقيسيالمعروفبابنوحشيّةفيسنةاحدىوتسعينوماتينمنتاريخالعربمنالهجرةواملاهعلىبنابيطالباحمدبنالحسنبنعليبناحمدبنمحمدبنعبدالملكالزيّاتفيسنةثمانيعشيرةوثلثمايةمنتاريخالعربمنالهجرةفقالله اعلميابنيانيوجدتهذاالكتابفيجملةماوجدتمنكتبالكسدانيينمترجمبترجمةمعناهابالعربيةكتابافلاحالارضواصلاحالزرعوالشجروالثمارودفعالافاتعنهافاستكبرتهواستطلتهوخطرببالياختصارهثمفكرتفاذاذلكخطاءغيرصوابمناجلانّقصديالاولوغرديانماهوايصالعلومهؤلاءالقوماعنيالنبطالكسدانيينمنهمالىالناسوبثهافيهمليعرفوامقدارعقولهمونعماﷲتباركوتعالىعندهمفيادراكالعلومالنافعةالغامضةواستنباطماعجزعنهغيرهممنالامموذلكانيوصلتالىكتبهمفيزمانقددرسفيهذكرهموانمحتاخبارهموعدمتاعلامهمحتىلميبقالاذكرهمفقطوذكربعضعلومهمذكراكالخرافاتبلامعرفةممنيذكرهافلمارايتذلكاجتهدتفيطلبكتبهمفوجدتهاعندقومهمبقاياالكسدانيينوعلىدينهموسنّتهمولغتهمووجدتعندهممنالكتبوهمفينهايةالكتمانوالاخفاءوالحجورلهاوالجزعمناظهارهاوكاناﷲتعالىوجلقدرزقنيقبلذلكمنالمعرفةبلغتهمالتيهيالسريانيةالقديمةمالمارهمعكثيراحدوذلكاننيمنهماعنيمننسلبعضهمومكّننياﷲتعالىمنالدراهموالدنانيرولهالحمدفوصلتالىمااحببتمنكتبهمبهذهالوجوهالتيعددتهامناننيمنهمواننيعارفبلغتهمواننيمتمكنمنالمالفاستعلمتالمدارةوالبدلولطيفالحيلةالىماامكننيمنكتبهمونظرفيها وهيفييدهانهمحتاجالىفهممافيهااذكانتالكافةمنهؤلاءالقومالذينهمفيبقاياهمكالبقروالحميروالعاجزينعنفهمشيءمنالعلوماسلافهمالاانالانسانالذيوجدتهذهالكتبمجموعةعندهيتميزعنهذهالجملةويتفضلعنحماريةهذهالكافةفلمتُهعلىالافراطفيكتمانهذهالكتبوخبيهذهالعلوموقلتلهانكتريدالاحتياطبفعلشيءهودرسذكرقومكوطمرمحاسنهموهذاالذيتعملهاقتديتفيهبمنمضىقبلكوانتوالماضينمحطينفيذلكعلىمنتقدّممنالعلمائكمالذينهمعلمايواسلافيمعكموهذاالفعلكانالطريقالىاندراسذكراسلافناوغيوبةعلومهمعنالناسواختفاءمحاسنهمعنهمولوقدنقلتهذهالكتباوبعضهاالىالعربيةحتىينظرالناسفيهاعرفوامقدارعلومناوانتفعوابماوضعاسلافناوصارفيذلكضربمنالفخرلنا

  5. Captain James Cook (1768)

  6. Charles Darwin (1859)

  7. Lt. Charles Wilkes (1838)

  8. Species Identification: A Pain-staking Process • Physical specimens are loaned and available to only one scientist for year(s) • Herbaria visited in person • Or samples are collected and never or incorrectly identified

  9. Our Goal: Accelerate Species Identification • Photograph and Electronically Catalogue Herbaria • Make data available in the field • Assist in locating the right information • Zoomable user interface • Search with metadata (text, gps, ….) • Visual Search • Not to build an autonomous system • Automatic species identification still out of reach • We want to enhance the capabilities of experts

  10. Project I:Digitizing the U.S. National Herbarium

  11. Acquisition System • Phase One H20 back on Hasselblad 502 with 80mm lens • Resolution: 3600x5000 = 18M pixels • Dynamic Range: 36 bits per pixel over 3 color channels

  12. Specimen Images • 95,000 Type Specimens • 15,000 non-type specimens digitized • Derivatives available on web • Thousands of hits a month • Hi-resolution available online and DVD • Special projects • Puerto Rico rarities, Mexican Boundary Survey, Hawaii….

  13. Acer politum Greene Type Specimen Digital Collection Digital Image of Type Specimen Textual Information Other Specimens (Not Types) 3D Computer Models Photographs of live plants Publications Decision Trees Verbal Description Locations Enhanced Type Specimen Digital Collection Basic Type Specimen Digital Collection

  14. 1/3 of the Earth’s Plants....

  15. Project II: Automating Visual Search

  16. Our Work in Plant Identification • We haven’t been trying to emulate botanists • Expert judgments can rely on features that are hard to automate • General purpose algorithms needed for basic analysis, though humans take these for granted. • Experience in other recognition tasks

  17. Plant ID: Visual Search Criteria Shape Search Criteria • Shape – our current focus • Texture • Venation Texture Venation

  18. Albrecht Dürer, “Four Books on Human Proportion” (1528)

  19. D’arcy Thompson, “On Growth and Form” (1917)

  20. D’arcy Thompson, “On Growth and Form” (1917)

  21. D’arcy Thompson, “On Growth and Form” (1917)

  22. Shape Matching in Computer Vision, abridged... • Grenander ’81 • Mumford ‘91 • Fry ‘94 • Kimia, Tannenbuam, and Zucker ’95 • Basri, Costa, Geiger and Jacobs ’95 • Zhu and Yuille ‘96 • Liu, Geiger, and Kohn ’98; Geiger, Liu, and Kohn ‘03 • Belongie, Malik, Puzicha ‘01 • Osada, Funkhouser, Chazelle, Dobkin ‘03 24

  23. Representations that Capture Part Structures • Part relationships can vary • The Euclidean distance • The inner-distance: length of the shortest path between landmark points (see also Bronstein et al. 2003)

  24. Inner-Distance Inner-Distance Shape Context (IDSC) 5 Angle Histogram for Point 1 θ 1

  25. IDSC: Histogram for Each Point Angle 5 θ 1 Histograms for All Points (see also Belongie et. al 2001)

  26. Experiment I: Swedish Leaf Dataset The Swedish leaf dataset contains leaf images from 15 species. Each species has 75 leaves, 25 training and 50 for testing. • How often is the algorithm correct on first guess? • Combination of simple features: 82% [Söderkvist 01] • Fourier descriptors: 89.60% • SC+DP: 88.12% • IDSC+DP: 94.13%

  27. Experiment I: Swedish Leaf Dataset

  28. Project III:Designing User Interfaces and Prototypes

  29. Devices wireless audio see-through head-worn display orientationsensors location sensor hand-held computer

  30. Augmented Reality Versions Rep. John Tanner

  31. Prototype: Tablet + Camera +

  32. Prototype: UMPC Version

  33. Prototype: Mobile Phone Version

  34. Not-There-Yet-Prototype: iPhone

  35. Project IV:Digitizing Regional Floras

  36. Plummers Island, MD

  37. Complete Collection of Vascular Plants Plummers Island dataset contains leaf images from 249 species, with 5000+ images.

  38. Woody Plants of DC Area

  39. Complete Collection of All Woody Plants DC Woody dataset contains 245 species, 10,000+ images

  40. System Performance

  41. Video Demo: UMPC Prototype Demo

  42. In Use by Botanists in the Field 44

  43. What’s Next?

  44. What’s Next? Temperate Floras: • Trees of Central Park, NYC (158 species) • Woody Plants of U.S., (5000+ species)

  45. 48

  46. 49

  47. 50

More Related