1 / 39

CUPUM 2007 – Foz do Iguaçu, Brazil NEW INFORMATION TECHNOLOGIES FOR COMMUNITY DEVELOPMENT:

CUPUM 2007 – Foz do Iguaçu, Brazil NEW INFORMATION TECHNOLOGIES FOR COMMUNITY DEVELOPMENT:. Intelligent Middleware for Understanding Neighborhood Markets Prof. Joseph Ferreira, Jr. and Dr. Raj Singh MIT, Urban Information Systems Department of Urban Studies & Planning July 12, 2007.

armand
Download Presentation

CUPUM 2007 – Foz do Iguaçu, Brazil NEW INFORMATION TECHNOLOGIES FOR COMMUNITY DEVELOPMENT:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CUPUM 2007 – Foz do Iguaçu, BrazilNEW INFORMATION TECHNOLOGIES FOR COMMUNITY DEVELOPMENT: Intelligent Middleware for Understanding Neighborhood Markets Prof. Joseph Ferreira, Jr. and Dr. Raj Singh MIT, Urban Information Systems Department of Urban Studies & Planning July 12, 2007

  2. NEW INFORMATION TECHNOLOGIES FOR COMMUNITY DEVELOPMENT • Applaud recent interest in communtiy data centers, mashups, and PPGIS • But, current practice cannot meet high expectations • Problems • Data maintenance and currency • Limited and inflexible modeling & analysis • Solution – more IT ??!! • Modular service chaining - (geo)web services, SOA • Intelligent middleware for modeling and analysis • Lessons (even without adding new IT) CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  3. Trends in IT for Community Development • Neighborhood Information Systems • Local Data Centers serving NGOs & sharing data • PPGIS (public participation GIS - beyond census tables) • Public/private/non-profit partnerships • Examples: NNIP website: http://www.urban.org/nnip • Mashups and Web 2.0 • Internet GIS: ArcIMS, MapServer, GeoServer… • Map ‘mashups’ on Google Earth/Maps, Virtual Earth, Yahoo maps… • User-generated content: Flickr photos, social networking… CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  4. GIS Office for State of MassachusettsMassGIS: http://www.mass.gov/mgis/ CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  5. MassGIS: Example of very good online mapping and download site for typical US State CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  6. Google Earth Mashup: Nanaimo, Canada • City website: http://earth.nanaimo.ca/data.html • NanaimoMaster and NanaimoVirtuallyDowntown using Google Earth+MapGuide CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  7. National Neighborhood Indicators Partnership: http://www.urban.org/nnip Non-governmental organizations focused on needs of particular communities CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  8. The Providence Plan Providence Rhode Island USA CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  9. Neighborhood Knowledge Los Angeles • http://nkla.sppsr.ucla.edu • Land Opportunities Tracking System • http://lots.ucla.edu CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  10. Limitations of Neighborhood Information Systems • Keeping data up to date • No problem for decennial census data • Big problem for land use/ownership (cadastre) and permitting • More difficult as more parties try to share data • Interpreting & Analyzing ‘official’ data • Must clean/re-interpret data for local use • Hard to include ‘local knowledge’ • Customized analyses are useful but non-trivial CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  11. Intelligent Middleware for Understanding Neighborhood Markets Urban Markets Initiative Pilot Project A collaborative effort among Massachusetts Institute of Technology (MIT), Boston Metropolitan Area Planning Council (MAPC) Boston’s Department of Neighborhood Development The Boston Foundation (TBF) Open Geospatial Consortium (OGC) with support from the Urban Markets Initiative of The Brookings Institution and participation of local CDCs.

  12. People Working on the ‘Intelligent Middleware’ Project Principal Investigator: Joe Ferreira, Professor, MIT Principal Collaborators: Holly St. Clair, Director, MAPC Data Center Larry Braman, Manager, Mapping and Data Services, Boston DND Charlotte Kahn, Director, Boston Community Indicators, TBF MIT Research Team: Raj Singh, system architect (now at OGC) Jiawen Yang, database services (now Assist. Prof. Georgia Tech) Myounggu Kang, mapping services (now Assist. Prof. Seoul U.) Mi Diao, Xiongjiu Liao, Weifeng Li (PhD candidates) Sponsored by: The Brookings Institution, Urban Markets Initiative

  13. ‘Intelligent Middleware’ ProjectGoals and Concepts Facilitate data sharing within metropolitan areas • Without everyone needing their own ‘data center’ • Because everyone needs to change their copy of the data Allow reinterpretation of official data • Based on local knowledge • Without explicitly changing the original data • Codify local knowledge as rules to interpret official data on the fly Deliver results into familiar desktop environment • Tables into spreadsheet or browser; maps as mashups • Examples: Identify key landowners, targeted land uses, … CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  14. Police Dept. Assessing Dept. Health Dept. Housing Dept. Data Sets Data Sets Data Sets Data Sets Read-only, ‘Official’ data repositories Processing Normalize, trim, merge, transform, overlay, cookie-cut, extrapolate, qualify, … Intelligent data intermediaries Housing portal Service Service Environmental portal ‘Smart’ Middleware AlternativeIntelligent Middleware for Understanding Neighborhood Markets CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  15. One Example: Land use & Ownership Patterns • Want: • REPORT & MAP of largest landowners: Top-10 Owners • Keep map/report accurate and up to date • Allow flexibility in definition of ‘neighborhood’ • Allow flexibility in which land uses to consider (residential, open space…) • Problem: Parcel map & Assessing database not enough • Spelling problems: many different spellings of city agencies and other large landowners • Updating: old snapshot of parcels is out of date CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  16. Desired Output: Table, Chart, and Map in Familiar Desktop Environment CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  17. CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  18. CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  19. Middleware Example: Prototype NGO Webpage (for DSNI) CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  20. …but Top-10 Landowner List has Problems: CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  21. Results AFTER Standardizing Some Owner Names CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  22. Solving the Spelling Problem • Problem: many different spellings of city agencies and large landowners in parcel data • Solution choices: • Top-down: standardize spellings in official datasets • Bottom-up: fix spellings in copy of dataset • Middle-out: accumulate spelling changes separately and apply on-the-fly as needed …build ‘middleware’ for middle-out strategy CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  23. Example of RULE to Fix Spelling Error • Standardize OWNER names • Create ‘update’ RULE to fix spellings • Apply on-the-fly to Boston’s parcel data (BASETABLE) to produce temporary tables (FACADES) with spelling corrections • Use temporary table for REPORTS and THEMATIC MAPS • Example of one RULE for City of Boston • UPDATE fas_parcel_bos05 • SET OWNER_FY05='CITY OF BOSTON' • where OWNER_FY05 like '%CITY%' • and ((OWNER_FY05 like '%BOSTON%' or OWNER_FY05 like %BSOTON%') • and OWNER_FY05 not like '%CITYWIDE%' • and OWNER_FY05 not like '%SUITES%' • and OWNER_FY05 not like '%LIGHT%‘ • and OWNER_FY05 NOT LIKE '%PROPERTIES%' • and OWNER_FY05 NOT LIKE '%LLC%' • and OWNER_FY05 NOT LIKE '%L L C%' • and OWNER_FY05 NOT LIKE '%CREDIT UNION%)'; CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  24. ‘Intelligent Middleware’ ProjectGoals and Concepts • Facilitate data sharing within metropolitan areas • Allow reinterpretation of official data based on local knowledge • Deliver results into familiar desktop environment HOW? • Modular design: interoperable parts, distributed access, secure access control • Data intermediary manages ‘virtual’ data center middleware • Middleware uses web services and open standards • Middleware is hidden from end-user applications • User/group logins and SSL provide data sharing control CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  25. Police Dept. Assessing Dept. Health Dept. Housing Dept. Data Sets Data Sets Data Sets Data Sets Read-only, ‘Official’ data repositories Processing Normalize, trim, merge, transform, overlay, cookie-cut, extrapolate, qualify, … Intelligent data intermediaries Housing portal Service Service Environmental portal ‘Smart’ Middleware AlternativeIntelligent Middleware for Understanding Neighborhood Markets CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  26. Middleware Components CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  27. Tables are not static but result from ‘web query’: • http://uis.mit.edu/umi/service/report.php? • Request=QueryReport& • name=top_own_parcels_a_jf1& • tablename=own05_mixed_after& • studyarea=dsni& • xsl=http://uis.mit.edu/umi/xl/report.xsl CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  28. Same URL in Browser: • http://uis.mit.edu/umi/service/report.php? • Request=QueryReport& • name=top_own_parcels_a_jf1& • tablename=own05_mixed_after& • studyarea=dsni& • xsl=http://uis.mit.edu/umi/xl/report.xsl CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  29. Thematic Maps are also web services: • http://uis.mit.edu/umi/mapservice/umimap.php? • request=GetMap& • layers=top_own_parcels_map1& • WIDTH=500&HEIGHT=500& • SRS=EPSG:4326& • TRANSPARENT=true • Map requests conform to Web Mapping Standard (WMS) of Open Geospatial Consortium (OGC) • Many applications now support WMS • E.g., Excel, Google, ArcGIS, … CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  30. A WMS call to the middleware services yields a thematic map top-10 landowners for all of Boston The URL: http://uis.mit.edu/umi/mapservice/umimap.php?request=GetMap&layers=testjf0004 The same URL works from inside of ArcMap (see next slide) CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  31. CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  32. Middleware Components • Prototype on RHEL4 Linux PC running open source applications • Database server: Postgres with Postgis spatial extensions • Mapping server: MapServer (or GeoServer) • Web server: Apache2 (with Tomcat) • Web services programmed in PHP • Additional middleware management tools in AJAX (asymmetric javascript and XML) CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  33. CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  34. CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  35. CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  36. Middleware Use: More than Spelling Fixes… Understand Land Use and Ownership Patterns: • Identify “top-10” landowners in neighborhood • Trace effective ownership: who controls ABC Corp. • Track (un)desired land use, vacant land suitable for reuse… • Study trends: gentrification impacts of condo conversion, predatory lending practices, … Measure urban performance: • Model affordability, accessibility, … • Estimate retail ‘gaps,’ development potential, … CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  37. Conclusions: (1) Key Concepts Sharing data within a metropolitan area • Avoid ‘data center’ limitations as small geographies and large/changing datasets explode data needs • Provide mechanism for accumulating,utilizing, and sharinglocal knowledge about land use, ownership, and market potential (independently of official datasets) • Use web services approach, with interoperability standards, to build ‘lego blocks’ for many management and end-user applications. • Examine the skill sets and institutional relationships that can take advantage of ‘service oriented architectures’. CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  38. Conclusions:(2) Community Development Implications • As volume and detail of ‘relevant’ data grows, traditional sharing of data and tools becomes more problematic • Corrolary: Google Earth is great but can’t meet expectations • Web services and ‘middleware’ offer promising avenues for deeper sharing, but • Require different mix of skills and institutional arrangements • Won’t be offered soon by vendors (unless we push!) • Suggests new direction and short-term strategies • More complex urban models and performance indicators have limited use unless built as plug-and-play modules that provide tunable web services. CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

  39. Thanks… Joe Ferreira, jf@mit.edu Raj Singh, rajsingh@mit.edu ‘Middleware’ project website: http://uis.mit.edu CUPUM07: Intelligent Middleware - Joe Ferreira - July 12, 2007

More Related