REGNET

1. REGNET Legal Information Retrieval and Application to E-Rulemaking Gloria Lau, Kincho Law, Gio Wiederhold June 8th, 2004

2. ADAAG in HTML UK DDA in HTML IBC in PDF Motivation • Multiple sources of regulations • Multiple jurisdictions: federal, state, local, etc. • Different formats, terminologies, contexts • Amending rules, conflicting ideas

3. Motivation • Multiple sources of regulations • Multiple jurisdictions: federal, state, local, etc. • Different formats, terminologies, contexts • Amending rules, conflicting ideas  Need for a repository • Locate relevant information • E.g., small business: penalty fees for violations  Need for analysis tool • Complexity of regulations • Multiple jurisdictions • Understanding of regulations & their relationships

4. Example 1: Related Provisions ADAAG Appendix 4.6.3 … Such a curb ramp opening must be located within the access aisle boundaries, not within the parking space boundaries. CBC 1129B.4.3 … Ramps shall not encroach into any parking space. Exception: 1. Ramps located at the front of accessible parking spaces may encroach into the length of such spaces … • CBC allows curb ramps encroaching into accessible parking stall access aisles, while ADA disallows encroachment into any portion of the stall.

5. Example 2: Related but Conflicting Provisions ADAAG 4.7.2 Slope. …Transitions from ramps to walks, gutters, or streets shall be flush and free of abrupt changes… CBC 1127B.5.5 Beveled lip. The lower end of each curb ramp shall have a ½ inch (13mm) lip beveled at 45 degrees as a detectable way-finding edge for persons with visual impairments. • ADAAG focuses on wheelchair traversal; CBC focuses on the visually impaired when using a cane.

6. Scope • Repository development • Relatedness analysis • Performance evaluation , results and applications Relatedness analysis Repository development

7. Repository development

8. Sources of data • Accessibility standards • Americans with Disabilities Act Accessibility Guide (ADAAG) • Drafted chapter for rights-of-way access • Associated public comments • Uniform Federal Accessibility Standards (UFAS) • British Standard BS 8300 • Scottish Technical Standards, Part S • International Building Code (IBC), Chapter 11 • Drinking water standards • Code of Federal Regulations, Title 40 (40 CFR) • California Code of Regulations, Title 22 (22 CCR) • Fire code • International Building Code (IBC), Chapter 9

9. Computational properties of regulations • Hierarchical tree structure • Referential structure • Discipline-centered, e.g., ADAAG for accessibility  Shallow parser to capture computational properties <regulation id="ibc" name="international building code" type="private"> <regElement id="ibc.1107" name="special occupancies"> … <regElement id="ibc.1107.2" name=“assembly area seating"> <reference id="ibc.1107.2.4.1" times="1" /> <concept name="assembl area" times="1" /> … <regText>Assembly areas with fixed seating shall comply … </regText> <regElement id="ibc.1107.2.1" name="services">...</regElement> <regElement id="ibc.1107.2.2" name=“wheelchair …">...</regElement> </regElement> </regElement> </regulation>

10. Scope • Repository development • Relatedness analysis • Performance evaluation , results and applications Relatedness analysis Repository development

11. Relatedness analysis ADAAG 4.1.6(3)(d) Doors (i) Where it is technically infeasible to comply with clear opening width requirements of 4.13.5, a projection ... UFAS 4.14.1 Minimum Number Entrances required to be accessible by 4.1 shall be part of an accessible route and shall comply with ... Related elements: door and entrance

12. Relatedness analysis • To utilize the computational properties of regulations for a complete comparison • Measure • Degree of relatedness: similarity score f(A, U)  (0, 1) • Nodes A and U are provisions from two different regulation trees f (0, 1)

13. Base score f0 computation • Linear combination of feature matching • F(A,U,i) = similarity score between Sections (A,U) based on feature i • N = total number of features •  = weighting coefficient • Feature matching • Based on the Vector model using cosine similarity as the distance between feature vectors • Non-Boolean features • A measurement of “2 inches max” can be a 70% match to “2 inches” • Synonyms exist, e.g., ontology defined for chemicals • Perform vector-space transformation prior to cosine computation

14. Score refinements based on regulation structure • Neighbor inclusion • Diffusion of similarity between clusters of nodes in the tree

15. Score refinements based on regulation structure • Reference distribution • Diffusion of similarity between referencing nodes and referenced nodes in the tree • E.g., f(A5.3, U6.4(a)) updates f(A2.1, U3.3)

16. Scope • Repository development • Relatedness analysis • Performance evaluation, results and applications Relatedness analysis Repository development

17. Performance evaluation • Conduct a user survey of rankings of similarity • 10 randomly chosen sections from the ADAAG and UFAS • Ranks 1 to 100 in the order of relevance • Root mean square error (RMSE) • = user-generated ranking vector • = machine-predicted ranking vector

18. Survey results - Tabulated RMSE’s • Compared our analysis to Latent Semantic Indexing (LSI) •  = structural weighting coefficient •  = feature weighting coefficient • Average RMSE smaller than LSI • Measurement feature performs best • No improvement in result observed for structural comparison

19. Results of comparisons: ADAAG vs. UFAS • Related accessible elements: door and entrance • Neighbor inclusion reveals higher similarity

20. Results of comparisons : UFAS vs. BS8300 • Terminological differences - revealed through neighbor inclusion

21. Results of comparisons : UFAS vs. Scottish Technical Standards • Terminological differences - revealed through reference distribution • Stairs and ramps

22. Application: e-rulemaking • Application domain: e-rulemaking • Comparison between draft of rules and the associated public comments • ADAAG Chapter 11, rights-of-way draft • Less than 15 pages • Over 1400 public comments received within 4 months • Comments ~10MBin size; most are several pages long  New regulation draft can easily generate a huge amount of data that needs to be reviewed and analyzed • Parsing of the draft and comments • From HTML to XML • Recreate structure of the draft using our shallow parser • Extract features from the draft and comments • Treat individual comments as provisions

23. E-rulemaking Drafted regulations compared with public comments

24. Results from e-rulemaking application • Related section in draft and public comment

25. Results from e-rulemaking application • No related provisions identified • Concern not addressed in the draft

26. Conclusions • A framework for regulatory repository • Structure of regulations recreated in XML • Feature extractions • Prototype for similarity comparisons • Contextual comparisons • Domain knowledge • Structural comparisons • Performance Evaluation, Results and Applications • User survey and comparisons with LSI • Observations of comparisons between Federal, State, non-profit organization mandated codes and European standards • Application on e-rulemaking

27. Future research directions • In the legal domain • Regulatory competition • Cross border data transfer laws • Especially in the polyglot countries in EU • Regulatory updates • Track changes in updates • Track cross references between regulations • Extension of application to other domains of semi-structured documents • Software specifications • User manuals • Similarity/relatedness is settled - how about differences and conflicts? • A lot of almost identical provisions

28. Thank You!