Persistent Data Structures in OCaml

1. Persistent Data Structuresin OCaml Anil Madhavapeddy, Imperial College Thomas Gazagnaire, Citrix Systems Inc DEFUN 2009, Edinburgh, UK http://www.flickr.com/photos/stuckincustoms/2826117627/

2. let sql = “SELECT middle.id, middle.f1_id, middle.f2_id, middle_f2.id, middle_f2.field1, middle_f2.date1, middle_f2.int1, middle_f1.id, middle_f1.field1, middle_f1.date1, middle_f1.int1 FROM middle LEFT JOIN base AS middle_f1 ON (middle_f1.id = middle.f1_id) LEFT JOIN base AS middle_f2 ON (middle_f2.id = middle.f2_id) where id=?” in let stmt = Sqlite3.prepare sqlin Sqlite3.bind stmt 1 (Sqlite3.Data.INT (500L)); match Sqlite3.step stmt with |Sqlite3.OK -> … AAAAAAAAAAAAAARGGGGH!

3. typeservice = | Email of string | Twitter of string | Other of (string * string) and person = { name: string; age: intoption; contacts: service list; } with persist ();

4. type service = | Email of string | Twitter of string | Other of (string * string) and person = { name: string; age: intoption; contacts: service list; } with persist ();

5. type service = | Email of string | Twitter of string | Other of (string * string) and person = { name: string; age: intoption; contacts: service list; } with persist ();

6. type service = | Email of string | Twitter of string | Other of (string * string) and person = { name: string; age: intoption; contacts: service list; } with persist ();

7. type service = | Email of string | Twitter of string | Other of (string * string) and person = { name: string; age: intoption; contacts: service list; } with persist ();

8. typeservice = | Email of string | Twitter of string | Other of (string * string) and person = { name: string; age: intoption; contacts: service; } with persist ();

9. avsm$ rlwrap ./top_talk Objective Camlversion 3.11.1 open Orm open Printf;; # let db = init "talk.db" ;; valdb : Sql_access.state = <abstr>

10. # let me = person { name="Anil”; age=(Some 31); contact=(Email "anil@recoil.org”) } db ;; valme :Talk.Orm.person = <obj>

11. # let tg = person { name="Thomas"; age=None; contact= (Other ("github","samoht") ) } db ;; valtg: Talk.Orm.person = <obj>

12. # printf"saved: %Lu %Lu\n%!" me#savetg#save ;; saved: 4 3 - : unit = () Object Call

13. # let print_resultshdr= printf “Query: %s\n" hdr; List.iter (fun x -> printf"found: <%s %s : %s>\n%!" x#name (match x#agewith |None -> "??" |Some a -> string_of_int a) (match x#contactwith |Email e -> e |Twitter t -> t |Other (s,u) -> s^":"^u)) valprint_results : string -> < age : int option; contact :Talk.service; name : string; .. > list -> unit = <fun>

14. #person_get;; • : ?age:[< `Exists of [< `Between of int * int | `Eq of int | `Ge of int | `Geq of int | `Le of int | `Leq of int | `Neq of int ] | `Is_none ] -> ?name:[< `Contains of string | `Eq of string ] -> ?id:[< `Exists of [< `Between of int64 * int64 | `Eq of int64 | `Ge of int64 | `Geq of int64 | `Le of int64 | `Leq of int64 | `Neq of int64 ] | `Is_none ] -> ?fn:(Talk.Orm.person -> bool) -> Sql_access.state ->Talk.Orm.person list = <fun>

15. #tg#set_age (Some 28);; - : unit = () # tg#save;; - : int64 = 3L #print_results"Find >=20" (person_get ~age:(`Exists(`Ge20)) db);; Query: Find >=20 found: <Anil 31 : anil@recoil.org> found: <Thomas 28 : github:samoht> - : unit = ()

16. # print_results (person_get ~name:(`Gt 5) db) ;; File ”talk.ml", line 429, characters 41-48: Error: This expression has type [> `Gt of int ] but an expression was expected of type [< `Contains of string | `Eq of string ] The second variant type does not allow tag(s) `Gt

17. #print_results"Find emails" (person_get ~fn:(funp -> match p#contactwith |Email _ -> true |_ -> false) db );; Query: Find emails found: <Anil 31 : anil@recoil.org> - : unit = ()

18. How it works • Sqlite3 as a database library • Custom OCaml callback functions in Sqlite3 • Camlp4 AST extension • Experience: used this in the 'LifeDB' project • AJAX / RESTful web servers are mostly mapping DB queries to/from JSON • with an ORM + json-static, not much left to do!

19. # method save = let _curobj_id = let stmt = Sqlite3.prepare db.Sql_access.db (match _id with | None -> "INSERT INTO person VALUES(?,?,?,NULL);" | Some _ -> "UPDATE person SET contact=?,age=?,name=? WHERE id=?;") in (Sql_access.db_must_ok db (fun () -> Sqlite3.bind stmt 1(Sqlite3.Data.TEXT (Sexplib.Sexp.to_string_hum(let sexp_of_contactv = sexp_of_servicev in sexp_of_contact _contact)))); Sql_access.db_must_ok db (fun () -> Sqlite3.bind stmt 2(match _age with | None -> Sqlite3.Data.NULL | Some _age -> Sqlite3.Data.INT (Int64.of_int _age))); Sql_access.db_must_ok db (fun () -> Sqlite3.bind stmt 3 (Sqlite3.Data.TEXT _name)); (match _id with | None -> () | Some _id -> Sql_access.db_must_bind db stmt 4 (Sqlite3.Data.INT _id)); Sql_access.db_must_step db stmt; match _id with | None -> let __id = Sqlite3.last_insert_rowid db.Sql_access.db in (_id <- Some __id; __id) | Some _id -> _id) in _curobj_id

20. let to_string _loc f = let id = <:expr< $lid:f.f_name$ >> in let pid = <:patt< $lid:f.f_name$ >> in let rec fn = function | <:ctyp@loc< unit >> -> <:expr< "1" >> | <:ctyp@loc< int >> -> <:expr< string_of_int $id$ >> | <:ctyp@loc< int32 >> -> <:expr< Int32.to_string $id$ >> | <:ctyp@loc< int64 >> -> <:expr< Int64.to_string $id$ >> | <:ctyp@loc< float >> -> <:expr< string_of_float $id$ >> | <:ctyp@loc< char >> -> <:expr< String.make 1 $id$ >> | <:ctyp@loc< string >> -> <:expr< $id$ >> | <:ctyp@loc< bool >> -> <:expr< string_of_bool $id$ >> | <:ctyp@loc< option $t$ >> -> <:expr< match $id$ with [ None -> "NULL" |Some $pid$ -> $fn t$ ] >>

21. Future Directions

22. Data Migrations • What happens if we change our schema? • Store schema in database • Structural type comparison on database attach • Tactics for migrations between two schemas • No manual migrations (A -> B -> C) as other ORMs

23. Other Languages • a portable “typed object filesystem” • No SQL in external interface, ever • Generate code for other languages: • Python : objects • F# : records / objects • Haskell: derive? • Carry your application data with you when new language arrives.

24. Info • http://github.com/avsm/ocaml-orm-sqlite • Release soon, announce on ocaml-list • Current release has no camlp4, deprecated Anil Madhavapeddy: anil@recoil.org http://twitter.com/avsm