Revealing the Man Behind the Curtain

1. Revealing the Man Behind the Curtain Designing a Runtime Library for D

2. The Language • High-level programming • Dynamic and associative arrays • Garbage collection • Unicode-aware • Systems programming • Pointers • Inline assembly • Direct calling of C code

3. User Requirements • High-level • Low barrier for entry / ease of use • It should be trivial to write a simple application • Applications should behave in a safe and predictable manner by default • Systems-oriented • Specialized information or access • Systems applications can often benefit from access to low-level information about the program or control over program behavior • Only get what you pay for • Systems programmers may have very explicit requirements regarding code size, so hidden dependencies must be avoided

4. Designer Requirements:The Compiler Writer • Language support routines • Dynamic array manipulation • Concatenation • Resizing • Sorting • Associative arrays • UTF conversions • Memory allocation / garbage collection

5. Designer Requirements:The Library Developer • User-accessible code • Thread management • Error handling • Input/output • Text processing

6. First Cut: Monolithic • Resources • Compiler team and library team • Team Interaction • Close collaboration for integrated features • Multithreading • Language / system exceptions • User control of garbage collection • Synchronized / inclusive releases

7. Second Cut: A Binary Design • Must formalize the interaction between components to decouple development effort • Language support component • Standard library component • Design limitations to eliminate compile-time dependencies • Functions • Structs • Simple data types

8. Standard Library Interface: Threads The garbage collector must interact with thread code to coordinate collection: void thread_suspendAll(); void thread_scanAll( void delegate( void*, void* ) scan, void* curStackTop = null ); void thread_resumeAll();

9. Standard Library Interface: Exceptions Expose a means of signaling system errors to decouple exception hierarchy from language support component: void onArrayBoundsError( char[] file, size_t line ); void onAssertError( char[] file, uint line ); void onOutOfMemoryError();

10. Third Cut: A Modular Design • Garbage collector research • Team skills: memory allocator and garbage collector design is a specialized field • Utilize externally developed garbage collectors to minimize team effort and obtain “best of breed” implementation • Varying needs of the users • Specialized garbage collectors • Typical vs. real-time

11. Language Support Interface: Platform-Specific Functionality The compiler is most suited for providing information about the target architecture: void* rt_stackTop(); void* rt_stackBottom(); void rt_scanStaticData( void delegate( void*, void* ) ); void rt_finalize( void* p, bool det = true );

12. Memory Management Interface void gc_enable(); void gc_disable(); void gc_collect(); uint gc_getAttr( void* p ); uint gc_setAttr( void* p, uint a ); uint gc_clrAttr( void* p, uint a ); void* gc_malloc( size_t sz, uint ba = 0 ); void* gc_calloc( size_t sz, uint ba = 0 ); void* gc_realloc( void* p, size_t sz, uint ba = 0 ); void gc_free( void* p ); size_t gc_sizeOf( void* p );

13. The Design Realized • Three components: • Language support • Garbage collector • Standard library • Proof of concept: Replaceable GC • Mark / sweep vs. C malloc • Link-time selection

14. private import tango.stdc.stdlib; private extern (C) void thread_init(); private extern (C) void onOutOfMemoryError(); extern (C) void* gc_malloc( size_t sz, uint ba = 0 ) { void* p = malloc( sz ); if( sz && p is null ) onOutOfMemoryError(); return p; } extern (C) void* gc_calloc( size_t sz, uint ba = 0 ) { void* p = calloc( 1, sz ); if( sz && p is null ) onOutOfMemoryError(); return p; } extern (C) void* gc_realloc( void* p, size_t sz, uint ba = 0 ) { p = realloc( p, sz ); if( sz && p is null ) onOutOfMemoryError(); return p; } extern (C) void gc_free( void* p ) { free( p ); }

15. import tango.core.Memory; import tango.stdc.stdio; class C { this( char[] n ) { printf( "ctor: %.*s\n", name = n ); } ~this() { printf( "dtor: %.*s\n", name ); } char[] name; } void main() { auto autoVal = new C( "autoVal" ); scope scopeVal = new C( "scopeVal" ); autoVal = null; GC.collect(); }

16. 31,744 gcbasic.lib 1,517 gcmalloc.d 383 Main.d 199,680 phobos.lib 66,048 tango.lib > dmd -release Main gcbasic.lib tango.lib > Main ctor: autoVal ctor: scopeVal dtor: autoVal dtor: scopeVal > dmd -release Main gcmalloc.d tango.lib > Main ctor: autoVal ctor: scopeVal dtor: scopeVal 104,476 Main.exe - using gcbasic.lib 91,164 Main.exe - using gcmalloc.d ------- 13,312

17. “Leak Detection” and Disposable • Validate program behavior • Specialized clean-up void rt_finalize( void* p, bool det = true ); bool onCollectResource( Object obj );

18. class Disposable { ~this() { printf( "dtor: Disposable\n" ); } void dispose() { printf( "dispose: Disposable\n" ); } } static this() { GC.collectHandler = &onCollect; } bool onCollect( Object o ) { if( auto d = cast(Disposable) o ) { d.dispose(); return false; } return true; } void discardDisposable() { auto d = new Disposable; } void main() { auto d = new Disposable; printf( "\n***** delete *****\n" ); d = new Disposable; delete d; printf( "\n*** GC.collect ***\n" ); GC.collect(); } > Main ***** delete ***** dtor: Disposable *** GC.collect *** dispose: Disposable

19. Overridable Object Monitors • Given the following interface: class Object { interface Monitor { void lock(); void unlock(); } } • Object monitors must be strucutred as: alias Object.Monitor IMonitor; struct Monitor { IMonitor impl; /* data */ }

20. Library Mutexes as Object Monitors auto mutex = new Mutex; auto cond = new Condition; bool ready = false; synchronized( mutex ) { while( !ready ) cond.wait(); // mutex released / acquired } synchronized( mutex ) { ready = true; cond.notify(); }