Half-Life 2 and Source

1. Half-Life 2 and Source Presented by Jay Stelly

2. Tech lead decisions • Design a better structure for code development • Investments in scripting • Improve iterative creative processes • Investments in distributed computing • Investments in optimization

3. Code development • Larger group of engineers • Larger code base • Lots of dependency • System coding vs. leaf coding • Lots of new technology, new interactions

4. Code development solutions • Reduce scope of programming decisions • Dynamic linking • Binary compatibility through interfaces • Similar to COM in theory, lighter weight in practice • Versioning system • Lots of changes behind layers • Build & deploy only changed components • One file leaf coding • Auto-registration systems • Systematic approaches to behavior, serialization, networking, hooks

5. Code development solutions • Faster compiles • DLLs are smaller • Reduced dependencies • Use incredibuild distributed to 12 machines • 4:1 - 8:1 improvement in most cases • Use VMPI • Many leaf coders use edit & continue • Console variables

6. Example: Shaders • Graphics API isolated in a component (Shaderapi.dll) • Reduced dependency on dx9 • Moved from dx8 to dx9 with almost no code changes outside this layer • Materials system isolated in a component • Allows mod authors to plug-in HLSL code • Leaf code uses material abstraction • Lots of custom shaders needed for speed • Unrolled into light/bone/state combinations • Used perl scripts to automate this metaprogramming • Write one, generate thousands • Use VMPI to distribute compilation of shaders • Incremental loading system • fast iteration by a single developer

7. Investments in scripting • General scripting languages failed for us • Shader language • AI behavior language • Animation description language (shipped) • Simple hierarchical text blocks • Simple • Easy to parse • Extensible • No conditionals, no state • Tools that operate on all scripts • Maps, soundscapes, materials, surfaceproperties, scenes, ragdolls, game_sounds, etc.

8. Improve iterative creative processes • Reduce scope of creative work • Decrease cost of iterations • Improve quality of iterations • Dynamic linking • Symbolic linking • Add layers of abstraction • Architect around workflow • Re-examined core production processes • Level design • Animation • Localization • Audio • Allow creative people to iterate in isolation • Also in parallel

9. Example: Level design • Symbolic links to audio, scenes, animations, code • Some embedded structures remain • Prefabs • templates • Entity I/O • Visual scripting language • Connect named inputs to outputs • Symbolic link to code • Extensible and backwards compatible • Create specialized inputs & outputs as needed

10. Example: Level design (continued) • Response system • Contextual symbolic links • Entity I/O can trigger concept • Set of contextual choices results in symbolic link • Example: male/female speech • Chooses an appropriate thing to say given the character and the current game context • Shaders instead of textures • Symbolic link to rendering code • Symbolic link to proxy code • Procedurally modify shader variables • Symbolic link to actual textures

11. Example: Animation • Symbolic link to animation • Can also be a blended set • Serialization uses symbols • Named blend parameters • Models share animation, can override by name • Symbolic links to bones • Add new bones to specific animations & characters • Share animation among models • Named attachments for code & levels • Named skeleton merges for models • Dynamically linked

12. Example: Animation (continued) • A composite model for animation • N-way blend matrix • Partial skeleton animations • Gestures • Facial animation + procedural eye movement • Lip sync • IK • All are independent layers • Author characteristic gestures & postures for characters • Build tons of higher-level animations out of these building blocks • Shipped ~3 hours of scenes in HL2 vs. ~10 minutes in HL1

13. Example: Scenes • Symbolic link to marks, objects, characters • Independently editable in the level • Link to AI commands • MoveTo, FaceTo, LookAt • Interactive with player • Link to animations, gestures, speech • Change gesture timing per-instance • Animation system builds composite from all of the above

14. Example: Audio • Game_sounds.txt • Simple config file for each sound effect • Allows single wave or set of random waves • Edit parameters (volume, attenuation, voice stealing, pitch shift, etc) per wave file • Soundscapes.txt • Position-based environmental sound • Also controls mixing • Soundmixers.txt • Classifies sound effects into groups across entire game • Named mixes of sound groups that can be linked into soundscapes • Dsp_presets.txt • DSP effect parameters, parametric algorithms • Automatic DSP tuning parameters

15. Example: Audio (continued) • Surfaceproperties.txt • Primarily for physics properties • Physically based sounds • Impacts • scrapes • footsteps • Audio reflectance properties • Audio hardness/softness for modifying physically based sounds • Symbolic links to actual sounds

16. Distributed computing • Half-Life used SMP/threaded systems for offline pre-processing • Source uses distributed systems instead (VMPI) • Scales higher • Cheaper • Better long-term (keep old machines) • Good for high latency jobs • Incredibuild • VMPI • Radiosity, hours in Half-Life, minutes in HL2 • Visibilty • Shader re-compile • Scales to 32 machines • Upgraded to 1Gb/s network • Use idle workstations as slaves

17. Optimization • Lots of future work • Memory bound more often than instruction bound • Much easier for leaf coders to get memory bound • Otherwise fill bound, or I/O bound usually

18. Summary • Isolate creative tasks, remove dependencies • Some architecture should be based on workflow & personnel • Programming in C++, scripting with simple text configuration files • Faster, higher quality iteration through focus, isolation, dynamic linking, reduced dependency, reduced compilation steps • Can use a larger team in a more parallel fashion