Ahead of time dynamic translation l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 43

Ahead of Time Dynamic Translation PowerPoint PPT Presentation


  • 190 Views
  • Updated On :
  • Presentation posted in: General

Ahead of Time Dynamic Translation. PreJit/NGEN by any other name George Bosworth Microsoft MRE’04 March 21, 2004. Dynamic Translation. What is it? – a compiler (not) What does it do? – produce good fast code (hopefully) When does it run? – Before the code is executed. i.e ahead of time.

Related searches for Ahead of Time Dynamic Translation

Download Presentation

Ahead of Time Dynamic Translation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Ahead of time dynamic translation l.jpg

Ahead of Time Dynamic Translation

PreJit/NGEN by any other name

George Bosworth

Microsoft

MRE’04

March 21, 2004


Dynamic translation l.jpg

Dynamic Translation

  • What is it? – a compiler (not)

  • What does it do? – produce good fast code (hopefully)

  • When does it run? – Before the code is executed. i.e ahead of time.

    How much before is the subject of this talk


Outline l.jpg

Outline

  • Code Generation and execution in the Common Language Runtime (CLR)

  • PreJit

    • What is it, why do it, at what cost

    • What is it really, and what is hard vs easy

  • Simple example

    • What the resultant code could look like


Common language runtime l.jpg

Common Language Runtime

Frameworks

Common Language Runtime

Base Classes

ExecutionSupport

Security

IL to native code compilers

GC, stack walk, code manager

Class loader and layout


Runtime control flow just in time l.jpg

Runtime Control FlowJust in Time

Assembly

ClassLoader

First reference to type

IL to nativecode compiler

ExecutionSupport

ManagedNativeCode

First call to method

CodeManagers

SecuritySystem

CPU


Execution model l.jpg

Execution Model

VB

VC

...

Script

NativeCode

PreJit timeCode Gen

IL

Common Language Runtime

“Econo”-JITCompiler

Standard JITCompiler

NativeCode


Compiling il to native l.jpg

Compiling IL To Native

  • “Econo” JIT

    • Generates unoptimized native code

    • Code can be discarded and regenerated

  • “Standard” JIT

    • Generates optimized native code

    • Includes verification of IL code

  • PreJit code generation

    • Done at install time

    • Reduces start-up time

    • Native code has version checks and reverts to runtime JIT if they fail


Assemblies l.jpg

Assemblies

  • Unit of deployment

    • One or more files, independent of packaging

    • Self-describing via metadata (“manifest”)

  • Versioning

    • Captured by compiler

    • Policy per-application as well as per-machine

  • Security boundary

    • Assemblies are granted permissions

    • Methods can demand proof that a permission has been granted to entire call chain

  • Mediate type import and export

    • Types named relative to assembly


What happens at compile time l.jpg

What Happens at Compile Time?

  • Compiler reads any imported metadata and builds internal symbol table

  • Compiler resolves method overloading using language-specific type matching rules

    • Compiler adds any required coercions or casts

    • Compiler selects and records a precise member reference

  • Compiler provides object layout information through its choice of

    • Layout technique

    • Non-static fields

    • Virtual methods, with “new slot” vs “use existing slot”

  • Compiler emits metadata by merging incoming metadata with metadata it has generated

  • Compiler emits IL with metadata tokens


What happens at class load time i l.jpg

What Happens at Class Load Time?, I

  • A type reference occurs and is resolved either to an existing type or to an assembly reference.

  • The assembly is loaded and some validation checks occur.

  • The assembly is asked to resolve the type reference and a module is located

    • It is at this point that the security system runs inheritance permission checks


What happens at class load time ii l.jpg

What Happens at Class Load Time?, II

  • If no managed native code is available, the IL module is loaded and preliminary validation checks occur.

  • The particular class is then validated and the CLR creates in-memory data structures representing the class, including thunks for use when a method is called the first time.

  • When a method on the class is called the first time, the code is JITted (and verified if needed).


What happens at jit time l.jpg

What Happens at JIT Time?

  • If a member reference appears in the code, the type is first loaded (see below), which will also load classes from which it inherits.

  • If the type does not itself have the member, the inheritance chain is searched to see if it can be resolved higher up in the class hierarchy.

  • The member access check, including a possible security check, occurs at this time.

  • The JITter may insert calls to security code for declarative security checks in addition to those specified using the imperative security mechanism.

  • The code runs.


Unmanaged traditional code l.jpg

DEVELOPMENT

DEPLOYMENT

Source code

PE header +

EH Table +

Native code

(incl. object layout)

public static void Main(String[] args )

{ String usr; FileStream f; StreamWriter w;

try {

usr=Environment.GetEnvironmentVariable("USERNAME");

f=new FileStream(“C:\\test.txt",FileMode.Create);

w=new StreamWriter(f);

w.WriteLine(usr);

w.Close();

} catch (Exception e){

Console.WriteLine("Exception:"+e.ToString());

}

}

Compiler

public static void Main(String[] args )

{ String usr; FileStream f; StreamWriter w;

try {

usr=Environment.GetEnvironmentVariable("USERNAME");

f=new FileStream(“C:\\test.txt",FileMode.Create);

w=new StreamWriter(f);

w.WriteLine(usr);

w.Close();

} catch (Exception e){

Console.WriteLine("Exception:"+e.ToString());

}

}

app. directory,

Win32 directory

Services

  • Exception

Native codeEH table

Unmanaged (Traditional) Code

EXECUTION


Managed code execution l.jpg

DEVELOPMENT

DEPLOYMENT

Source code

Assembly

PE header + MSIL + Metadata + EH Table

PEVerify

public static void Main(String[] args )

{ String usr; FileStream f; StreamWriter w;

try {

usr=Environment.GetEnvironmentVariable("USERNAME");

f=new FileStream(“C:\\test.txt",FileMode.Create);

w=new StreamWriter(f);

w.WriteLine(usr);

w.Close();

} catch (Exception e){

Console.WriteLine("Exception:"+e.ToString());

}

}

Compiler

public static void Main(String[] args )

{ String usr; FileStream f; StreamWriter w;

try {

usr=Environment.GetEnvironmentVariable("USERNAME");

f=new FileStream(“C:\\test.txt",FileMode.Create);

w=new StreamWriter(f);

w.WriteLine(usr);

w.Close();

} catch (Exception e){

Console.WriteLine("Exception:"+e.ToString());

}

}

NGEN

Evidence

EXECUTION

Host

Policy Manager

Assembly info

Module + Class list

AssemblyLoader

GAC, app. directory,

download cache

Policy

<?xml version="1.0" encoding="utf-8" ?>

<configuration>

<mscorlib>

<security>

<policy>

<PolicyLevel version="1">

<CodeGroup class="UnionCodeGroup"

version="1"

PermissionSetName="Nothing"

Name="All_Code"

Description="Code group grants no permissions and forms the root of the code group tree.">

<IMembershipCondition class="AllMembershipCondition"

version="1"/>

<CodeGroup class="UnionCodeGroup"

version="1"

PermissionSetName="FullTrust"

Permission request

Granted permissions

(class)

(assembly)

CLR Services

  • GC

  • Exception

  • Class init

  • Security

(method)

ClassLoader

JIT +verification

Vtable +Class info

Native code+ GC table

Managed Code Execution


Prejit what is it an msil to native compiler not l.jpg

PreJit: What Is It?An MSIL-to-Native Compiler (Not)

  • Pre-compile, pre-load, pre-layout

    • Validate at execution time

      • May factor in processor type, other installed assemblies, etc.

    • If invalid, use the normal execution path

  • Goal: faster start-up

    • Because of smaller start-up working set

  • Usually as part of assembly installation

    ngen.exe, ships in Visual Studio® .Net


What happens at prejit time l.jpg

What Happens at PreJit Time?

  • Record version information about any dependents

  • Classes are laid out

  • Internal runtime data structures allocated

  • Generate managed native code, verifying as needed

  • Emit a specially formed PE file (.exe, .dll) in the code cache and associate it with the input IL assembly


Ngen code creation l.jpg

DEVELOPMENT

Source code

Assembly

PE header + MSIL + Metadata + EH Table

public static void Main(String[] args )

{ String usr; FileStream f; StreamWriter w;

try {

usr=Environment.GetEnvironmentVariable("USERNAME");

f=new FileStream(“C:\\test.txt",FileMode.Create);

w=new StreamWriter(f);

w.WriteLine(usr);

w.Close();

} catch (Exception e){

Console.WriteLine("Exception:"+e.ToString());

}

}

Compiler

public static void Main(String[] args )

{ String usr; FileStream f; StreamWriter w;

try {

usr=Environment.GetEnvironmentVariable("USERNAME");

f=new FileStream(“C:\\test.txt",FileMode.Create);

w=new StreamWriter(f);

w.WriteLine(usr);

w.Close();

} catch (Exception e){

Console.WriteLine("Exception:"+e.ToString());

}

}

Evidence

EXECUTION

Host

Policy Manager

Assembly info

Module + Class list

Policy

<?xml version="1.0" encoding="utf-8" ?>

<configuration>

<mscorlib>

<security>

<policy>

<PolicyLevel version="1">

<CodeGroup class="UnionCodeGroup"

version="1"

PermissionSetName="Nothing"

Name="All_Code"

Description="Code group grants no permissions and forms the root of the code group tree.">

<IMembershipCondition class="AllMembershipCondition"

version="1"/>

<CodeGroup class="UnionCodeGroup"

version="1"

PermissionSetName="FullTrust"

Permission request

Granted permissions

(class)

(assembly)

CLR Services

  • GC

  • Exception

  • Class init

  • Security

(method)

ClassLoader

JIT +verification

Vtable +Class info

Native code+ GC table

NGen Code Creation

DEPLOYMENT

PEVerify

NGEN

AssemblyLoader

GAC, app. directory,

download cache


Ngen code execution l.jpg

Host

Policy Manager

Assembly info

Module + Class list

Prebuilt

(check

Version)

AssemblyLoader

Policy

<?xml version="1.0" encoding="utf-8" ?>

<configuration>

<mscorlib>

<security>

<policy>

<PolicyLevel version="1">

<CodeGroup class="UnionCodeGroup"

version="1"

PermissionSetName="Nothing"

Name="All_Code"

Description="Code group grants no permissions and forms the root of the code group tree.">

<IMembershipCondition class="AllMembershipCondition"

version="1"/>

<CodeGroup class="UnionCodeGroup"

version="1"

PermissionSetName="FullTrust"

Permission request

Precomputedpermissions

(check)

Granted permissions

(class)

(assembly)

CLR Services

  • GC

  • Exception

  • Class init

  • Security

(method)

ClassLoader

Vtable +Class info

Prebuilt

Vtable +Class info

Native code+ GC table

NGen Code Execution

DEPLOYMENT

EXECUTION

GAC, app. directory,

download cache

JIT +verification

Fixups /

Relocs

(or JIT)


Deployment model l.jpg

Deployment Model

  • Shipping and installation

    • Compile IL in build lab

    • Ship IL

    • PreJit during installation (optional)

    • Possible JIT during execution

  • Patching

    • Deliver new IL image

    • Repair native images (all depending assemblies)


Why add complexity of prejit l.jpg

Why Add Complexity of PreJit?

  • Performance

    • Remove expensive post-deployment work from app startup

      • Signature validation

      • Cross-component validity checks

      • Type safety verification

      • Optimization (distribution -> execution)

        • Code generation

        • Data structures

        • Packing

    • Disk-based images allow sharing data/code across processes and across machine runs


Performance gains l.jpg

Performance Gains

  • Warm startup time

    • Up to 6x faster

  • Private pages

    • Reduced by as much as 2/3

  • Startup working set

    • Reduced by as much as 1/2

  • Basis

    • Whidbey (the upcoming release of the CLR)

    • Reference WinForm app with 100s of controls

      Caveats: your mileage may very and these are very preliminary based on pre-release builds and hence subject to change


Costs l.jpg

Costs

  • Disk space

    • Typical prejit image is ~2-2.5x IL size

    • Stored in Native Image Cache (alongside GAC)

  • Installation time

    • Typically ~3s per MB of IL

    • 40s to prejit Framework assemblies

  • Steady state performance

    • Slowdown due to indirections in code

    • ASP.net

      • V1: PreJit 7-12% slower

      • Whidbey: PreJit 2-5% slower

    • Bytemark

      • Whidbey: same performance

    • Future: PreJit will have better performance

  • Recompilation


Scenarios l.jpg

Scenarios

  • Critical: libraries

    • Loaded into multiple processes

      • Warm startup is the norm

      • Maximize shared pages

    • Used by OS like components

  • Important: large apps

    • Large percentage of code is app code

    • Large impact on machine

    • Benefit is primarily startup time

  • Important: shared apps

    • Terminal server

    • Shared pages are key


Scenarios24 l.jpg

Scenarios

  • Questionable: small single instance apps and addins

    • ~1MB working set improvement

    • ~15% startup improvement

    • Relatively small amount of compilation overhead

    • Likely to be numerous

    • Recommendation: do not ngen unless heavily used

  • Not helpful: transient code

    • Reflection emit

    • “Run and (probably) throw away” code

    • Frequently updated apps (relative to execution)

    • Code under development (edit-compile-debug)

  • Assumption: some processes will be jitting some assemblies


Prejit architectural background l.jpg

PreJit Architectural Background

  • IL was designed as CLR deployment format

    • Verifiable type safety

    • Small

    • Self describing

    • Portable

  • PreJit image was designed as internal format

    • Designed for execution speed

      • Working set rather than disk size

      • Data structures paged off of disk & used directly

    • Highly dependent on CLR implementation

      • Data structure snapshots

      • Complex codegen requirements

      • High rate of churn over time

    • Specialized for particular scenario


Specialization and recompilation l.jpg

Specialization and Recompilation

  • PreJit images are specialized translations of an IL assembly for a particular environment

    • Processor

    • Version of CLR

    • Versions of assembly dependencies

    • Development scenario (debug, profile, IBC instrument)

    • Other factors

      • Processor features

      • Security policy

      • Hosting hooks (SQL, IE,???)

      • AppDomain shareable (state access)

      • Profiling data

  • Different environments require different prejit images

    • Different machines may have different environments

    • Servicing can change CLR or assembly dependencies

    • Environmental changes can invalidate prejit images

    • App-level configuration can change requirements across apps

  • Recompilation isneeded to adjust available prejit images over time to adapt to a changing environment


Client code generation motivation l.jpg

Client code generation motivation

  • IL Distribution

    • Verifiable (limited trust)

    • Small

    • Self describing

  • Specialized native code

    • Reach

      • Processor

      • CLR Implementation

    • CLR Version

    • Platform (assemblies) Version

  • Recompilation

Brittleness okay

Rebuilt as needed


Why is specialization important l.jpg

Why is specialization important?

  • Innovation

    • Managed APIs and implementation

    • CLR implementation

  • Compatibility

    • Application needs to deploy on multiple OS’s

    • OS update must retain application compatibility

  • Plus performance (of course)


Framework evolution recompilation l.jpg

Framework evolution & recompilation

  • We want to be able to recompile application native code as frameworks (platform assemblies) evolve

  • Flexible compatible update rules

    • Add public stuff

    • Change any non-public stuff

    • Rules based on semantics – no implementation driven constraints

      • Rearrange and add fields to public structures & classes

      • Rearrange and add virtual methods

      • Insert into inheritance heirarchy

    • Non-compatible changes are possible and handled relatively gracefully

  • Versionability is a fundamental assumption when designing managed APIs

    • Implementation inheritance

      • Each layer up the hierarchy may add fields, methods, virtuals

      • All are rolled into single flat structure

      • All can version independently from each other and from client

    • Typesafe extensibility

      • Public fields on a structure or class

      • Set can be extended as needed

      • Example: delegates on form, properties on window

    • Fine grained, version-resilient component interaction

      • Properties

      • Can protect access and future evolution with a function call but still get efficiency

    • Raises the bar for API quality

      • Uniform mechanisms with language support

      • Patterns can be enforced with tools


Clr evolution recompilation l.jpg

CLR evolution & recompilation

  • We want to be able to recompile native code as CLR evolves

  • Much of runtime implementation is “baked” into code

    • Code sequences

      • Helper calls, Dispatch sequences, Global variable access

    • Runtime data structures

      • Highly specialized, Slow to build from IL, Likely to change over time

    • Invariants

      • Garbage collection, Lazy initialization

  • Recent Changes

    • Interface/Virtual dispatch

    • Working set tuning

    • Single file loading

    • Generics

    • And on and on…

  • Future potential

    • More data structure tuning

    • Phoenix

      The CLR is a great vehicle for delivering improvements into Windows


Future directions l.jpg

Future Directions

  • Phoenix

    • Changes prejit equation

      • Higher compile cost

      • Better perf for prejit vs JIT

    • More cross-assembly analysis

  • Client compilation optimizations

    • Profile data feedback

    • Active base address management

  • Enable some additional specialization conditionally

    • Allow developer to dial conservative behavior vs. aggressive optimizations


Why it s not so easy let me count the ways l.jpg

Why it’s Not So Easylet me count the ways


Issue compilation overhead l.jpg

Issue: Compilation Overhead

  • Installation cost

  • Recompilation cost

    • Cascaded low level changes have very high cost

  • Image churn can have secondary costs (OS optimizations, virus scan)

  • Goals:

    • Ensure compilation performance is well tuned

    • Minimize recompilation

      • Minimize factors which cause recompilation

      • Allow limited assembly servicing without cascading recompilation

        • Ensure that most low-level servicing falls in this limited category

    • Be smart about deferring compilation work

    • However, some compilation cost is fundamental


Issue managed watson dumps l.jpg

Issue: Managed Watson Dumps

  • Native code and CLR data structures are generated on user machine, not available in any shipped dlls

  • Much of this data is required to do a stack trace

  • IL<->Native mapping info is typically not available unless debug code is in use

    • Source mapping is described in terms of IL

  • Goals:

    • Leverage repeatability of codegen to optimize ngen image contents out of process dump

      • Result is faster dump collection and smaller dumps

    • Also leverage repeatability to regenerate IL<->native mapping info post mortem


Cross assembly optimizations l.jpg

Cross-assembly optimizations

  • These optimizations expose internal implementation details to other assemblies’ code generation

  • Examples

    • Method inlining

    • Constant value inlining

    • Persisted instance data created at compile time

    • Class constructors

    • Struct enregistration

  • Likely to be much more in Phoenix


Ngen fixup blobs l.jpg

NGen Fixup Blobs

  • Descriptive reference to something in another module

  • Blobs are grouped by referenced module?

    • Module level table describes ranges of blobs

    • Each table entry has an assembly ref token + file ref token

  • Could roughly follow metadata signature byte encoding?

    • Byte sized tags with fixed following values

    • Variable length encoded numbers & tokens

  • Treat blob is a type signature?

    • However, scope is in destination assembly

    • This avoids symbolic lookups, however introduces fragility if token numbers change

  • Other possible blob formats

    • MODULE (no args)

    • TYPE + type signature

    • METHOD + <vtable index> + containing type signature

    • FIELD + <field index> + containing type signature

    • STRING + <string token>

    • SIGNATURE + <signature token>

  • Blobs are read-only? & interned within a module?

  • Typically described with an RVA?

Details

Details

Details


Ngen fixup tables l.jpg

NGen Fixup Tables

  • Set of tables which provide fixup cells for code and data to reference

  • Header describes ranges of tables?

  • Table entry initially contains tagged RVA of fixup blob?

  • Table index implies type of fixup

    • Handle (runtime data structure)

    • Method call address

    • Static field address

    • Varargs signature

    • Others?

    • Side effects? Which ones? (e.g. type initializations?)

  • Entries are fixed up lazily?

    • All fixups are idempotent?, tag allows atomic update

    • Eager update is too expensive?

    • Fixup can fail?

And they never end


But when it works l.jpg

But when it works

it can work well

An example

(take with a grain of salt)


Example l.jpg

Example

Assembly B (b.cs)

public class InheritFromA : A

{

    static public int a1()

    {     ...    }

}

public class B

{

    public static void b()

    {

        A a = new A();

        A.a();

        InheritFromA.a1();

    }

}

Assembly A (a.cs)

public class A

{   

    static public int a()

    {    ...    }

}


Example naive l.jpg

Example: Naive

b() has 4 fixups & requires fixup guarantee before execution

{

A a = new A();

20002552 mov     ecx,[20006180]       // Fixup on A's method table pointer

20002558 call    [email protected] // Call to new

2000255d mov     ecx,eax

2000255f mov     eax,[20006190]       // Fixup on A.ctor

20002565 call    dword ptr [eax]      // Call to A.ctor

A.a();

20002567 mov     eax,[20006194]       // fixup on A.a

2000256d call    dword ptr [eax]      // Call to A.a

InheritFromA.a1();

2000256f call    dword ptr [20005cb8] // Indirect call to InheritFromA.a1

// (InheritFromA may need one time init)

}

20002575 ret    


Example in the limit l.jpg

Example: In the Limit

b() has no fixups and can be called directly

{

A a = new A();

2000254a mov     ecx,0x6ab95b98      // Note no fixup on A's method table pointer

2000254f call    [email protected]    // call to new

20002554 mov     ecx,eax

20002556 call    a!A..ctor()         // Note direct call to A.ctor

A.a();

2000255b call    a!A.a()             // Note direct call to A.a

InheritFromA.a1();

20002560 call    b!InheritFromA.a1() // Note direct call to // InheritFromA.a1  (InheritFromA

// is pre-inited)

}

20002565 ret


In conclusion l.jpg

In Conclusion

Ahead of Time Dynamic Translation (PreJit)

  • What is it? – a compiler (not)

  • What does it do? – produce good fast code (hopefully)

  • When does it run? – Before the code is executed. i.e ahead of time, (just a bit earlier than a JIT).


Bottom line l.jpg

Bottom Line

All that is left is compiler style optimizations

Prejit makes sense in some scenarios, but not all


  • Login