slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
WEAG – THALES – JP 11.20 PowerPoint Presentation
Download Presentation
WEAG – THALES – JP 11.20

Loading in 2 Seconds...

play fullscreen
1 / 74

WEAG – THALES – JP 11.20 - PowerPoint PPT Presentation


  • 279 Views
  • Uploaded on

WEAG – THALES – JP 11.20. REVVA. Presentation Outline. Project facts ; the consortium Background and rationale The REVVA concepts The REVVA generic process The REVVA products Techniques and support REVVA technical achievements REVVA future.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'WEAG – THALES – JP 11.20' - Thomas


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

WEAG – THALES – JP 11.20

REVVA

WE 0201 – PKIT2 Date : November2004 Slide : 1 / 74

slide2

Presentation Outline

Project facts ; the consortium

Background and rationale

The REVVA concepts

The REVVA generic process

The REVVA products

Techniques and support

REVVA technical achievements

REVVA future

WE 0201 – PKIT2 Date : November2004 Slide : 2 / 74

slide3

Common Validation, Verification and Accreditation Framework for Simulation (REVVA)

Project Facts

Purpose:

The THALES JP 11.20 is a Western European Union / WEAG research program funded by 5 nations: France (lead nation), Denmark, Italy, the Netherlands, Sweden.

Objectives:

To develop a basis for a common methodological framework for the Verification, Validation and Accreditation (VV&A) of data, models and simulations. To be a starting point for a draft standard defining VV&A taxonomy, process and methodology.

Project timeline:

March 2003 – September 2004

WE 0201 – PKIT2 Date : November2004 Slide : 3 / 74

slide4

The consortium

The THALES JP 11.20 is conducted by a multi-national consortium led by ONERA (France).

FOI – FMV

(SW)

UNI-C (DK)

TNO (NL)

DATAMAT (IT)

ONERA – ALEKTO

(FR)

WE 0201 – PKIT2 Date : November2004 Slide : 4 / 74

slide5

REVVA background and rationale

WE 0201 – PKIT2 Date : November2004 Slide : 5 / 74

slide6

DMSO

ITOP

Exchange

SISO

Capitalization

Information

JP 11.20

REVVA

Capitalization

NATO

Exchange

AFDRG

RTP 11.13

REVVA Background (1)

WE 0201 – PKIT2 Date : November2004 Slide : 6 / 74

slide7

REVVA Background (2)

  • VV&A state-of-the-art:
    • DMSO, SISO, NATO, ITOP.
  • M&S scientific contributions:
    • Systems Theory (B. Zeigler)
  • Contributions on the concept of validation coming from the epistemology of simulation

WE 0201 – PKIT2 Date : November2004 Slide : 7 / 74

slide8

VV&A rationale (1)

Doing (only) what makes sense for customers and users in their search for trust.

  • Trust:
    • identification of trust components,
    • support (based on project knowledge, consolidation by shared domain knowledge)

WE 0201 – PKIT2 Date : November2004 Slide : 8 / 74

slide9

VV&A

Informal

Formal

VV&A rationale (2)

M&S supplier skillsand reputation

Audit trail of experiments and uses

Formal analysis ofproducts, processes andorganisation

Trust

WE 0201 – PKIT2 Date : November2004 Slide : 9 / 74

slide10

Needs

Purpose

Organisation

Products

Processes

VV&A rationale (3)

Validation

Verification

FormalAnalysis

Acceptance

WE 0201 – PKIT2 Date : November2004 Slide : 10 / 74

slide11

VV&A rationale (4)

Activities driven and prioritised by:

  • Requirements (acceptance requirements)
  • Identification / perception of risks
  • Organisational needs, risks and benefits
  • Costs

WE 0201 – PKIT2 Date : November2004 Slide : 11 / 74

slide12

REVVA Concepts

WE 0201 – PKIT2 Date : November2004 Slide : 12 / 74

slide13

REVVA Terminology (1)

  • Decomposition of VV&A in Verification, Validation and Acceptance; (temporal) independence of each concept.
    • Verification: Model Verification.
    • Validation: Model Validation within an Experimental Frame (EF)
    • Acceptance: Before use / After use

Focus on Acceptance and Validation

WE 0201 – PKIT2 Date : November2004 Slide : 13 / 74

slide14

REVVA Terminology (2)

  • Correctness:
    • The property of a simulation model to comply with formal rules and bodies of reference information for its representation and for the transformation of its representation into another one.
  • Verification:
    • The process which is used to construct, under a set of time, cost, skills, and organisational constraints a justified belief about model correctness.

WE 0201 – PKIT2 Date : November2004 Slide : 14 / 74

slide15

REVVA Terminology (3)

  • Validity:
    • The property of a simulation model to have, within a specific experimental frame, a behaviour which is indistinguishable under a set of validation criteria from the behaviour of the System of Interest.
  • Validation:
    • The process which is used to construct, under a set of time, cost, skills, and organisational constraints a justified belief about model validity.

WE 0201 – PKIT2 Date : November2004 Slide : 15 / 74

slide16

REVVA Terminology (4)

  • Acceptance:
    • The informed decision of a M&S customer to use the results of a simulation for a specific purpose.
    • The informed decision of a M&S customer to use a simulation, within an experimental frame, for a specific purpose

WE 0201 – PKIT2 Date : November2004 Slide : 16 / 74

slide17

REVVA Assumptions

  • Acceptance (and Validation) should:
    • be purpose oriented,
    • be development neutral,
    • be technology neutral,
    • recognise the impact of organisations,
    • recognise the impact of timeframe of M&S and VV&A,
    • rely on data and knowledge management.

WE 0201 – PKIT2 Date : November2004 Slide : 17 / 74

slide18

A comprehensive view

The

"M&S Product World"

The

"Real World"

The

"Problem World"

The

"M&S World"

System

Requirements

M&S

Questions

Objectives

Needs

Problem

Formalisation

Decisions

Actions

M&S Enabling Products

Problem solution

M&S Answers

WE 0201 – PKIT2 Date : November2004 Slide : 18 / 74

slide19

Methodological hints

  • Independence
  • Incremental nature of activities and results
  • Focus on the intended purpose
  • Objective assessment (Acceptance Criteria)
  • Careful construction of a justified belief
  • Tailoring

WE 0201 – PKIT2 Date : November2004 Slide : 19 / 74

slide20

EF

M

Tailoring

Question

Answer

The EF used for Acceptance (and for Validation) is derived from the Acceptance Requirements

  • All the activities are planed, designed and controlled by them.
  • The Acceptance Requirements are not (necessarily) the M&S
  • Requirements ; they are specific of each use and purpose

WE 0201 – PKIT2 Date : November2004 Slide : 20 / 74

slide21

Before M&S use1

AfterM&S use1

BeforeM&S use2

|

|

|

t

t1

t2

t3

Validation t1

Validation t2

Validation t3

Impact of time

  • Verification is (mostly/hopefully) a one-shot process
  • Validation is a permanent/recurrent issue
  • Acceptance and Validation are purpose oriented and tailored

WE 0201 – PKIT2 Date : November2004 Slide : 21 / 74

slide22

The REVVA Generic Process

WE 0201 – PKIT2 Date : November2004 Slide : 22 / 74

slide23

Intended purpose

Acceptance Recommendation

Develop

ToA

Evaluate V&V Report

The REVVA Generic Process

ToA

Acquire Information

Product flow

Model information and system knowledge

V&V report

Back step

Develop ToVV

Assess Evidence Integration

Phase

Assessed items of evidence

Product

ToVV

ToA: Target of Acceptance

Assess Evidence

Conduct V&V

ToVV: Target of Verification

and Validation

V&V Items

WE 0201 – PKIT2 Date : November2004 Slide : 23 / 74

slide24

Phase Overlaps

Develop ToA

Acquire Information

Develop ToVV

Conduct V&V

Assess Items of Evidence

Assess Evidence Integration

Evaluate V&V Report

Main effort

VV&A Endeavour Duration

WE 0201 – PKIT2 Date : November2004 Slide : 24 / 74

slide25

Develop ToA

  • Performed by the Acceptance Agent in close cooperation with the user
  • Develop Acceptability Criteria (AC), including Validity Criteria (VC)
  • Document rationale for development

WE 0201 – PKIT2 Date : November2004 Slide : 25 / 74

slide26

Develop ToA (2)

Intended Purpose

Objectives and Sub-Objectives

(”Questions”)

(Fictive) Experimental Frames

Precision Requirements for the

Behavior Characteristics

WE 0201 – PKIT2 Date : November2004 Slide : 26 / 74

slide27

Intended Purpose

Sub-Objective 1

Sub-Objective 2

Sub-Objective n

ToA

SO1.1

SO1.1

SO1.1

AC2

AC3

Acceptability Criterion m

AC1

WE 0201 – PKIT2 Date : November2004 Slide : 27 / 74

slide28

Acquire Information

  • Led by the V&V Agent, performed by the V&V Team
  • Get overview over the data, information, and knowledge (likely to become) available
    • about the Simulation Executable Model
    • about the System of Interest
  • Create data, information, and knowledge repository
  • Identify additional sources

WE 0201 – PKIT2 Date : November2004 Slide : 28 / 74

slide29

Develop ToVV

  • Performed by the V&V Agent, supported by the V&V Team
  • Develop an approach to demonstrate that the Validation Criteria are passed or failed, with the data, information, and knowledge available
  • Identify data, information, and knowledge desirable for more efficient V&V

WE 0201 – PKIT2 Date : November2004 Slide : 29 / 74

slide30

Intended Purpose

Sub-Objective 1

Sub-Objective 2

Sub-Objective n

ToA

SO1.1

SO1.1

SO1.1

AC2

AC3

Acceptability Criterion m

AC1

Sub-Criterion m.1

SC m.1.1

SC m.1.2

SC m.1.3

Task

ToVV

Task

Task

WE 0201 – PKIT2 Date : November2004 Slide : 30 / 74

slide31

Conduct V&V

  • Led by the V&V Agent, performed by the V&V Team
  • Choose techniques and tools
  • Run the technique, e.g.,
    • Use 2-test to measure deviation
    • Use SME to assess structural decomposition
  • Record technique, referrenced information, and result as V&V item
  • Modify ToVV, if creation of V&V item fails

WE 0201 – PKIT2 Date : November2004 Slide : 31 / 74

slide32

Assess Evidence

  • Performed by the V&V Agent
  • Review documented V&V items
  • Assess the possibility of an erroneous result
  • Accept V&V item as assessed item of evidence, or reject it
  • Direct feedback loop

WE 0201 – PKIT2 Date : November2004 Slide : 32 / 74

slide33

Assess Evidence Integration

  • Performed by the Acceptance Agent
  • Assemble items of evidence according to most recent ToVV
  • Derive which VC are passed or failed
    • Intuitively, subjectivly
    • Based on methods for dealing with uncertainty
  • Assess possibility of misperception

WE 0201 – PKIT2 Date : November2004 Slide : 33 / 74

slide34

Intended Purpose

Sub-Objective 1

Sub-Objective 2

Sub-Objective n

ToA

SO1.1

SO1.1

SO1.1

AC2

AC3

Acceptability Criterion m

AC1

Sub-Criterion m.1

ToVV

SC m.1.1

SC m.1.2

SC m.1.3

Task

Task

Task

Item of

Evidence

IoE

IoE

WE 0201 – PKIT2 Date : November2004 Slide : 34 / 74

slide35

Evaluate V&V Report

  • Performed by the Acceptance Agent
  • Review expressiveness and completeness of AC with newly gained knowledge
  • Weigh impact of misperception according to possibility of misperception
  • Make acceptance recommendation

WE 0201 – PKIT2 Date : November2004 Slide : 35 / 74

slide36

The REVVA Products

WE 0201 – PKIT2 Date : November2004 Slide : 36 / 74

slide37

Intended purpose

Acceptance Recommendation

Develop

ToA

Evaluate V&V Report

The REVVA Generic Process

ToA

Acquire Information

Model information and system knowledge

V&V report

Product flow

Back step

Develop ToVV

Assess Evidence Integration

Phase

Assessed items of evidence

ToVV

Product

Assess Evidence

Conduct V&V

V&V Items

WE 0201 – PKIT2 Date : November2004 Slide : 37 / 74

slide38

Target of Acceptance (ToA)

Intended purpose

  • Precise specification of the Acceptability Criteria (coming from validity related functional requirements)
  • Stakeholder oriented
  • “What exactly needs to be assessed?”
  • Important elements in the ToA nodes: requirement, MoE, constraint, level of impact, decomposition argument

Develop

ToA

ToA

Acquire Information

Develop ToVV

WE 0201 – PKIT2 Date : November2004 Slide : 38 / 74

slide39

ToA (2)

  • Building
    • Getting the requirements
    • Deriving MoE and constraints
    • Determining the level of impact
    • Validation of the ToA
    • Reuse
  • Using
    • Part of contract
    • Basis for assessment report
  • Maintaining
    • Due to changes in science, stakeholder changing mind, corrections in ToA found during V&V work, V&V findings may indicate more limited validity range

WE 0201 – PKIT2 Date : November2004 Slide : 39 / 74

slide40

Model information and system knowledge

  • Information used to determine the V&V strategy for demonstrating the model’s correctness and validity
  • Identifies all sources of information and knowledge
  • Important elements:
    • design documents, code, modeller’s perception of the real world, behaviour data (of model and sub-models), conceptual model.
    • Real world data (referent): data on (related or similar) systems,expert opinions

Acquire Information

Model information and system knowledge

Develop ToVV

Assess Evidence Integration

Conduct V&V

WE 0201 – PKIT2 Date : November2004 Slide : 40 / 74

slide41

Target of Verification and Validation (ToVV)

Develop ToVV

Assess Evidence Integration

  • V&V items:

evidence required to substantiate the ToA Acceptability Criteria

  • When necessary:

further decomposition of criteria (extension of ToA)

ToVV

Conduct V&V

  • Important additional elements in the ToVV:

method to use for collecting evidence, required rigor, cost estimate, aggregation instructions

  • Tasks

WE 0201 – PKIT2 Date : November2004 Slide : 41 / 74

slide42

V&V Items

  • A V&V item consists of:
    • relevant information,
    • the evaluation objective,
    • reference information,
    • an evaluation technique,
    • evaluation result.
  • Must be reproducible

Assess Evidence

Conduct V&V

V&V Items

WE 0201 – PKIT2 Date : November2004 Slide : 42 / 74

slide43

Assessed items of evidence

  • Assessed items of evidence:
    • V&V agent assessment of the V&V result,
    • the acceptance or rejection of each V&V item,
    • probative force (depending on method or technique and reference information)

Assess Evidence Integration

Assessed items of evidence

Assess Evidence

WE 0201 – PKIT2 Date : November2004 Slide : 43 / 74

slide44

V&V report

  • The assessed items of evidence assembled and integrated based on the ToVV
  • Links the assessed items of evidence to the Acceptability Criteria
  • Indicates if
    • evidence is considered sufficient to pass an acceptability criterion
    • V&V items for a criterion are missing or too weak,
    • disproving evidence was found

Evaluate V&V Report

V&V report

Assess Evidence Integration

WE 0201 – PKIT2 Date : November2004 Slide : 44 / 74

slide45

Acceptance Recommendation

  • The final recommendation on acceptance
  • Based on the ToA
  • Residual uncertainty after V&V
    • from probative forces,
    • use level of impact to determine recommendation
  • The recommendation states (with consequences for M&S product’s utility):
    • if required evidence is missing or too weak,
    • if a weakness in the ToA was detected

Acceptance Recommendation

Evaluate V&V Report

WE 0201 – PKIT2 Date : November2004 Slide : 45 / 74

slide46

A VV&A project management view

The "3 pillars" concept

Purpose

Organisation

Products

Objectives

Process

WE 0201 – PKIT2 Date : November2004 Slide : 46 / 74

slide47

The M&S and VV&A roles

  • M&S stakeholders
  • M&S suppliers
  • VV&A agents
  • Experts

WE 0201 – PKIT2 Date : November2004 Slide : 47 / 74

slide48

Levels of independence

  • DV&V:
    • dependent V&V; V&V performed by suppliers and accepted as it is
  • IA:
    • independent assessment; V&V performed by supplier and assessed by the V&V agent
  • IV&V:
    • independentV&V; V&V performed under the control of the V&V agent

WE 0201 – PKIT2 Date : November2004 Slide : 48 / 74

slide49

Feasibility assumptions

  • High-level organisational decision
  • Standards of V&V documentation
  • Incremental accumulation of knowledge:
    • project vs corporate knowledge management
  • Process for knowledge revision

WE 0201 – PKIT2 Date : November2004 Slide : 49 / 74

slide50

The REVVA Techniques

and support

WE 0201 – PKIT2 Date : November2004 Slide : 50 / 74

slide51

REVVA point of view

  • Identification of the most important techniques and support
  • Techniques to support decision making, …
  • Next direction: develop techniques to support the whole process

WE 0201 – PKIT2 Date : November2004 Slide : 51 / 74

slide52

Most important techniques and support

  • Most importance of Requirements Engineering
    • Develop ToA & ToVV
  • Design of Experiments (DOE)
    • Conduct V&V: calibration, sensitivity analysis & validation
  • Computation of probative force and convincing force
    • Assess evidence
    • Assess argument
  • Uncertainty analysis
    • Assess evidence
    • Evaluate V&V Report
  • Decision tools
    • Evaluate V&V Report

WE 0201 – PKIT2 Date : November2004 Slide : 52 / 74

slide53

Claim-Argument-Evidence

Types of argument

Evidence

Claim

Evidence

Evidence

Argument

  • Deterministic – relying upon axioms, logic and proof
  • Probabilistic – relying upon probabilities, statistical analysis
  • Qualitative – relying upon adherence to standards, design codes

WE 0201 – PKIT2 Date : November2004 Slide : 53 / 74

slide54

Decision Tools

  • The model is fit for purpose
  • The model is not fit for purpose

WE 0201 – PKIT2 Date : November2004 Slide : 54 / 74

slide55

Uncertainty Analysis

  • Integrated approach for uncertainty estimation based on
    • Behaviour Characteristics
    • Propositional Logic
    • Hierarchical Breakdown Structures
    • Possibility Theory
    • Basic Model of Decision Theory
  • Increases quantitative share in decision making
  • Leaves ”subjectivity gaps” with
    • Probative Forces
    • Impressions of confirmation and denial

WE 0201 – PKIT2 Date : November2004 Slide : 55 / 74

slide56

Composition and Inference

+

: inference relation (p1’  p1 : p1’ is a sufficient condition, but not a necessary one)

p1

+

(--)

: composition relation (p1’ is a conjunction

of p1.1 through p1.3 )

p1’: p1.1 p1.2  p1.3

(--)

: quantification of denial impression (from (-) to (----))

(if p1’ is not verified, how strongly would it mean that p1 is not verified?)

p1.1

p1.2

p1.3

WE 0201 – PKIT2 Date : November2004 Slide : 56 / 74

slide57

Completeness and consistency checking

  • The dependencies and consistency between the sponsor needs and requirements, any structured problem description, the model (conceptual, formal or executable), the ToA and ToVV and the final simulation results are checked
  • Contents (evidences, specifications, etc.) without requirements and requirements without contents need to be resolved

WE 0201 – PKIT2 Date : November2004 Slide : 57 / 74

slide58

REVVA technical achievements

WE 0201 – PKIT2 Date : November2004 Slide : 58 / 74

slide59

REVVA Project Breakdown

The project includes six technical Work Packages:

WP 1 : Problem Analysis

WP 2 : Definition of AccreditationTarget

WP 3 : VV&A Techniques

WP 4 : Optimisation of Testing

WP 5 : Outline for a VV&A Methodological Framework

WP 6 : Evaluation of Results

WE 0201 – PKIT2 Date : November2004 Slide : 59 / 74

slide60

Technical achievements (1)

  • Taxonomy: Initial contribution dealing with V&V state of the art
  • VV&A problem statement
  • Review of acceptance criteria and levels of confidence
  • Process models

WE 0201 – PKIT2 Date : November2004 Slide : 60 / 74

slide61

Technical achievements (2)

  • Emphasis on AcceptanceAgent role: Target of Acceptance (ToA) and assessment of ToA satisfaction
  • Integration of qualitative and quantitative confidence in a logical framework of argumentation
  • Guidance for ToA and ToVV definition

WE 0201 – PKIT2 Date : November2004 Slide : 61 / 74

slide62

Technical achievements (3)

  • Review of existing techniques and tools resources
  • V&V techniques state-of-the-art
  • User guidance

WE 0201 – PKIT2 Date : November2004 Slide : 62 / 74

slide63

Technical achievements (4)

  • Capitalisation of data and knowledge
  • Experience optimisation
  • Relation with field trial data
  • Identification of data/knowledge management issues

WE 0201 – PKIT2 Date : November2004 Slide : 63 / 74

slide64

Technical achievements (5)

  • Deep understanding of the terminology
  • Definition of the VV&A scope
  • Identification of VV&A organisations depending on level of impact
  • Definition of VV&A products and processes

WE 0201 – PKIT2 Date : November2004 Slide : 64 / 74

slide65

REVVA Outcomes

  • Generic VV&A process, described by:
    • Methodology Reference manual (METHGU)
    • User’s Manual (PROSPEC)
  • Illustration by case studies
  • Draft contributions to guidance
  • REVVA Website (www.JP1120-REVVA.com)

WE 0201 – PKIT2 Date : November2004 Slide : 65 / 74

slide66

Lessons learnt at methodological level

  • Compliant with the mainstream idea that every euro spent in requirements generates high Returns on Investment (ROI)
  • The cost driver is the criticality of the use, not the quality of the process
  • Generic, versatile methodology: impact on the easiness of tailoring
  • Ability to comply with formal supports for guidance or automation of the process

WE 0201 – PKIT2 Date : November2004 Slide : 66 / 74

slide67

Lessons learnt from the case studies

The REVVA application on Case Studies has

shown capital points:

  • To answer the right question:
    • It is important to formalise at early steps the problem to be solved by M&S use AND the question asked to VV&A.
  • To answer the question right:
    • It is enabled by the derivation of the ToVV from the ToA.
  • To construct a justified belief:
    • The confidence is substantiated by gathering and assessing the items of evidence.

WE 0201 – PKIT2 Date : November2004 Slide : 67 / 74

slide68

Practical experience on case studies

  • REVVA method helps to balance between purpose, model and data
  • Difficult to gather the right information
  • ToA seems to be particularly useful for large projects

WE 0201 – PKIT2 Date : November2004 Slide : 68 / 74

slide69

REVVA communications

Three papers have been presented in a special session during the Spring '04 SIW

  • 04S-SIW-153 : A Common Validation, Verification and Accreditation Framework for Simulations.
  • 04S-SIW-005 :T&E Based on M&S VV&A:  The Importance of Experience Capitalization.
  • 04S-SIW-152 :Assessment Criteria and V&V Levels in JP11.20.

WE 0201 – PKIT2 Date : November2004 Slide : 69 / 74

slide70

REVVA Future

WE 0201 – PKIT2 Date : November2004 Slide : 70 / 74

slide71

Follow-up

REVVA is a first methodological step;

the work will continue with

a EUROPA project (REVVA2) which is to start during Spring 05.

WE 0201 – PKIT2 Date : November2004 Slide : 71 / 74

slide72

REVVA2: Conceptual challenges

  • Organisation and roles clarification
  • Incremental knowledge management (i.e. repository …)
  • Formalisation of VV&A products
  • Formalisation of the methodology
  • Tailoring
  • Costs model and project management

WE 0201 – PKIT2 Date : November2004 Slide : 72 / 74

slide73

REVVA2: Technical challenges

  • VV&A requirements formalisation
  • V&V evidences : collection, organisation
  • V&V argumentation: formalisation and support
  • V&V and decision-making under uncertainty
  • Formal foundations for V&V tools
  • V&V tools selection and guidance (tailoring)
  • V&V actual processes support

WE 0201 – PKIT2 Date : November2004 Slide : 73 / 74

slide74

REVVA2: Current status

  • Objective: to produce draft documents for standardisation.
  • The project timeline will be 36 months, expected start in March 2005.
  • EUROPA MoU
  • FR, DK, SW and NL intend to join REVVA2.
  • The UK may consider to join after CConVV&A.
  • The WBS are refined taking into account the roadmap defined in Munich.

WE 0201 – PKIT2 Date : November2004 Slide : 74 / 74