1 / 7

Evaluation Plan

Evaluation Plan. Stuart Wheater Arjuna Technologies Ltd., UK. Deliverables. Month 6 Evaluation Plan (D15) Month 12 Updated Evaluation Plan (D16) Month 18 Draft Revised Evaluation Plan (D?) Month 24 Revised Evaluation Plan (D?)

dolf
Download Presentation

Evaluation Plan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation Plan Stuart Wheater Arjuna Technologies Ltd., UK

  2. Deliverables • Month 6 • Evaluation Plan (D15) • Month 12 • Updated Evaluation Plan (D16) • Month 18 • Draft Revised Evaluation Plan (D?) • Month 24 • Revised Evaluation Plan (D?) • “… should evolve in order to provide quantitative evaluation …”

  3. Review Feedback • “The evaluation plan, which in its current form mainly addresses lower level technical questions well need to be updated with the higher level issue, which will allow meaningful measurement of project results” • Draft Revised Evaluation Plan, M18 • Revised Evaluation Plan, M24

  4. Suggestions Functional Evaluation • “Functional issues should be addressed at a much higher level of abstraction” • What can be adapted, and when? • What are the characteristics of a composite service executed? • What about a service can be described?

  5. Suggestions Functional Evaluation (Cont.) • “Functionality should be be evaluated in terms of the initial requirements that have been set out for ADAPT.” • “Technical annex”, not “Call” • “... , by providing a proof-of-concept implementation of standards on top of ADAPT interfaces should give sufficient insight to evaluate the final system.” • ? • “In summary, with respect to functionality, a higher-level grouping of functions and accompanying evaluations criteria is needed.”

  6. Suggestions Non-Functional Evaluation • “… scientifically more interesting, is the evaluation of ADAPT with respect to its non-functional aspects, notably availability and adaptability.” • Appears not to be interested in “standard performance” • “… setting up experiments and evaluation criteria.” • Hinted at, but no detail • “I expect that experiments will be (partly) based on trace-driven simulation, for which reason it is important to know in advance exactly what will be evaluated and how”

  7. Summary • Reorganize into higher-level grouping of functions and evaluations criteria • Determine exactly what non-functional aspects will be evaluated and how

More Related