1 / 23

3A Group

3A Group. Dr Maria Athelogou. 3A group member contributions. Grace Kim:

pahana
Download Presentation

3A Group

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3A Group Dr Maria Athelogou 3A group member contributions Grace Kim: Clarification in the study design: Since we have contours of lesions from many readers (for 1A, 1B, and 1C: will be), are we planning to use this as Gold standard? This means that we DO NOT require the radiologists’ visual readings to test inter-algorithm variation.   Please confirm. This means in the study design that we will have a metric of overlap measures between a radiologist’s contour and result from an algorithm. GraceKim@mednet.ucla.edu - We have to define a metric for the overlap Answer Andy Buckler: discuss this during the next 3A call.  My feeling is that we have two types of analyses: one being against true volume (in the phantom studies such as 1A and 1C directly, and indirectly by averaging readers results for the clinical data such as 1B), and the other type being MICCAI-type metrics such as volume overlap (and others from a variety of papers such as http://books.google.com/books?id=Ot_3klqqO3cC&pg=PA977&lpg=PA977&dq=miccai+metrics+segmentation&source=bl&ots=hrJgveEFDs&sig=inZv8lbHcJ4rC9PiPtUQxIKSPSI&hl=en&ei=kzyrTcHzDITMgQfzypD0BQ&sa=X&oi=book_result&ct=result&resnum=4&ved=0CCAQ6AEwAw#v=onepage&q=miccai%20metrics%20segmentation&f=false).

  2. 3A Group Dr Maria Athelogou

  3. 3A Group Dr Maria Athelogou

  4. 3A Group Dr Maria Athelogou

  5. 3A Group Dr Maria Athelogou Keep it simple?

  6. 3A Group Dr Maria Athelogou

  7. 3A Group Dr Maria Athelogou http://lts08.bigr.nl/about.php

  8. 3A Group Dr Maria Athelogou

  9. 3A Group Dr Maria Athelogou Each algorithm needs a different computing time for the volume calculation. We need a appropriate metric for the time factor (performance).

  10. 3A Group Dr Maria Athelogou Hubert Beaumont

  11. 3A Group Dr Maria Athelogou Hubert Beaumont

  12. 3A Group Dr Maria Athelogou

  13. 3A Group Dr Maria Athelogou According to study design page 3 We need a exact Workflow for the measuring process (protocol)

  14. 3A Group Dr Maria Athelogou From last 3A call (April, 14th 2011) Regarding the definition of intra software variability, may I suggest instead to consider algorithm variability as a function of initial conditions? (e.g. From variation of the seed). It would give in the text: The aim of the study is twofold: To estimate inter-algorithm variability To estimate independently the variability of each algorithm regarding variation of initial condition (eg. the seed point) These estimations rely on the volume estimation of synthetic nodules from CT Asked: As all the evaluation is remotely processed via a batch processing, I cannot see how to interactively change contours afterward. How to check the result of a given segmentation and redraw part of the segmentation? If I have correctly understood, it would be quite tedious to process the first segmentation, download the result, check the contours, correct the segmentation, get the list of point, upload the list to the server and so on may be several time and this for a significant set of lesions. Hubert Beaumont

  15. 3A Group Dr Maria Athelogou Image Analysis algorithms and Batch Analysis Service conformity? May we give the users the possibility to download cases, if needed?Seed points could be predefined by QIBA Using predefined template for up- and download of control and result parameters. Clarify 3.3.1

  16. 3A Group Dr Maria Athelogou CTZ: Anthropomorphic phantoms have to be measured exact to guaranty connections to the SI Normals (page 2) CTZ : 2) Very clear algorithm classification page 2 3) make sure that digital phantom data are produced from physical objects instead of mathematical algorithms (page 7)

  17. 3A Group Dr Maria Athelogou David Gustafson

  18. 3A Group Dr Maria Athelogou David Gustafson

  19. 3A Group Dr Maria Athelogou David Gustafson

  20. 3A Group Dr Maria Athelogou David Gustafson

  21. 3A Group Dr Maria Athelogou David Gustafson

  22. 3A Group Dr Maria Athelogou David Gustafson

  23. 3A Group Dr Maria Athelogou McNitt-Gray, Michael A.B.: “ground truth.”  Would it also be possible to use LIDC data, where ground truth is the aggregate of the radiologist readers on this data? • If CoreLabs makes available the contours they generated from the 1B data (I understand from David Clunie they did record • them, so in theory it is possible), then 3A might be able to use these as comparisons as “truth”. • If not, then curating an evaluation set from LIDC, which would also have contours, would be possible as well. • LIDC would have a larger set of lesions to draw from (hundreds, not 32). • In both cases, first things to address would be: • Format of contours – can you read in and decipher contour formats. • LIDC used an xml format; CoreLabs presumably used some DICOM SR format (not sure) • How to deal with four (LIDC) or five (QIBA 1B) contours so that you can use as a basis for • comparison with algorithm. • Do you want to compare actual contours? (voxel by voxel comparison of boundaries using something like STAPLE?) • Or just compare outcome variable – volume, diameter, etc. • And in each case, you will have 4 or 5 potential individual comparisons; or just take the average • of the human contour results and compare algorithm to that. • McNitt-Gray, Michael

More Related