1 / 19

Optimizing Noise Reduction Algorithm T42 Implementation for Data Cleaning

This webpage details the implementation of the T42 algorithm for noise reduction in data processing. The algorithm involves selecting high signal cells and retaining their significant neighbors based on specific thresholds. The cal_t42 package is utilized for organizing the data, with classes like Calt42 and Calt42Reco managing the cleaning process efficiently. The Calt42Struct class is key, providing necessary information for the algorithm's complexity. The structured objects in the spine array facilitate noise assessment and neighbor identification. The webpage includes methods for user-friendly interaction, such as reading neighborhood information from text files. Standalone executables and code checks aid in assessing algorithm outcomes using various data sets like recoT_all and WZ tests, with population reports illustrating improvements after implementing T42. The conclusions highlight remaining tasks like cross-checking neighbors and optimizing threshold values for minimal data loss, emphasizing the need for further studies on the algorithm's impact on physics analysis. Collaboration is encouraged for enhancing the rootuple data block.

alexander-woodard
Download Presentation

Optimizing Noise Reduction Algorithm T42 Implementation for Data Cleaning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CalTask Force Workshop T42 algorithm dedicated webpage :http://www-d0.fnal.gov/~vlimant/noise-suppdescr/Description.html VLIMANT Jean-Roch LPNHE 10 feb 2003 • Principale • Implementation • Effect on data • To come

  2. T42Principale : reject isolated medium cellsNamed after an algorithm used in H1 • Select high signal cells (4 sigma) • Keep their significant neighbours (2 sigma) Thresholds to be tuned

  3. cal_t42 Packageusing same frame as cal_nada Classes : • Calt42 : container for cell lists • Calt42Chunk : container for Calt42 object • Calt42Reco : • does the cleaning and chunks managment. • Get the neighbourhood info from a Calt42Struct object which allows a linear algorithm complexity.

  4. Calt42Reco

  5. Calt42Reco

  6. Calt42StructClass content • Objects • spine : array[81600] of structured object • Mapped to ieta, iphi, layer by :INDEX = iphi + 64( layer - 1 ) + 1088( ieta + 37 ) - 1 • Structured object fields • sigma : tells the noise rms of the cell • pointers to all possible neighbours within the array • single pointer to be assigned to a CalCell available in data • lots of methods user-friendly • method to read neighbourhood information from a text file (rather than doing awfull, uncheckable code)

  7. Calt42StructNeighbourhood informationhttp://www-d0.fnal.gov/~vlimant/noise-suppdescr/Description.html • How to check ? • Standalone executable return neighbours for a given cell (exist as a method) • Check cell_files and pictures for exceptions descriptions (webpage) • Check the code of the four executables (simple algorithms)

  8. Study effects on data • Data sets • Debugging (preliminary result ) with • recoT_all_0000168619_mrg_116-143.raw_p13.04.00 • MET test (sophie) 1000 evts of • recoT_all_0000168725_mrg_100-107.raw_p13.04.00 output : /work/coco-clued0/vlimant/TK2/runsophie/Recoanalyze.root • W, Z tests (WZ) 10000 evts of • WZskim-emStream-*-*.raw_p13.04.00 under the project name WZstreamTMBp13.04

  9. Population reportoverall result for 10000 evts processing (WZ)http://www-d0.fnal.gov/~vlimant/noise-suppps/WZ_plots/Pop_report_10000_WZ.ps Before1815 After1265 At first(0)1815 At second(4)840

  10. Population reportRegion result

  11. Population reportRegion result Clustered cells Isolated cells

  12. EM changes • Isolation decreases and tightens by 10% • EM fraction tightens by 25% • Hmatrix8 decrease by 5% EMIds increases by 10%

  13. before after

  14. before after

  15. before after

  16. MET changes • MET increase (in mean) W transverse mass with more entries • MEx centered (- 30%) • MEx made more even fewer noise

  17. before after

  18. before after

  19. T42 conclusions Remains to do • Ultimatetly cross check neighbours • Study thresholds for reasonnable loss of cells • More studies on effect on physics • Rootuple with dedicated data block available, help is welcome

More Related