Update on the CAR Evaluation of Irreversible Compression for Medical Images

1. Update on the CAR Evaluation of Irreversible Compression for Medical Images David A. Koff MD Peter Bak PhD Paul Brownrigg MBA Luigi Lepanto MD Tracy Michalak MSc Harry Shulman MD Andrew Volkening MSc

2. Background Storage volume … • Even if the cost of storage is dropping, the volume of data to store keeps increasing, mainly CT Scan and MRI. • 35 million diagnostic imaging exams are performed annually in Canada. • With a legal retention period of 7 years • Use of irreversible compression could save M$100 million per year.

3. Background … and transmission times • and if access to high bandwidth gets more available in local hospital networks, it is still premature to expect any health professional to use 100 mbps connections on their computers. • EHR networks cannot support large medical images and timely access to diagnostic images requires adequate level of compression.

4. The CAR PACS committee position • The Canadian Association of Radiologists PACS/Teleradiology committee has accepted the principle of irreversible (“lossy”) compression for use in primary diagnosis and clinical review, using DICOM JPEG or JPEG-2000 compression algorithms, at specific compression ratios set by image type.

5. The CAR PACS committee position • The most commonly used techniques for lossy compression and decompression are JPEG and more recently DICOM JPEG 2000. • The JPEG 2000 algorithm allows higher levels of compression than JPEG at the same image quality. • However, at lower levels of compression (e.g. 30:1 in chest radiography), both methods produce equal results. (Slone et Al.) • This fact should be kept in mind as it is likely that most users will “play it safe” and opt for a compression ratio below the maximum possible for any given modality.

6. The CAR PACS committee position • This assessment is based on: • 2 literature reviews • 2 legal opinions • Before making a final decision, more evaluation is required.

7. Literature review • Lossy compression is a clinically acceptable option for the compression of medical images. • In comparison to wavelet, JPEG exhibits poorer ratios, PSNR values, artifacts and flexibility…but new JND research contradicts research to date. • Large matrix images can be compressed up to 50:1 (using wavelet) – 25:1 seems to be an optimum ratio. • Small matrix images can be compressed up to 20:1 (using wavelet) – 10:1 seems to be an optimum ratio. • The extent of allowable lossy compression ratio is dependent on the modality of the image and the nature of the imaged pathology and anatomy. • JPEG2000 performed as well as most other wavelet schemes but with the added benefit of being an ISO (and DICOM) standard.

8. Literature review

9. Legal opinion • 2 legal reviews: • Penny Washington of Bull, Housser and Tupper • Mitchel McInnes of University of Western Ontario • The conclusion is that use of lossy compression does not present any greater risk to the physicians then the typical risks that we live with in normal circumstances. • To date, the use of Lossy Compression has not yet been considered in a court of law in the United States or the Commonwealth.

10. Legal opinion • Provided that: • Appropriate ratios are used • There is no clinically significant loss of data • Lossy compression is used in primary reading (avoid altering records after primary reading) • The technology is not adopted recklessly and due diligence is applied, such as • Literature reviews • Education • Supervision, and • Technology is used appropriately

11. The CAR PACS committee position • The Canadian Association of Radiologists, with assistance from the Fraser Health Authority and Infoway, has decided to conduct a clinical evaluation to assess the most appropriate compression ratios in JPEG and JPEG 2000. • Guidelines will be presented to the CAR steering committee and recommended for adoption based on the results of our large scale evaluation study.

12. The evaluation project • Mandated by the Canadian Association of Radiologists in order to evaluate the acceptability of lossy compression and establish guidelines for its members. • Supported by Canada Health Infoway, an independent not-for-profit organization responsible for fostering the development and adoption of the EHR in Canada.

13. Methodology

14. Methodology • The goal of our study is to evaluate objectively if lossy compression can be used for all or part of the modalities and body parts assessed. • We don’t try to break records, but to find the lowest acceptable common denominator. • We don’t test readers skills.

15. Methodology Based on previous studies, our evaluation results from the association of 2 accepted methods: • Diagnostic accuracy evaluation with ROC analysis • Image comparison with original-revealed forced choice (JND)

16. Methodology 5 modalities (CR/DR, CT, US, MR, NM) and 7 radiological areas (Angio, Body, Breast, Chest, MSK, Neuro, Pediatrics)

17. Methodology • Images are compressed in JPEG and JPEG 2000 at 3 different ratios, close to acceptability, based on previous studies, below and beyond.

18. Methodology • Diagnostic Accuracy: objective measurement of diagnostic accuracy using blinded evaluation methods. • ROC Analysis: Receiver Operating Characteristic Analysis measures difference in perceived quality between original and reconstructed image through a confidence rating assigned by trained observers.

19. Diagnostic accuracy • Images are a mixed of normal cases and identified pathologies (4 abnormal:1 normal) • Images presented full screen compressed in JPEG and JPEG 2000 at 1 of the 3 different ratios or in original size.

20. Diagnostic accuracy Using normal images is important for two reasons: • It allows us to examine the possibility of artifacts that simulate abnormalities in a normal image; • It addresses the possibility of bias that could occur if readers know that each image contains some abnormality.

21. Diagnostic accuracy • The entire set of images is randomized to readers in such a way that each reader does not see the same image twice under different levels of compression; this eliminates bias due to recognizing the image. • A restricted number of pathologies are listed in a drop-menu from which the reader has to chose.

22. Diagnostic accuracy • Reviewer specifies in which sector of the image s/he sees the pathology and gives a confidence rating on a scale from 1 to 5(1=definite absence of lesion and 5= definite presence). 1 2 3 4

23. Image comparison with original-revealed forced choice (JND) • Each compressed image is paired with the original • Observer is asked to rate the degradation in image quality, which could impair interpretation, on a scale of 1 to 6. 30:1 Original

24. Image comparison with original-revealed forced choice (JND)

25. Methodology • 3 readers per session, 27 sessions, 81 readers in total from all across Canada. • Must have appropriate workstations (minimum 1.5 MPixel upon modality) and videocards (Matrox type). • The workstation must be connected to the Internet ideally. • Appropriate reading environment with correct ambient light.

26. Methodology • The reader receives a CD/DVD with 70 images or image stack of no more than 20 images for CT scan. • The reading session should not take more than 2 hours. • The session can be interrupted at any time and resumed automatically where it stopped. • The answers are filled on-line and directly transferred to our server. • If user cannot connect his workstation to the Internet, an alternate paper-based solution is provided.

27. The team

28. Sunnybrook • David Koff Principal Investigator • Tracy Michalak Project Manager • Andrew Volkening PACS administrator • Harry Shulman Scientific advisor • Engineering team for software development: 3 programmers and a tester with DICOM and medical imaging experience.

29. Steering committee • Peter Bak Infoway • Paul Brownrigg Fraser Health Authority • Luigi Lepento CAR • Jane Van Essen Infoway

30. Readers • Approximately 100 radiologists. • Readers representing all provinces.

31. Technical developments

32. Technical developments • An industry recognized compression engine from Pegasus Inc. (Tampa, FL): PICTools with JPEG 2000 and Lossy JPEG. • DICOM communication: MergeCOM-3 from Merge-Cedara

33. Technical developments • A standalone administrator application allowing: • Database of anonymized cases • Database of participating radiologists • Compression engine • Creation and compilation of worklists • CD/DVD engraving

34. Technical developments • A specially designed viewing software allows to display images to review at the different levels of compression then side by side paired with the original when revealed. • Limited processing functions are provided: • Zoom • Pan • Window-leveling • Reset

37. Timeframe

38. Timeframe • Initially • Software: April 28 • Radiologist recruitment: April 28 • Test software: May 12 • Completion of data acquisition: August 25 • Findings presented at RSNA: November 26

39. Timeframe • Revised • Radiologist recruitment: April 28 • Test software: July 31 • Completion of data acquisition: November 20 • Findings: January 30

40. Timeframe • Late, but stronger: • Evaluation of methodology • Evaluation of compression • Evaluation of source code

41. Challenges

42. Challenges • Technical development longer than expected: • Different types of images • Complexity of database: • Shuffle cases • Shuffle compression levels • Track answers • Multiplicity of reading environments • Different monitors and video cards

43. Challenges • Data acquisition: • Creating a database of 2500 cases has required more time than expected, as images have been provided by multiple sites and there has been some ownership issues on some datasets.

44. Challenges • Specific issues • Lossy JPEG accepts only 8 and 12 bit images, therefore necessity to convert 16 bit images into 12 bit images. • Compatibility issue with CR images from AGFA CR

45. Challenges • Specific issues • Conversion of Quality factor in size ratio and quality control to ensure accuracy of results. • Image quality issues, mainly in pediatrics due to: • Low radiation dose • Low acquisition times (MR)

46. Challenges • Connectivity • For several sites, we have been unable to connect the workstations to the Internet, mainly due to IT resilience to open firewalls. • This has now been solved for our largest site in BC, using a different port on our server (port 400).

47. Status

48. Status • As of February 8, 41 readers (on 81 required) have completed their evaluation. • This does not include the CR study, for which we have been delayed due to compatibility problems, and does not include readers from BC with whom we had connectivity issues. • These issues have now been solved.

49. Status

50. Status • Statistical analysis has started. • No result yet for the Diagnostic Accuracy with ROC analysis. • Preliminary results for the original revealed forced choice comparison showed that at the compression ratios chosen, there is no significant difference between original and compressed image.