University of Colorado at Colorado Springs

1. University of Colorado at Colorado Springs Design of a Parametric Outlier Detection System Ronald Erickson as part of the requirements for the degree of Master of Engineering in Software Engineering

2. Example Integrated Circuit Test Floor Configuration A typical ATE test floor example is shown above. This type of configuration allows each platform to continue to operate independent in the event of a network problem. A database server with outlier detection PC has been shown attached to this configuration.

3. Background & Motivation Information • To stay competitive in the global market, manufacturing, packaging, & testing activities can be implemented anywhere. DfM can be used as a potential pre-test screen, but do not account for all failure mechanisms or the effects of commercial radiation hardened (CRH) implant. Ultimately electrical test and robust statistical screening of very large amounts parametric data must be accomplished to ensure the highest quality products are shipped to customers. • By utilizing a fab-lessmanufacturing system with no capital costs, it allows the company stay price competitive. However by using different suppliers many differing failure variables can be introduced into the final integrated circuits. • Integrated circuit defects are discovered during the electrical test (exercise) of the integrated circuit. These tests are performed on wafers and packaged devices • Companies need a tool that processes large amounts of device parametric data and performs a robust statistical analysis of the parametric data in an effort to screen out of family parametrics within a lot, product line, technology, or fab.

4. Goals of this project • The proposed tool would use parametric data, and perform a test for normality. • After a parametric value distribution has been deemed normal. The 75th percentile analysis by quartile is completed • If the test data distribution pass a normalcy test then the data will be presented in a histogram format for an engineer to intervene and set the lower and upper fences of the distribution. • In the case of a bi-model or non-normal distribution the data will be presented as a histogram, an engineer will have decide which distribution to use as the correct representation. • Engineer decisions into the histogram formatted fence boundaries decisions will be captured into the database for future use.

5. Challenges of this Project • AutomaticDistribution Modeling and implementation of very complex equations • Anderson-Darling with quartile analysis of normalized data only “ties” OR • Shapiro-Wilks with quartile analysis of normalized data only “ties”. OR • Skewness-Kurtosis All with quartile analysis of normalized data only “not robust”. • If the data set is a fails the normalcy tests due to bi-model or a non-normal distributions, including too many “ties“ encountered. • Can an alternative implementation be performed when false positive failing normalcy is implemented on Anderson-Darling “tie” related fails? • Data set will be shown in a histogram format and require user intervention to pick the data modeling performed. • Future enhancements on non-normal data sets will utilize machine learning to track the type of data modeling that was performed by user, device type, and distribution. All user interventions must be tracked for future analysis.

6. Tasks • Already Complete • Developed an application in C# to analyze datasets per the reference “Precision Estimates for AASHTO Test Method T308 and the Test Methods for Performance-Graded Asphalt Binder in AASHTO Specification M320” • In Progress: Intent to complete in Spring 2011 Semester • Develop the normalcy tests: • Shapiro-Wilks • Anderson-Darling • Skewness-Kurtosis All • Develop the quartile tests on the normalized data. • Future: Intent to complete in Spring 2011 Semester • Analyze the results of pre-determined datasets to the test-beds. • Write the project report

7. Deliverables • The outlier detection test-bed, including a device parametric data-log loaded into a database and a data modeling response that resembles a real product manufacturing scenario. • The outlier detection engine code that implements the Anderson-Darling, Shapiro-Wilk , or Skewness-Kurtosis All normalcy tests. Then if the data set is normal complete a 75th percentile on the data set. • If the data set does not pass the normal distribution tests, then present the data in a histogram format for user intervention. • A masters project report documenting the outlier detection design and the results of implementing the data-log within the outlier detection design prototype. • An analysis report describing the software engineering principles selected and how the selected techniques are applied in the outlier detection implementation.

8. References • Anil Kumar Jain, M Narasimha Murty, Patrick Joseph Flynn: Data clustering: a review. ACM Computing Surveys: Volume 31, Issue 3, Pages: 264 – 323, September 1999. • Ronald Holsinger, Adam Fisher, Peter Spellerberg, : Precision Estimates for AASHTO Test Method T308 and the Test Methods for Performance-Graded Asphalt Binder in AASHTO Specification M320. National Cooperative Highway Research Program, AASHTO Materials Reference Laboratory, Gaithersburg, Maryland, February, 2005 • Joao Gama, Pedro Pereira Rodrigues, and Raquel Sebastiao: Evaluating Algorithms that Learn from Data Streams. ACM: SAC '09:Proceedings of the 2009 ACM symposium on Applied Computing, March 2009. • Jennifer G. Dy, and Carla E. Brodley: Feature Selection for Unsupervised Learning. JMLR.org: The Journal of Machine Learning Research , Volume 5, December 2004. • Tony Jebara, Jun Wang, and Shih-Fu Chang: Graph Construction and b-Matching for semi-Supervised Learning. ACM: Proceedings of the 17th ACM international conference on Multimedia, October 2009. • David Moran, Daria Dooling, Tom Wilkins, Ralph Williams, and Gary Ditlow:Integrated Manufacturing and Development (IMaD). ACM: Supercomputing '99:Proceedings of the 1999 ACM/IEEE conference on Supercomputing, Jan 1999. • R.A Perez, J.T Lilkendey, and S. W Koh. Machine Learning for a Dynamic manufacturing Environment. ACM: SIGICE Bulletin , Volume 19, Issue 3 , February 1994. • [8] Khaled Saab, Naim Ben-Hamida, and Bozena Kaminska: Parametric Fault Simulation and Test Vector Generation. ACM: Proceedings of the conference on Design, automation and test in Europe, January 2000.

9. Prototype Application View 1 This application performs histogram fence choice and quartile of the reference “Precision Estimates for AASHTO Test Method T308 and the Test Methods for Performance-Graded Asphalt Binder in AASHTO Specification M320” see go 195

10. Prototype Application View 2 This application performs histogram fence choice and quartile of the reference “Precision Estimates for AASHTO Test Method T308 and the Test Methods for Performance-Graded Asphalt Binder in AASHTO Specification M320” see go 196

11. References Continued • SoumendaBhattachatya and AbhijitChatterjee. Optimized Wafer-Probe and Assembled Package Test Design for Analog Circuits. ACM: Transactions on Design Automation of Electronic Systems (TODAES), Volume 10 Issue 2, April 2005. • Wei-Shen Wang and Michael Orshansky: Robust Estimation of Parametric Yield under Limited Descriptions of Uncertainty. ACM: ICCAD '06: Proceedings of the 2006 IEEE/ACM international conference on Computer-aided design, November 2006. • Anne Gattiker: Using Test Data to Improve IC Quality and Yield. IEEE Press: ICCAD '08:IEEE/ACM International Conference on Computer-Aided Design, November, 2008 • Ashish Kumar Singh, Murari Mani, and Michael Orshansky: Statistical Technology Mapping for Parametric Yield. IEEE Computer Society: ICCAD '05:Proceedings of the 2005 IEEE/ACM International conference on Computer-aided design, May 2005. • KeesVeelenturf: The Road to better Reliability and Yield Embedded DfM tools. ACM: Proceedings of the conference on Design, automation and test in Europe, January 2000. • Erik Jan Marinissen, Bart Vermeulen, Robert Madge, Michael Kessler, Michael Muller: Creating Value Through Test: DATE '03: Proceedings of the conference on Design, Automation and Test in Europe - Volume 1, March 2003.