Samira Khan University of Virginia Aug 28, 2019

1. Data-Centric System Design CS 6501 Samira Khan University of Virginia Aug 28, 2019 Some content of this course is adapted from CMU ECE 740

2. ROTUNDA, PAVILLIONS AND THE LAWN

3. A DESIGN BASED ON PRINCIPLES 1856 engraving • Designed by Thomas Jefferson • Unique design at that time (1819) • University surrounded by a library • Separation of church and education • Students and professors living in the pavilions • A holistic approach to training and education

4. A KEY QUESTION • How was Jefferson able to design a school that was so different from precedents? • Can have many guesses • Visionary • Passion for architecture and education • (Ultra) hard work, perseverance, dedication (over decades) • Experience of decades • Creativity • Out-of-the-box thinking • Principled design • A good understanding of past designs • Good judgment and intuition • Strong combination of skills (architecture, art, law, mathematics, horticulture, philosophy …) • … • (You will be exposed to and hopefully develop/enhance many of these skills in this course)

5. MAJOR HIGH-LEVEL GOALS OF THIS COURSE • Understand the principles • Understand the precedents • Based on such understanding: • Enable you to evaluate tradeoffs of different designs and ideas • Enable you to develop principled designs • Enable you to develop novel, out-of-the-box designs • The focus is on: • Principles, precedents, and how to use them for new designs • In Computer Architecture and Systems

6. ROLE OF THE (COMPUTER) ARCHITECT from Yale Patt’s lecture notes

7. ROLE OF THE (COMPUTER) ARCHITECT • Look backward (to the past) • Understand tradeoffs and designs, upsides/downsides, past workloads. Analyze and evaluate the past • Look forward (to the future) • Be the dreamer and create new designs. Listen to dreamers • Push the state of the art. Evaluate new design choices • Look up (towards problems in the computing stack) • Understand important problems and their nature • Develop architectures and ideas to solve important problems • Look down (towards device/circuit technology) • Understand the capabilities of the underlying technology • Predict and adapt to the future of technology (you are designing for N years ahead). Enable the future technology

8. TAKEAWAYS • Being an architect is not easy • You need to consider many things in designing a new system + have good intuition/insight into ideas/tradeoffs • But, it is fun and can be very technically rewarding • And, enables a great future • E.g., many scientific and everyday-life innovations would not have been possible without architectural innovation that enabled very high performance systems • E.g., your mobile phones • This course will teach you how to become a good computer architect

9. INSTRUCTOR • Samira Khan • Assistant Professor • samirakhan@virginia.edu • Post Doctorate in CMU • PhD from UT San Antonio • Worked at Intel, AMD, EPFL • Research Interest • Computer Architecture and Systems • Memory System Design • New Emerging Technologies • Hardware-Software Co-Design • Performance, Power, Reliability

10. TEACHING ASSISTANT • Yizhou Wei • CS PhD Student • yw2bc@virginia.edu • Research Interests • Computer Architecture and Systems • Memory • Processing in Memory

11. AGENDA • Course Overview • How to Review Papers • First Homework (for NextWeek) • First Review Assignment (for Next Week)

12. WHERE TO GET UP-TO-DATE COURSE INFO? • Website http://www.cs.virginia.edu/~smk9u/CS6501F19/home.html • Piazza piazza.com/virginia/fall2019/cs6501f19/home • Your email: start the subject line with CS 6501 Fastest Response • Office Hours By appointment

13. WHAT WILL YOU LEARN • Computer Architecture: The science and art of designing, selecting, and interconnecting hardware components and designing the hardware/software interface to create a computing system that meets functional, performance, energy consumption, cost, and other specific goals. • Traditional definition:“The term architecture is used here to describe the attributes of a system as seen by the programmer, i.e., the conceptual structure and functional behavior as distinct from the organization of the dataflow and controls, the logic design, and the physical implementation.”Gene Amdahl, IBM Journal of R&D, April 1964

14. LEVELS OF TRANSFORMATION Problem Operating System Interface between Software and Hardware Architecture Circuits

15. THE POWER OF ABSTRACTION • Isolation • A higher level only needs to know about the interface to the lower level, not how the lower level is implemented • For example, high-level language programmer does not really need to know about the architecture • Productivity • No need to worry about decisions made in underlying levels • For example, programming in Java vs. C vs. assembly vs. binary vs. by specifying control signals of each transistor every cycle

16. CROSSING THE ABSTRACTION LAYERS • Should we always focus on our own layer? • As long as everything goes well, not knowing what happens in the underlying level (or above) is not a problem • What if • One of the layers reach a limit, there is no way to improve • There is a new disruptive change in technology that cannot be contained in a layer • New Applications that are too slow for today’s system

17. LEVELS OF TRANSFORMATION Problem Broader Scope of Computer Architecture Operating System Architecture Circuits

18. SCOPE OF THE COURSE • This course will take a broad view of architecture • Beyond the ISA+microarchitecture levels • E.g., system-architecture interfaces and interactions • E.g., application-architecture interfaces and interactions • E.g., Domain specific designs • Out-of-the-box thinking is greatly encouraged • E.g., research projects and readings on architectures that challenge the current dominant paradigms • processing in memory, emerging memory technologies, specialization everywhere, … • E.g., readings on topics that are traditionally covered less in computer architecture courses

19. WHAT WILL YOU LEARN? • Hardware/software interface, major components, and programming models of a modern computing platform • State-of-the-art as well as research proposals (lots of them) • Tradeoffs and how to make them • Emphasis on cutting-edge (research & state-of-the-art) • Hands-on research in a computer architecture topic • Semester-long research project • Focus: How to design better architectures (not an intro course) • How to dig out information • No textbook really required • But, see the syllabus

20. ONE EXAMPLE New Challenge: Memory Wall

21. DRAM Latency-Capacity Trend 16X -20% DRAM latency continues to be a critical bottleneck

22. Memory Wall 1) High latency Memory L3 CPU L2 L1 MC 2) Low bandwidth Memory is the main bottleneck

23. Processing in Memory Calculate in memory Smart Memory L3 CPU L2 L1 MC Compute Only send back the result Memory is the main bottleneck

24. IN-MEMORY COMPUTING IN-MEMORY LOGIC PROBLEM SPECIALIZED CORES APPLICATION Which functions to offload? Software OS What is the interface? RUNTIME Who provides data consistency? How to schedule data access and offloading? ARCHITECTURE How to enable computation? Hardware CIRCUITS MEMORY WITH LOGIC Offload computation to appropriate computing model based on application, runtime, and accelerator characteristics Image: Micron Technology, 2015

25. The Problem Processing of data is performed far away from the data

26. A Computing System • Three key components • Computation • Communication • Storage/memory Burks, Goldstein, von Neumann, “Preliminary discussion of the logical design of an electronic computing instrument,” 1946. Image source: https://lbsitbytes2010.wordpress.com/2013/03/29/john-von-neumann-roll-no-15/

27. A Computing System • Three key components • Computation • Communication • Storage/memory Burks, Goldstein, von Neumann, “Preliminary discussion of the logical design of an electronic computing instrument,” 1946. Image source: https://lbsitbytes2010.wordpress.com/2013/03/29/john-von-neumann-roll-no-15/

28. Today’s Computing Systems • Are overwhelmingly processor centric • All data processed in the processor  at great system cost • Processor is heavily optimized and is considered the master • Data storage units are dumb and are largely unoptimized (except for some that are on the processor die)

29. Even Bigger Problem at a Global State

30. The Problem Processing of data is performed far away from the data The Solution Processing data should be performed where it is generated

31. COURSE GOALS • Goal 1: To familiarize you with both fundamental design tradeoffs and recent research issues/trends in processor, memory, and platform architectures in today’s and future systems • Strong emphasis on fundamentals and design tradeoffs. • Strong emphasis will be given on data-centric system design • Goal 2:To provide the necessary background and experience toadvance the state-of-the-art in computer architecture by performing cutting-edge research • Strong emphasis on • Critical analysis of research papers (through reading and literature review assignments) • Developing new mechanisms that advance the state of the art (through the course research project).

32. THIS IS AN ADVANCED CLASS • Required background: • basic architecture (3330) • basic compilers • basic OS • programming skills • spirit, excitement, and dedication for deep exploration of a topic in computer architecture

33. WHAT DO I EXPECT FROM YOU? • Learn the material & dig deeper • Work hard • Ask questions, take notes, participate in discussion • Critically review the assigned research papers & readings • Discuss/critique them online with peers and us • Use Piazza frequently… • Start the research project early and focus • Remember “Chance favors the prepared mind.” (Pasteur)

34. HOW WILL YOU BE EVALUATED? • Research Project: 40% • Critical Reviews and Paper Presentation: 25% • Exam(s): 30% • Homework, class participation, and my evaluation of your performance: 5% • Participation+discussion is very important • Grading will be back-end heavy. Most of your grade will be determined late. How you prepare and manage your time is important • But grades should not be the reason for taking this course

35. RESEARCH PROJECT • Your chance to explore in depth a computer architecture topic that interests you • Perhaps even publish your innovation in a top computer architecture conference. • Start thinking about your project topic from now! • Interact with me and mentors • Read the project topics handout well • Groups of 1/2 students (will finalize this later) • Proposal due: Oct 2, 2019

36. AGENDA • Course Overview • How to Review Papers • First Homework (for Next Week) • First Review Assignment (for Next Week)

37. HOW TO DO THE PAPER REVIEWS • 1: Brief summary • What is the problem the paper is trying to solve? • What are the key ideas of the paper? Key insights? • What is the key contribution to literature at the time it was written? • What are the most important things you take out from it? • 2: Strengths (most important ones) • Does the paper solve the problem well? • 3: Weaknesses (most important ones) • This is where you should think critically. Every paper/idea has a weakness. This does not mean the paper is necessarily bad. It means there is room for improvement and future research can accomplish this. • 4: Can you do (much) better? Present your thoughts/ideas. • 5: What have you learned/enjoyed/disliked in the paper? Why? • 6: Detailed comment on the contribution of the paper • Review should be short and concise (~half a page to a page)

38. ADVICE ON PAPER REVIEWS • When doing the reviews, be very critical • Always think about better ways of solving the problem or related problems • Do background reading • Reviewing a paper/talk is the best way of learning about a research problem/topic • Think about forming a literature survey topic or a research proposal based on the paper (for future studies)

39. PAPER PRESENTATION • Group of 1/2 students • Each class two groups will present two papers • I will provide a list of papers and you can choose • Have to submit the slides one week before • Everyone else will submit reviews on those papers • Ask questions during/after the presentation • Remember 5% grade is on class participation • The presenters will also initiate discussion on piazza • Each person must have at least one post for each paper on piazza

40. PROJECT • More information to come… In the meantime: • Read a lot of papers; find focused problem areas to survey papers on • We will provide a list of project ideas and papers associated with them • A good way of finding topics to survey or do projects on is: • Examining the provided project ideas and papers • Reading assigned papers in lectures • Examining papers from recent conferences (ISCA, MICRO, HPCA, ASPLOS, …)

41. PROJECT TIMELINE • Project Proposal Due: Oct 2, 2019 • Project Proposal Presentation: Oct 9, 2019 • Project Milestone Slides Due: Oct28, 2019 • Project Milestone Presentation: Oct30, 2019 • Final Presentation: Dec9, 2019 (2.00-5.00 pm) • Final Report Due: Dec9, 2019

42. AGENDA • Course Overview • How to Jump Into Research • How to Review Papers • First Homework (for Next Week) • First Review Assignment (for Next Week)

43. HOMEWORK0 • Due on next Monday Sep 2, 2019 • Student information • Want to know about you • What to know about your prior computer architecture background • All future grading is dependent on this homework • Submit a pdf file in Collab

44. AGENDA • Course Overview • How to Review Papers • First Homework (for Next Week) • First Review Assignment (for Next Week)

45. REQUIRED REVIEWS • Due on next Wednesday Sep 4, 2019 • Enter your reviews in Collab • Start discussing ideas and thoughts on Piazza

46. REVIEW PAPER 1 (REQUIRED) • OnurMutlu, "Enabling the Adoption of Processing-in-Memory: Challenges, Mechanisms, Future Research Directions", Invited Article, 2018. • Richard Hamming, "You and Your Research”.

47. Data-Centric System Design CS 6501 Samira Khan University of Virginia Aug 28, 2019 Some content of this course is adapted from CMU ECE 740