Knowledge Grid and Cooperative Computing

1. Knowledge Grid and Cooperative Computing 窦 万 春 南京大学计算机科学与技术系 Douwc@nju.edu.cn

2. Context • Section 1: Concepts,State of art, Tendency, Applications related to Grid Computing • Section 2: Concepts,State of art, Tendency, Applications related to Knowledge Grid • Section 3: Cooperative Computing in Knowledge Grid environments

3. Section 1: Grid Computing

4. What is “Grid Computing”? • Grid Computing = Grid + Computing • What is Grid? • What is Computing

5. What is Grid?

7. What Defines The Internet “We will probably see the spread of ‘computer utilities, which, like present electric and telephone utilities, will service individual homes and offices across the country” -Len Kleinrock. 1969 [1]

8. Let begin… IBM对网格计算的技术理解：

9. Ian Foster:Professor of Computer ScienceThe University of ChicagoDepartment of Computer ScienceUniversity of Chicago Ryerson Hall, Room 250 1100 E. 58th St. Chicago, Illinois 60637 312-702-3487 (OFFICE)312-702-8487 (FAX)foster@cs.uchicago.edu

10. What is Grid? “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities.” - Ian Foster and Carl Kesselman, 1998

11. Do you know… • 2003年10月中国国家教育部先后与Intel 和IBM公司联手共建中国教育科研网格并宣称“是迄今为止有政府推出的最宏大的网格工程” • 联想与曙光推出了面向网格的高性能计算机 • 中科院计算所开发了网格系统软件（织女星） • 《福布斯》杂志预测网格技术将在2004-2005年出现高峰，到2020年则将产生一个年产值达20万亿美元的大产业 • IBM在 2001年就 宣布投资40亿美元大规模地进入网格计算领域，2003年10月又宣布投资100亿美元启动面向网格的“按需计算”计划 • ……

12. ChinaGrid

15. Context of Grid • 网格是一门技术 • 网格是一种标准 • 网格是一种方法（实现资源共享的方法） • 网格是一种中间件 • 网格是一种支持高级计算的基础设施 • ……

16. General ideas about the Grid Computing • 网格计算是利用互联网把分散在不同地理位置上的多个计算资源，通过逻辑关系组成一台“虚拟的超级计算机”，而这台机器把每一台参与其中的计算机都作为自己的一个“结点”，成千上万的这样的“结点”并联起来，就组成了“一个有超级计算能力的网格”。每一位将自己的计算机连接到网格上的用户，也就“拥有了”这台超级计算机，然后就可以随时随地地调用其中的计算和信息资源，在获得一体化信息服务的同时，最大限度的实现资源共享。

17. Basic Concepts • 网格(Grid) • 网格计算 • 计算网格 • 资源网格 • 数据网格 • 信息网格 • 知识网格 • 网格计算机 • …… • xxx网格 • 网格xxx E-Science P2P Semantic Web Knowledge Grid

18. Ian Foster’s Three Point Checklist (2002) A Grid is a system that: • “coordinates resources that are not subject to centralized control” • “using standard, open, general-purpose protocols and interfaces” • “to deliver nontrivial quantities of service”

19. 2 What is Computing

20. Paradigms of Computing • Distributed Computing • Cooperative Computing • Agile Computing • Pervasive/Ubiquitous Computing • Service Computing • Mobile Computing • Grid Computing • Dependable Computing • Parallel Computing • Autonomic/Autonomous Computing • … xx Computing

21. Further Comprehension • Pervasive Computing: Context-Aware + Transparent +Consistency • Autonomic/Autonomous Computing Self-*: Self-Management, Self-Governing,Self-Organization, Self-Configuration, self-optimization, self-healing, and self-protection • Dependable Computing: Available + Reliable + Safe + Secure

22. Essence of Computing • Computing = Computed Behavior • What is a behavior? • From sociological view of point… • Precondition: Formalization • Role: The bridge of theory and experiment

23. 中文译文 • 理论、计算与实验是科学领域中最为基本的研究手段和应用方式。围绕以“计算”(Computing)为主题的理论与应用研究是计算机科学与应用领域中的核心问题之一。在计算机科学与应用领域中，计算的概念往往具有非常泛化的含义，所有针对具体问题求解的行为和操作都可以看作是计算的具体表现形式；因此，从本质上而言，计算过程是一种基于特定理论基础与应用技术的计算机化的运算行为或任务执行过程。针对特定问题的计算工具、计算环境都可以为计算主体提供有效的技术支撑。而计算机化的问题求解环境，则为计算过程的有序开展提供了非常有效的技术手段和环境支持。

24. xx Grid • Grid xx • xx Computing Grid Computing

25. The definition of Grid Computing in our discussion • The Grid Computing is the process of implementation of a computed behavior, which mainly designed, developed, deployed, executed in Grid environments based on certain application scenarios.

26. Essence of Grid Computing • Target: Enhance the Computing Power • Infrastructure: Grid

27. Grid: An Evolution, not revolutionSource: IBM Grid Computing Grid can be seen as the latest and most complete evolution of more familiar development. • Like the Web: Grid keeps complexity hidden: multiple users enjoy a single unified experience. • Unlike the Web: enables full collaboration toward real business goal. • Like Peer-to-Peer It allows user to share files. • Unlike Peer-to-Peer Not only files, but everything which could be shared . • Like Clusters and distributed computing It bring computing resource together. • Unlike Clusters and distributed Computing Grid can be geographically distributed and heterogeneous. • Like Virtualization technologies enables virtualization of IT resources. • Unlike Virtualization technologies It can enable virtualization of vast and disparate resources.

28. Originally Targeted Applications What types of applications will grid be used for ? • Distributed Supercomputing • On-demand Computing NetSolve, large archives • Data-Intensive Computing SloanDigital Sky Survey, Weather forecasting • Collaborative Computing Insors, GriPhyN, SciRUN

29. Top 500 Supercomputers (June 2003) Earth Simulator: NEC : Yokohama : 35.86 TFlops ASCI Q: LANL: Los Alamos: HP Alphaserver SC: 13.88 TFlops MCR Linux Cluster: LLNL Livermore, 7.634 TFlops ASCI White: LLNL, Livermore IBM SP Power3, 7.304 TFlops Seaborg: NERSC/LBNL, Berkeley, IBM SP Power3, 7.303 TFlops Source : http://www.top500.org

30. General highlights from Top 500 (June 2003) • 157 systems reported to have peak performance above 1 TFlops. • Total accumulated performance is 375 TFlops. ( up from 293 TFlops )(floating point operations per second) • Entry level performance is 245.1 GFlops. (Up from 195.8) • A Total of 119 systems (up from 56) uses Intel processors. • 149 systems are now labeled as clusters ( up from 53 ) • 23 of them are self-made ( Up from 14 ) • Among top 10, 7 from US, 2 from Japan, 1 from France.

31. Economics and Control The infrastructures are very expensive and require years of hard work. The shear force of economics will require that these resources are under strict control and are optimally utilized. Many times freedom is costly and chaotic. This is the starting what we call Grid Computing

32. Changing face of Enterprise Computing • Most of the recent, enterprise systems are collection of heterogeneous resources. • Quality of services traditionally associated with mainframe centric computing are now essential to the effective conduct of e-business across distributed resources, inside as well as outside the enterprise. • Recently there is upsurge of services providers of various types such as web-hosting SP, storage SP, application SP All these require standardization.

33. 网格技术的应用范例 • 计算网格 • 资源网格 • 数据网格 • 信息网格 • 知识网格 • 网格工作流应用 • 虚拟组织 • ……

34. 小结论 • Killer application:高性能技术领域; • “造房不如买房,买房不如租房”的应用思想; • 首先需要一定的基础设施; • 在特定应用需求情况下的一种计算模式; • 技术发展到一定阶段的集成模式(应用驱动); • 现有技术的应用挑战; • 泛在协同的应用思想;

35. Development Path

36. 1. 计算机的性能已经提高了80亿倍

37. 2. 但应用的需求增长得更快

38. 3. 人们建造越来越大的超级计算机

39. 4. 但超级计算机造起来很麻烦

40. 5. 集群计算机很容易造

41. 6. 但它的规模也不能无限扩张

42. 7. P2P计算解决了规模问题

43. 8. 但大多数问题割不成P2P喜欢的样子

44. 9. 网格虚拟出空前的超级计算机 成为下一代Internet的发展方向

45. 10. 但它是不是最终的必杀武器？……

46. Section 2: Knowledge Grid (KG)

47. What is KG • Fran Berman put it forward that a Knowledge Grid is the convergence of a comprehensive computational infrastructure along with the scientific data collections and applications for routinely supporting the synthesis of knowledge from that data.

48. Cont. • Hai Zhuge believed that the Knowledge Grid is an intelligent, sustainable Internet application that enables people or virtual roles (mechanisms that facilitate interoperation among users, applications, and resources) to effectively capture, publish, share, and manageexplicit knowledge resources.

49. Five issues related to KG • The first is theories, models, methods, and mechanisms for capturing and representing knowledge. • The second issue is knowledge visualization and innovation. • The third issue is effective propagation and management of knowledge in dynamic virtual organizations. • The fourth issue is effective knowledge organization, evaluation, refinement, and derivation. • The fifth is knowledge association and integration. Based on those ideas, a Knowledge Grid should synthesize the integration of the data, computing, and the network hardware, the development of the software, and the coordination of a large and distributed human infrastructure.

50. The main function of Knowledge Grid is to synthesize knowledge from data by means of mining and reference, to enable search engines to make references, answer questions, and to draw conclusions from masses of data.