110 likes | 262 Views
NIST BIG DATA WG Reference Architecture Subgroup Intermediate Report. Co-chairs: Orit Levin ( Microsoft) James Ketner ( AT&T) Don Krapohl (Augmented Intelligence ) August 21, 2013. Starting Point F rom the Previous J oint C all. Transformation includes Processing functions
E N D
NIST BIG DATA WGReference Architecture SubgroupIntermediate Report Co-chairs: Orit Levin (Microsoft) James Ketner (AT&T) Don Krapohl (Augmented Intelligence) August 21, 2013
Starting Point From the Previous Joint Call • Transformationincludes • Processing functions • Analytic functions • Visualization functions • Data Infrastructureincludes • Data stores • In-memory DBs • Analytic DBs Sources Transformation Data Infrastructure Security Cloud Computing Management Network Usage NIST Big Data WG / Ref Arch Subgroup
The Latest RA Diagram NIST Big Data WG / Ref Arch Subgroup
RA Current View • The current RA view is technological, meaning that • It doesn’t show topology or deployment models • It doesn’t show business stakeholders or business relationships • By using different sets of taxonomies, the same baseline diagram can be modified to show topologies or business relationships • These different views would need to be shown separately (e.g., technologies vs. deployment models vs. business stakeholders) • A proposal: Discuss these aspects in the context of the RoadMap scope and direction and sync on the direction and responsibilities NIST Big Data WG / Ref Arch Subgroup
Deliverables • Delivery #1: White Paper describing different RA approaches • The proposed Outline is in M0150 • Call for review • Call for volunteers to produce content for different sections • Delivery #2: Common RA Draft • The proposed Outline is in M123 • Call for additional volunteers to produce content for different sections • A number of participants volunteered already, more are welcome NIST Big Data WG / Ref Arch Subgroup
Deliverable #2:Dependencies on Other Subgroups 2 Big Data Reference Architecture: Use Cases Requirements • This section will be prepared by the NIST BDWG Requirements SG and will contain high level requirements relevant to the design of the Reference Architecture. 6 Big Data Reference Architecture: Security and Privacy • This section will be prepared by the NIST BDWG Security and Privacy SG and will contain high level security and privacy considerations relevant to the design of the Reference Architecture. 7 Big Data Taxonomy • This section will be prepared by the NIST BDWG Def&Tax SG and will contain high level taxonomy relevant to the design of the Reference Architecture. NIST Big Data WG / Ref Arch Subgroup
Backup Slides NIST Big Data WG / Ref Arch Subgroup
Reference Architecture Objectives(No Change) • Addresses a broad range of stakeholders (e.g., data owners, industries, academia, policy makers) • Wide scope: • Encompasses the whole data life cycle or in the ecosystem • Can be applied to different use cases (including various verticals) • Represents different system architectures (e.g., an enterprise data warehouse, distributed cloud-based system using multiple service providers) • Focus • Potentially with initial focus on the Big Data analytics and tools • Assists in identifying security and privacy issues • Agnostic to any specific technologies NIST Big Data WG / Ref Arch Subgroup
Input for Discussion on the BD RANext Level of Details • The next slides are based on various input documents to NIST BD subgroups • They are included here to provide insight on the evolution of the NIST BD RA work NIST Big Data WG / Ref Arch Subgroup
Proposal I: Highlighting the Different DB Approaches Sources Data Infrastructure VOLUME VARIETY VELOCITY Collection Streaming Transformation Aggregation Curation Integration Buffer Manager Data Manager Relational DB Pre-analytics Transfer Security Cloud Computing Management Network Search Storage (disk, memory, etc.) Specialized Abstractions Analytics Statistics RT Analytics Data Mining NoSQL DB File System Map Reduce Interactive An. Visualization Batch Analytics Export Usage NIST Big Data WG / Ref Arch Subgroup
Proposal II: Based on Submitted Requirements NIST Big Data WG / Ref Arch Subgroup