1 / 11

Stream Processing in PNEs

Stream Processing in PNEs. George Porter Edge Services Session Winter Retreat - 2004. Overview. Streams prevalent in the edge network PNEs—general platform for packet processing (roughly layers 2-4) in the edge

syshe
Download Presentation

Stream Processing in PNEs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stream Processing in PNEs George Porter Edge Services Session Winter Retreat - 2004

  2. Overview • Streams prevalent in the edge network • PNEs—general platform for packet processing (roughly layers 2-4) in the edge • We need a general mechanism for stream processing (roughly layer 7) that is integrated with the current PNE design and architecture • I am proposing an extension to PNEs to specify streams and a mechanism for executing that specification at high speeds in a general way from the PNE interface • This mechanism provides a clear separation between the transport of packets and the structure of the protocol

  3. Ethernet IP TCP Data Data Data Packet-vs.-stream processing: what’s the difference? • Packet Processing • Ethernet, IP, and TCP headers in every packet • Decisions are stateless between packets • Resilient to loss, reorder, duplication • Stream Processing • Assumes reliable transport • Sequence of variable length data units over TCP • Simple for endhosts, since O/S delivers in order • Loss, reorder, dups a problem for in-network elements • PNEs must track data unit boundaries Ethernet … IP TCP Ethernet Ethernet IP IP TCP TCP iSCSI1 Data1 iSCSI2 Data2 Data Data1

  4. Examples of observed phenomenon header1 data1 header2 data2 (observed on OASIS testbed) Original Stream: 1) 2) 3) (overlaps with first And last part of header)

  5. Need for stream processing • Necessary to track any layer-7 protocol that exists on top of TCP • Storage • iSCSI: storage virtualization, load-balancing, security, caching, in-network optimizations • Web/P2P • HTTP pipelining: load-balancing • P2P: HTTP often used as a transport protocol • Measurement/Monitoring • Tracking state of observed protocols over time

  6. H H PNE mechanism for tracking ADUs StreamTracker Packet flow (fast path) Packet(s) containing ADU boundary Priority Queue Computed ADU size FixedLogic (iscsi) Paramaterized by specification

  7. PNE mechanism for tracking ADUs StreamTracker Packet flow (fast path) Packet(s) containing ADU boundary Priority Queue Computed ADU size VariableLogic (http) Paramaterized by specification

  8. Evaluation Platform • Deployed onto testbed consisting of • 20 Pentium-500 Linux machines • 100 Mbit ethernet-based network • iSCSI source and target through Intel drivers • 100MB ramdisk on target / 30gig IDE Seagate drive • Code deployment • Baseline: Click modular router running in Kernel • “Tracking”: Click router in kernel with custom elements

  9. Results: #pkts examined • Only a small number of packets are examined in detail: FS caching does not effect above measurements

  10. Results: overhead of packet examination • Overhead of examining packets in depth is large, but infrequent:

  11. Next Steps • Implement in PNE software artifact • Write compiler for specification language • Deployment • (see poster for deployment overview) • “VideoCollective”, “VideoCollective with sharing”, “VideoCollective with collaborative sharing” • Measurement of storage statistics in above deployment

More Related