1 / 19

An Exploitation of Tufte’s Small Multiples

An Exploitation of Tufte’s Small Multiples. Carl T. Russell Colorado Springs, Colorado russellcarl@earthlink.net. Operational Overview. Theater Missile Defense System Exerciser (TMDSE) Geographically-distributed hardware-in-the-loop (HWIL) simulation

wilton
Download Presentation

An Exploitation of Tufte’s Small Multiples

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Exploitation ofTufte’s Small Multiples Carl T. Russell Colorado Springs, Colorado russellcarl@earthlink.net

  2. Operational Overview • Theater Missile Defense System Exerciser (TMDSE) • Geographically-distributed hardware-in-the-loop (HWIL) simulation • Used to investigate terminal-phase theater Family of Systems (FoS) interoperability • TMDSE stimulates Tactical Segments • Provides simulated threat environment via DIS PDUs • Provides communications connectivity • Emulates Joint Data Network (JDN) • Tactical Segments exchange TADIL-J messages over an emulated JTIDS communications network consisting primarily of gateways and T1 lines Operational Question Addressed in This Paper:What are the time delays (latencies) associated with the emulated tactical communications network?

  3. Analytical Overview • Large sample sizes • Widely varied sample sizes • ANOVA dumb, tables unwieldy • Everything significant in ANOVA • Detailed tables of latencies have several hundred pages Analytical Question Addressed in This Paper:How well can quantitative graphics be used toassess the communications latencies?

  4. AFC TMD Operational Architecture

  5. TIBS TDDS JNIC Test Exercise Controller AFC (JNIC) Schriever AFB, CO Syracuse, NY USMC ADCP Dahlgren, VA Azusa, CA Huntsville, AL Kirtland AFB, NM Redstone Arsenal, AL AEGIS AAW & AEGIS Linebacker JTAGS PATRIOT THAAD MTS TMDSE Tactical Segments

  6. TIBS / TDDS TIBS TDDS TIBS/TDDS TIBS Joint Data Net (Link 16) Emulation Tactical Communications Remote Remote Remote Remote Remote Remote Central Remote Environment * Gateway Gateway Gateway Gateway Gateway Gateway Gateway Gateway Segment (TCES) X2 X1 X1 X1 X1 X1 (TCS) X3 X.25 1553 1553 X.25 1553 X.25 LAN RS232 Tactical Hardware-In-the AEGIS AEGIS CRC Loop AAW USMC ADCP LINEBACKER (MTS only) JTAGS PATRIOT THAAD AFC & TPS 59 X2 X1 X1 X1 X1 X1 X1 X1 RES Tactical System ACSIS ACSIS Tactical X1 RES Tactical MSIM & DIS Drivers FMS/D TTC Driver Gateways Driver RES Work Segment (TSDS) Adjunct Adjunct X1 X2 X1 X1 X1 Station Processor Processor Remote Remote Remote Remote Remote Remote Remote Environment Environment Environment Environment Environment Environment Environment TMDSE Control Segment (TCS) * Router Router Router Router Router Router Router T1 Link Local T1 Link T1 Link = DIS + TADIL J T1 Link T1 Link T1 Link Link + Voice Coord Router Test Exercise Controller (TEC) Central TEC Sub- Gateway Systems Joint National Integration Center (JNIC), Colorado X1 * Each Router provides a connection (not shown) to the JDN Gateway TMDSE Test Architecture

  7. TransmittingNode Receiving Nodes end-to-endlatencies point-to-pointlatency point-to-pointlatencies Transmitting Gateway ReceivingGateways point-to-pointlatencies Communications Latency in TMDSE • TADIL-J messages recorded and time-stamped • At transmitting node • At transmitting gateway • At receiving gateways • At receiving nodes • TADIL-J messages matched end-to-end for analysis • Latencies calculated point-to-point and end-to-end • Flawed process, 95%+ successful

  8. Analysis Challenges (1 of 3)—Large Sample Sizes—

  9. 100 Number of Factor Combinations 50 0 0 4 16 36 64 100 144 196 256 324 400 484 576 676 784 900 1024 Number of Messages per Factor Combination(Square Root Scale) * * Min=1 10%=11 25%=40 50%=103 75%=217 90%=450 max=1099 Total Number of Factor Combinations=1143 Analysis Challenges (2 of 3)—Widely Varied Sample Sizes— Factors and levels (1143 Factor Combinations) • Test run (3 levels) • Message type (5 levels) • Transmitting segment (8 levels nested in message type) • Receiving segment (7 levels nested in message type)

  10. Analysis Challenges (3 of 3)—ANOVA Dumb, Tables Unwieldy— • Everything is significant in ANOVA • Tables of latencies have several hundred pages

  11. Time Delay 0 TADIL-J Message Type and Receiving Segment A A A A B B B B C C C C D D D D E E E E F F F F G G G G A B C D E F G b a c d e Graphical Approach Works—Initial Display— Time Delays for Key Messages Transmitted by Segment “B” on Test Run I,by Message Type and Receiving Segment • 80% boxplots used • No latencies for receipts by “B” (“B” was transmitting) • No latencies for message type “e” (“B” did not transmit any messages of type “e”) • No latencies for receiving node “H” (message matching unreliable) Tufte’s notion of “small multiples”—“graphics can be shrunk way down”—will be used on subsequent slides to exploit this initial display

  12. Transmitting Segment TestRun A C D E F G H ALL B I 0 0 0 0 II III ALL Time Delay Time Delay Time Delay Time Delay a a a a a a a a a b c d e b b b b b b b b c c c c c c c c d d d d d d d d e e e e e e e e Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Within message type, the “whiskers” on the boxplots extend from the 10th to the 25th percentile and the 75th to the 90th percentile of the latency distribution for each receiving segment; the 50th percentile (median) is marked with a horizontal line. Within message type, left to right receiving segment order is: A, B, C, D, E, F, G.Delays corresponding to Segment H node receipts are not displayed, and the sending node for each cell has no receiving latency distribution except node to gateway. Transmit to Receive Time Delays by Test Run, Transmitting Segment, Message Type and Receiving Segment (80% Boxplots). Times Adjusted for Apparent Node Clock Errors.

  13. Transmitting Segment A C D E F G H ALL B 0 0 0 0 Within message type, the “whiskers” on the boxplots extend from the 10th to the 25th percentile and the 75th to the 90th percentile of the latency distribution for each receiving segment; the 50th percentile (median) is marked with a horizontal line. Within message type, left to right receiving segment order is: A, B, C, D, E, F, G.Delays corresponding to Segment H node receipts are not displayed, and the sending node for each cell has no receiving latency distribution except node to gateway. a a a a a a a a a b c d e b b b b b b b b c c c c c c c c d d d d d d d d e e e e e e e e Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Time Delays on Run I by Type Delay, Transmitting Segment, Message Type and Receiving Segment (80% Boxplots). Times Adjusted for Apparent Node Clock Errors. Type Delay Nodeto Gateway Time Delay Gatewayto Gateway Time Delay Gatewayto Node Time Delay Transmitter to Receiver Time Delay

  14. Transmitting Segment Type Delay A C D E F G H ALL B Nodeto Gateway 0 0 0 0 Gatewayto Gateway Gatewayto Node Transmitter to Receiver Time Delay Time Delay Time Delay Time Delay a a a a a a a a a b b b b b b b b b c c c c c c c c c d d d d d d d d d e e e e e e e e e Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Within message type, the “whiskers” on the boxplots extend from the 10th to the 25th percentile and the 75th to the 90th percentile of the latency distribution for each receiving segment; the 50th percentile (median) is marked with a horizontal line. Within message type, left to right receiving segment order is: A, B, C, D, E, F, G.Delays corresponding to Segment H node receipts are not displayed , and the sending node for each cell has no receiving latency distribution except node to gateway. Time Delays on Run II by Type Delay, Transmitting Segment, Message Type and Receiving Segment (80% Boxplots). Times Adjusted for Apparent Node Clock Errors.

  15. Transmitting Segment Type Delay A C D E F G H ALL B Nodeto Gateway 0 0 0 0 Gatewayto Gateway Gatewayto Node Transmitter to Receiver Time Delay Time Delay Time Delay Time Delay a a a a a a a a a b b b b b b b b b c c c c c c c c c d d d d d d d d d e e e e e e e e e Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Within message type, the “whiskers” on the boxplots extend from the 10th to the 25th percentile and the 75th to the 90th percentile of the latency distribution for each receiving segment; the 50th percentile (median) is marked with a horizontal line. Within message type, left to right receiving segment order is: A, B, C, D, E, F, G.Delays corresponding to Segment H node receipts are not displayed, and the sending node for each cell has no receiving latency distribution except node to gateway. Time Delays on Run III by Type Delay, Transmitting Segment, Message Type and Receiving Segment (80% Boxplots). Times Adjusted for Apparent Node Clock Errors.

  16. Transmitting Segment TestRun A C D E F G H ALL B I 0 0 0 0 II III ALL Time Delay Time Delay Time Delay Time Delay a a a a a a a a a b b b b b b b b b c c c c c c c c c d d d d d d d d d e e e e e e e e e Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type Within message type, the “whiskers” on the boxplots extend from the 10th to the 25th percentile and the 75th to the 90th percentile of the latency distribution for each receiving segment; the 50th percentile (median) is marked with a horizontal line. Within message type, left to right receiving segment order is: A, B, C, D, E, F, G.Delays corresponding to Segment H node receipts are not displayed, and the sending node for each cell has no receiving latency distribution except node to gateway. Transmit to Receive Time Delays by Test Run, Transmitting Segment, Message Type and Receiving Segment (80% Boxplots). Times Not Adjusted for Apparent Node Clock Errors.

  17. TestRun Transmitting Segment A C D E F G H B I Time Delay 0 0 0 Segment E biased up( fast node clock) } } Segment F biased up( fast node clock) II Time Delay Segment E biased up( fast node clock) } Segment F biased down( slow node clock) } Segment G biased down( slow node clock) III Time Delay Segment F biased up( fast node clock) a b c d e a b c d e a b c d e a b c d e a b c d e a b c d e a b c d e a b c d e Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type } } Segment E biased up( fast node clock) Within message type, the “whiskers” on the boxplots extend from the 10th to the 25th percentile and the 75th to the 90th percentile of the latency distribution for each receiving segment; the 50th percentile (median) is marked with a horizontal line. Within message type, left to right receiving segment order is: A, B, C, D, E, F, G.Delays corresponding to Segment H receipts are not displayed, and the sending node for each cell has no receiving latency distribution except node to gateway. Gateway to Node Time Delays by Test Run, Type, Transmitting Segment, Message Type and Receiving Segment (80% Boxplots).Times Not Adjusted for Apparent Node Clock Errors. }

  18. TestRun Transmitting Segment A C D E F G H B I Segment E biased down( fast node clock) Segment F biased down( fast node clock) Segment H biased up( slow node clock) Time Delay 0 0 0 } } } } } } } } } } } II Segment E biased down( fast node clock) Segment F biased up( slow node clock) Segment G biased up( slow node clock) Segment H biased up( slow node clock) Time Delay } } } } } } } } } } } III Segment E biased down( fast node clock) Segment F biased down( fast node clock) Time Delay a b c d e a b c d e a b c d e a b c d e a b c d e a b c d e a b c d e a b c d e Message Type Message Type Message Type Message Type Message Type Message Type Message Type Message Type } } } } } } } } Within message type, the “whiskers” on the boxplots extend from the 10th to the 25th percentile and the 75th to the 90th percentile of the latency distribution for each receiving segment; the 50th percentile (median) is marked with a horizontal line. Node to Gateway Time Delays by Test Run, Type, Transmitting Segment, Message Type and Receiving Segment (80% Boxplots).Times Not Adjusted for Apparent Node Clock Errors. } } }

  19. Summary • Simple statistical graphics can produce rich, easily understood analyses via tables of miniature displays • Motivated by Tufte and others, “Data Density and Small Multiples” “Shrink Principle”—“Graphics can be shrunk way down” • Works very well on 1200 dpi printer with tabloid paper • Still marginally acceptable on letter paper (or transparencies) • Such graphics still a hard sell • DoD likes three bullets per slide, three words per bullet • Tufte says decision makers can be led through dense graphics but must be kept focused—then credibility of analysis increased Moral:Push for denser graphics despite resistance

More Related