1 / 23

Fusion Data Management and Visualization for long time simulations.

Fusion Data Management and Visualization for long time simulations. China ITER Workshop Beijing, China May 18, 2006 Scott A. Klasky (ORNL) M. Beck (UTK) M. Parashar (Rutgers University) SDM Center + many other collaborators. My talk.

aquila
Download Presentation

Fusion Data Management and Visualization for long time simulations.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Fusion Data Management and Visualization for long time simulations. China ITER Workshop Beijing, China May 18, 2006 Scott A. Klasky (ORNL) M. Beck (UTK) M. Parashar (Rutgers University) SDM Center + many other collaborators

  2. My talk. 4443 0146 0000 0000 0000 0a00 0000 0200 0000 0d00 6f70 6f6c 6469 6c61 675f 6972 0064 0000 1600 e18d 0000 0d00 6f74 6f72 6469 6c61 675f 6972 0064 0000 0000 0100 0000 0000 0000 0000 0000 0b00 0000 0100 0000 0900 6f50 6574 746e 6169 006c 0000 0000 0200 0000 0100 0000 0000 0000 0000 0000 0000 0000 0500 5a00 8437 0000 8000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 b8a7 7ace 9aa7 7129 75a7 9a23 7ea7 663f aca7 cfa9 dfa7 8484 f1a7 126c e1a7 e295 cea7 18c9 cea7 569e d8a7 5ef3 dba7 4ed4 d2a7 7e58 c2a7 6e55 b4a7 e39a b7a7 0cb8 d9a7 9b13 07a8 866b 1ba8 3926 1ba8 e61a 0da8 b6dc 05a8 0073 0ba8 f80d 15a8 4afd 16a8 848b 07a8 4b3d e2a7 a65e c0a7 def3 b2a7 eb10 b1a7 23ae bfa7 3709 dba7 f4bc 00a8 81c4 12a8 1470 1fa8 164b 24a8 040c 19a8 8b63 f3a7 48f5 9ba7 c98b 1ea7 13f7 a8a6 4b34 98a6 c438 bca6 5b77 e5a6 2568 08a7 d1d7 2ba7 0e5b 71a7 4229 b7a7 6adc 04a8 09fe 27a8 7c31 35a8 ea72 32a8 a27b 29a8 765c 1da8 ac42 07a8 54a9 cba7 a254 82a7 af6a 2da7 a2d4 3ca7 fadc 8fa7 6088 c9a7 ae7b f1a7 8505 01a8 60ae 05a8 dcfa 09a8 f4a8 0fa8 6ede 1ca8 c9c7 2fa8 08ad 3da8 4c63 37a8 668b 1ba8 5157 eda7 7eb4 bca7 dcb2 bba7 9f0f e5a7 5856 12a8 1b68 2ca8 e050 31a8 e77b 1fa8 0105 02a8 2eca e3a7 e43a eba7 3b92 05a8 18fd 0ea8 6635 02a8 27b7 d1a7 22a1 9aa7 8c3c 5ca7 14ce 26a7 78c1 1ca7 8d1e 3ba7 9c8c 68a7 6ecc 81a7 b11a 81a7 8897 7ea7 7e9f 7aa7 f051 66a7 f663 3da7 5a31 14a7 7ea1 03a7 dc3c 07a7 c0c3 1da7 5a03 54a7 0800 9fa7 de25 e1a7 9698 0aa8 8819 13a8 b56f 11a8 a5e9 0ca8 a8cf 03a8 fe61 e5a7 ccea c1a7 f79a ada7 5eb1 b3a7 c296 cda7 10d5 f1a7 ca0c 08a8 d6f0 13a8 8e94 17a8 d65e 18a8 282d 1ba8 6028 21a8 ce1c 26a8 ae93 29a8 4ecf 2da8 4308 30a8 dc0e 2aa8 4aca 13a8 6efd dea7 9187 9fa7 06cc 90a7 e4f2 aea7 4ef0 d1a7 ac2f d7a7 d635 cba7 b025 cea7 4cae eca7 0074 04a8 8424 fda7 f394 c7a7 9e0a 87a7 5e6b 54a7 fb90 70a7 acc3 8ea7 125e 86a7 7a44 49a7 e75f 20a7 3d38 41a7 b8fb 8da7 df3d bea7 76f5 e5a7 7c61 eda7 d83a c5a7 27af 73a7 9bfa cda6 1032 26a6 805b 79a6 0d07 eca6 9f37 2aa7 2a24 51a7 c0bd 74a7 8ea6 8ca7 66f3 a1a7 25ca b7a7 8d59 c9a7 e6d1 d7a7 0b66 e5a7 1222 f9a7 bc78 08a8 b529 0ea8 f23d 0ca8 fe1f 05a8 5ace fda7 830e e6a7 2cc9 c4a7 9f67 a4a7 8b1e 9ea7 2319 bba7 5024 eba7 a392 0aa8 88a5 11a8 dabf 07a8 d82f e1a7 cd4b b8a7 e61a a8a7 f90b a9a7 9103 9fa7 8bea 81a7 50e5 49a7 0301 4ba7 403b 87a7 d083 ada7 535c bea7 7c76 c3a7 a44e d4a7 58da f7a7 e0a4 0ca8 4312 13a8 ca67 13a8 9c13 0fa8 825c 0aa8 8063 04a8 12fb fea7 fa19 eea7 dfa9 dca7 be74 cfa7 4a3e cea7 ed5f daa7 6762 f2a7 5024 0aa8 a0ec 1ea8 969a 29a8 3ed6 25a8 8963 15a8 4ec3 05a8 da0e eca7 725a c9a7 a614 9fa7 f449 7fa7 369f 77a7 c8cb 92a7 d261 b2a7 0036 caa7 a8fa d7a7 bad4 dda7 fa6d e2a7 3829 e9a7 e8da f6a7 ad90 04a8 4ea1 0fa8 1458 15a8 0cdf 11a8 ae02 fea7 90e8 d4a7 bebc b9a7 f8f1 aca7 5c54 92a7 38dc 47a7 1201 dfa6 4bdc bca6 b675 08a7 4fe9 2aa7 5c85 13a7 b0bd bea6 cdc5 b1a6 c617 36a7 4b80 b4a7 d05e 07a8 ac1e 1fa8 ec82 1fa8 a1f2 17a8 1e2e 13a8 e4ac 12a8 188e 04a8 b833 c2a7 9042 57a7 44df c6a6 f10f aaa6 c63a 1ca7 264e 88a7 b3b1 cea7 b369 08a8 9ef1 1ca8 c601 15a8 9337 f4a7 89f0 c1a7 dab6 b6a7 9c8b daa7 bb6a 09a8 e496 19a8 a13c 09a8 6489 bfa7 4a48 6ba7 fddd 66a7 40dd b2a7 ed0b f0a7 882e f8a7 f3d8 d5a7 c490 b1a7 ed63 9ba7 09ac 78a7 c561 06a7 1133 fca5 5cb3 39a6 d8e9 4ea7 64ac d1a7 c2a6 0ba8 fe47 0fa8 239c 04a8 62c6 00a8 d874 09a8 68a8 16a8 7082 19a8 109f 0da8 68ec f1a7 72c5 cba7 d8f3 bda7 17cc c7a7 9ed9 d5a7 8528 cca7 aae3 aba7 1828 85a7 0efb 69a7 1a55 6aa7 2031 64a7 fd1d 47a7 960a 38a7 2a9f 61a7 d49e 98a7 d49f b4a7 8e3b aea7 0837 8da7 a4c2 54a7 2a36 2ea7 dd1c 40a7 5400 89a7 b859 cba7 b691 07a8 4381 1aa8 d247 13a8 fc77 e8a7 9868 95a7 db78 2fa7 06c0 19a7 5378 60a7 4cd4 a3a7 a3c8 c7a7 f20d d1a7 a47c d1a7 0e2a d6a7 cb01 e1a7 05cc efa7 18b7 01a8 9b45 0ea8 1e9d 17a8 38aa 0da8 6963 d6a7 ac34 80a7 8c1e 09a7 73c7 d0a6 9c9d e2a6 e5a2 e2a6 8d18 c2a6 6e47 afa6 0015 b8a6 2ef2 cda6 ec73 dba6 3e67 d8a6 80c8 c9a6 a6ea dca6 9c7d 24a7 f455 7aa7 a2da 9ba7 946b 95a7 ee38 68a7 7cc3 32a7 06c8 3da7 6088 88a7 b062 c6a7 9aa4 00a8 b2f1 11a8 67df 0ea8 50b9 efa7 9051 aca7 5c24 5aa7 fe2b 01a7 86f3 bda6 818a 08a7 c512 81a7 9a22 c9a7 68ba f3a7 5dd9 f2a7 a462 d9a7 f6a3 c0a7 a064 b3a7 9ffa c2a7 5db7 f1a7 a6b1 11a8 38dc 12a8 62f0 e4a7 6334 84a7 c7a5 03a7 8b1d cda6 220b 13a7 daac 5fa7 9f6f 9ca7 1cb5 caa7 963d eda7 51e3 04a8 6152 14a8 080b 24a8 f8d8 26a8 89ff 0fa8 d325 d5a7 b45b a9a7 a1d3 b0a7 7bf3 caa7 6611 c9a7 2c80 a9a7 36ef 8ea7 6230 99a7 2917 c9a7 0a4e fda7 35e2 0da8 e0fc 13a8 48fa 19a8 8698 1ca8 06b4 10a8 2e1d daa7 a6ba 7ea7 8d08 caa6 c405 a9a5 5a05 01a5 bd80 f8a5 99db 89a6 80cf 02a7 90fc 6da7 8464 b6a7 af89 dca7 422e cfa7 04a5 a4a7 92e0 83a7 1a66 70a7 6cde 6aa7 efa4 6ca7 58f5 93a7 d8eb dfa7 72fd 20a8 a4b6 3fa8 3edc 3ba8 65bb 1ca8 880a eda7 31b2 bca7 ea91 b3a7 52de cda7 46bb efa7 0a44 f8a7 fc91 e3a7 3ee1 cba7 40e0 c9a7 aa9d d6a7 0850 d8a7 e89f c7a7 ed13 b1a7 bdf9 aea7 7a22 bda7 7d97 cda7 d092 c7a7 fffd a7a7 db5e 7ba7 3ac2 56a7 ca94 86a7 3a91 c6a7 be8e fea7 7cb4 00a8 b00d c9a7 5ac6 91a7 168f 91a7 4fb6 d1a7 b471 12a8 f444 2aa8 f6a9 23a8 8ac8 02a8 69b9 bba7 b458 99a7 8030 a6a7 a2ce bea7 0a15 bba7 fc9d aba7 8cf5 bba7 e280 fea7 d26e 29a8 ebd2 3fa8 d20f 2ba8 e88b eea7 5974 81a7 a780 1aa7 f24b 43a7 dd28 a1a7 0b51 e7a7 028e 12a8 2add 2da8 e2d6 41a8 2b5e 47a8 5716 3ea8 8bba 31a8 c8a4 2ba8 9243 2fa8 10da 3ba8 7626 44a8 0b40 40a8 db31 26a8 375d f3a7 2b26 95a7 a351 2ea7 ab9b 06a7 828a 1ca7 6152 5fa7 d61e a7a7 172e eda7 3a50 14a8 18d4 1fa8 3e28 0ea8 3104 cea7 81dd 75a7 4d5f 13a7 f58f 2da7 2e34 9ba7 ae9c f5a7 262e 1da8 895c 2ea8 ea29 32a8 4e1f 2da8 de5e 1aa8 c659 e7a7 389a 86a7 b0b2 f8a6 9006 b3a6 bdd0 17a7 a44f 6fa7 c26e 97a7 364c aba7 0226 c0a7 6c64 d9a7 0f22 e9a7 b6d0 eca7 7d57 eea7 4f99 03a8 1954 1aa8 30f1 2ea8 6ee1 2da8 2462 14a8 b5c8 e9a7 4c2c bba7 0a60 ada7 38f7 bca7 73a0 d6a7 72f5 e6a7 435e e0a7 4a84 d6a7 9e63 e3a7 2cfa 07a8 54c2 20a8 97df 2ea8 665c 28a8 4ed9 11a8 87ec e0a7 c2e5 99a7 4e92 36a7 06bd b9a6 4477 3da6 a27f 51a6 f395 cba6 aecf 2fa7 04a9 77a7 ed59 94a7 ecb8 a2a7 aced b0a7 cad9 c7a7 2e1f dca7 2550 d9a7 5c9d b3a7 b9e6 74a7 b205 11a7 d093 dea6 1ed0 20a7 157f 8da7 bb05 d9a7 8eeb 0ca8 664f 16a8 22c5 08a8 9a2b d3a7 80d0 9ca7 fea1 8ba7 7eb1 aba7 ada8 eea7 d9b7 1da8 d6af 3da8 7ece 52a8 8c66443 0146 0000 0000 0000 0a00 0000 0200 0000 0d00 6f70 6f6c 6469 6c61 675f 6972 0064 0000 1600 e18d 0000 0d00 6f74 6f72 6469 6c61 675f 6972 0064 0000 0000 0100 0000 0000 0000 0000 0000 0b00 0000 0100 0000 0900 6f50 6574 746e 6169 006c 0000 0000 0200 0000 0100 0000 0000 0000 0000 0000 0000 0000 0500 5a00 8437 0000 8000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 b8a7 7ace 9aa7 7129 75a7 9a23 7ea7 663f aca7 cfa9 dfa7 8484 f1a7 126c e1a7 e295 cea7 18c9 cea7 569e d8a7 5ef3 dba7 4ed4 d2a7 7e58 c2a7 6e55 b4a7 e39a b7a7 0cb8 d9a7 9b13 07a8 866b 1ba8 3926 1ba8 e61a 0da8 b6dc 05a8 0073 0ba8 f80d 15a8 4afd 16a8 848b 07a8 4b3d e2a7 a65e c0a7 def3 b2a7 eb10 b1a7 23ae bfa7 3709 dba7 f4bc 00a8 81c4 12a8 1470 1fa8 164b 24a8 040c 19a8 8b63 f3a7 48f5 9ba7 c98b 1ea7 13f7 a8a6 4b34 98a6 c438 bca6 5b77 e5a6 2568 08a7 d1d7 2ba7 0e5b 71a7 4229 b7a7 6adc 04a8 09fe 27a8 7c31 35a8 ea72 32a8 a27b 29a8 765c 1da8 ac42 07a8 54a9 cba7 a254 82a7 af6a 2da7 a2d4 3ca7 fadc 8fa7 6088 c9a7 ae7b f1a7 8505 01a8 60ae 05a8 dcfa 09a8 f4a8 0fa8 6ede 1ca8 c9c7 2fa8 08ad 3da8 4c63 37a8 668b 1ba8 5157 eda7 7eb4 bca7 dcb2 bba7 9f0f e5a7 5856 12a8 1b68 2ca8 e050 31a8 e77b 1fa8 0105 02a8 2eca e3a7 e43a eba7 3b92 05a8 18fd 0ea8 6635 02a8 27b7 d1a7 22a1 9aa7 8c3c 5ca7 14ce 26a7 78c1 1ca7 8d1e 3ba7 9c8c 68a7 6ecc 81a7 b11a 81a7 8897 7ea7 7e9f 7aa7 f051 66a7 f663 3da7 5a31 14a7 7ea1 03a7 dc3c 07a7 c0c3 1da7 5a03 54a7 0800 9fa7 de25 e1a7 9698 0aa8 8819 13a8 b56f 11a8 a5e9 0ca8 a8cf 03a8 fe61 e5a7 ccea c1a7 f79a ada7 5eb1 b3a7 c296 cda7 10d5 f1a7 ca0c 08a8 d6f0 13a8 8e94 17a8 d65e 18a8 282d 1ba8 6028 21a8 ce1c 26a8 ae93 29a8 4ecf 2da8 4308 30a8 dc0e 2aa8 4aca 13a8 6efd dea7 9187 9fa7 06cc 90a7 e4f2 aea7 4ef0 d1a7 ac2f d7a7 d635 cba7 b025 cea7 4cae eca7 0074 04a8 8424 fda7 f394 c7a7 9e0a 87a7 5e6b 54a7 fb90 70a7 acc3 8ea7 125e 86a7 7a44 49a7 e75f 20a7 3d38 41a7 b8fb 8da7 df3d bea7 76f5 e5a7 7c61 eda7 d83a c5a7 27af 73a7 9bfa cda6 1032 26a6 805b 79a6 0d07 eca6 9f37 2aa7 2a24 51a7 c0bd 74a7 8ea6 8ca7 66f3 a1a7 25ca b7a7 8d59 c9a7 e6d1 d7a7 0b66 e5a7 1222 f9a7 bc78 08a8 b529 0ea8 f23d 0ca8 fe1f 05a8 5ace fda7 830e e6a7 2cc9 c4a7 9f67 a4a7 8b1e 9ea7 2319 bba7 5024 eba7 a392 0aa8 88a5 11a8 dabf 07a8 d82f e1a7 cd4b b8a7 e61a a8a7 f90b a9a7 9103 9fa7 8bea 81a7 50e5 49a7 0301 4ba7 403b 87a7 d083 ada7 535c bea7 7c76 c3a7 a44e d4a7 58da f7a7 e0a4 0ca8 4312 13a8 ca67 13a8 9c13 0fa8 825c 0aa8 8063 04a8 12fb fea7 fa19 eea7 dfa9 dca7 be74 cfa7 4a3e cea7 ed5f daa7 6762 f2a7 5024 0aa8 a0ec 1ea8 969a 29a8 3ed6 25a8 8963 15a8 4ec3 05a8 da0e eca7 725a c9a7 a614 9fa7 f449 7fa7 369f 77a7 c8cb 92a7 d261 b2a7 0036 caa7 a8fa d7a7 bad4 dda7 fa6d e2a7 3829 e9a7 e8da f6a7 ad90 04a8 4ea1 0fa8 1458 15a8 0cdf 11a8 ae02 fea7 90e8 d4a7 bebc b9a7 f8f1 aca7 5c54 92a7 38dc 47a7 1201 dfa6 4bdc bca6 b675 08a7 4fe9 2aa7 5c85 13a7 b0bd bea6 cdc5 b1a6 c617 36a7 4b80 b4a7 d05e 07a8 ac1e 1fa8 ec82 1fa8 a1f2 17a8 1e2e 13a8 e4ac 12a8 188e 04a8 b833 c2a7 9042 57a7 44df c6a6 f10f aaa6 c63a 1ca7 264e 88a7 b3b1 cea7 b369 08a8 9ef1 1ca8 c601 15a8 9337 f4a7 89f0 c1a7 dab6 b6a7 9c8b daa7 bb6a 09a8 e496 19a8 a13c 09a8 6489 bfa7 4a48 6ba7 fddd 66a7 40dd b2a7 ed0b f0a7 882e f8a7 f3d8 d5a7 c490 b1a7 ed63 9ba7 09ac 78a7 c561 06a7 1133 fca5 5cb3 39a6 d8e9 4ea7 64ac d1a7 c2a6 0ba8 fe47 0fa8 239c 04a8 62c6 00a8 d874 09a8 68a8 16a8 7082 19a8 109f 0da8 68ec f1a7 72c5 cba7 d8f3 bda7 17cc c7a7 9ed9 d5a7 8528 cca7 aae3 aba7 1828 85a7 0efb 69a7 1a55 6aa7 2031 64a7 fd1d 47a7 960a 38a7 2a9f 61a7 d49e 98a7 d49f b4a7 8e3b aea7 0837 8da7 a4c2 54a7 2a36 2ea7 dd1c 40a7 5400 89a7 b859 cba7 b691 07a8 4381 1aa8 d247 13a8 fc77 e8a7 9868 95a7 db78 2fa7 06c0 19a7 5378 60a7 4cd4 a3a7 a3c8 c7a7 f20d d1a7 a47c d1a7 0e2a d6a7 cb01 e1a7 05cc efa7 18b7 01a8 9b45 0ea8 1e9d 17a8 38aa 0da8 6963 d6a7 ac34 80a7 8c1e 09a7 73c7 d0a6 9c9d e2a6 e5a2 e2a6 8d18 c2a6 6e47 afa6 0015 b8a6 2ef2 cda6 ec73 dba6 3e67 d8a6 80c8 c9a6 a6ea dca6 9c7d 24a7 f455 7aa7 a2da 9ba7 946b 95a7 ee38 68a7 7cc3 32a7 06c8 3da7 6088 88a7 b062 c6a7 9aa4 00a8 b2f1 11a8 67df 0ea8 50b9 efa7 9051 aca7 5c24 5aa7 fe2b 01a7 86f3 bda6 818a 08a7 c512 81a7 9a22 c9a7 68ba f3a7 5dd9 f2a7 a462 d9a7 f6a3 c0a7 a064 b3a7 9ffa c2a7 5db7 f1a7 a6b1 11a8 38dc 12a8 62f0 e4a7 6334 84a7 c7a5 03a7 8b1d cda6 220b 13a7 daac 5fa7 9f6f 9ca7 1cb5 caa7 963d eda7 51e3 04a8 6152 14a8 080b 24a8 f8d8 26a8 89ff 0fa8 d325 d5a7 b45b a9a7 a1d3 b0a7 7bf3 caa7 6611 c9a7 2c80 a9a7 36ef 8ea7 6230 99a7 2917 c9a7 0a4e fda7 35e2 0da8 e0fc 13a8 48fa 19a8 8698 1ca8 06b4 10a8 2e1d daa7 a6ba 7ea7 8d08 caa6 c405 a9a5 5a05 01a5 bd80 f8a5 99db 89a6 80cf 02a7 90fc 6da7 8464 b6a7 af89 dca7 422e cfa7 04a5 a4a7 92e0 83a7 1a66 70a7 6cde 6aa7 efa4 6ca7 58f5 93a7 d8eb dfa7 72fd 20a8 a4b6 3fa8 3edc 3ba8 65bb 1ca8 880a eda7 31b2 bca7 ea91 b3a7 52de cda7 46bb efa7 0a44 f8a7 fc91 e3a7 3ee1 cba7 40e0 c9a7 aa9d d6a7 0850 d8a7 e89f c7a7 ed13 b1a7 bdf9 aea7 7a22 bda7 7d97 cda7 d092 c7a7 fffd a7a7 db5e 7ba7 3ac2 56a7 ca94 86a7 3a91 c6a7 be8e fea7 7cb4 00a8 b00d c9a7 5ac6 91a7 168f 91a7 4fb6 d1a7 b471 12a8 f444 2aa8 f6a9 23a8 8ac8 02a8 69b9 bba7 b458 99a7 8030 a6a7 a2ce bea7 0a15 bba7 fc9d aba7 8cf5 bba7 e280 fea7 d26e 29a8 ebd2 3fa8 d20f 2ba8 e88b eea7 5974 81a7 a780 1aa7 f24b 43a7 dd28 a1a7 0b51 e7a7 028e 12a8 2add 2da8 e2d6 41a8 2b5e 47a8 5716 3ea8 8bba 31a8 c8a4 2ba8 9243 2fa8 10da 3ba8 7626 44a8 0b40 40a8 db31 26a8 375d f3a7 2b26 95a7 a351 2ea7 ab9b 06a7 828a 1ca7 6152 5fa7 d61e a7a7 172e eda7 3a50 14a8 18d4 1fa8 3e28 0ea8 3104 cea7 81dd 75a7 4d5f 13a7 f58f 2da7 2e34 9ba7 ae9c f5a7 262e 1da8 895c 2ea8 ea29 32a8 4e1f 2da8 de5e 1aa8 c659 e7a7 389a 86a7 b0b2 f8a6 9006 b3a6 bdd0 17a7 a44f 6fa7 c26e 97a7 364c aba7 0226 c0a7 6c64 d9a7 0f22 e9a7 b6d0 eca7 7d57 eea7 4f99 03a8 1954 1aa8 30f1 2ea8 6ee1 2da8 2462 14a8 b5c8 e9a7 4c2c bba7 0a60 ada7 38f7 bca7 73a0 d6a7 72f5 e6a7 435e e0a7 4a84 d6a7 9e63 e3a7 2cfa 07a8 54c2 20a8 97df 2ea8 665c 28a8 4ed9 11a8 87ec e0a7 c2e5 99a7 4e92 36a7 06bd b9a6 4477 3da6 a27f 51a6 f395 cba6 aecf 2fa7 04a9 77a7 ed59 94a7 ecb8 a2a7 aced b0a7 cad9 c7a7 2e1f dca7 2550 d9a7 5c9d b3a7 b9e6 74a7 b205 11a7 d093 dea6 1ed0 20a7 157f 8da7 bb05 d9a7 8eeb 0ca8 664f 16a8 22c5 08a8 9a2b d3a7 80d0 9ca7 fea1 8ba7 7eb1 aba7 ada8 eea7 d9b7 1da8 d6af 3da8 7ece 52a8 8c664443 0146 0000 0000 0000 0a00 0000 0200 0000 0d00 6f70 6f6c 6469 6c61 675f 6972 0064 0000 1600 e18d 0000 0d00 6f74 6f72 6469 6c61 675f 6972 0064 0000 0000 0100 54c2 20a8 97df 2ea8 665c 28a8 4ed9 11a8 87ec e0a7 c2e5 99a7 4e92 36a7 06bd b9a6 4477 3da6 a27f 51a6 f395 cba6 aecf 2fa7 04a9 77a7 ed59 94a7 ecb8 a2a7 aced b0a7 cad9 c7a7 2e1f dca7 2550 d9a7 5c9d b3a7 b9e6 74a7 b205 11a7 d093 dea6 1ed0 20a7 157f 8da7 bb05 d9a7 8eeb 0ca8 664f 16a8 22c5 08a8 9a2b d3a7 80d0 9ca7 fea1 8ba7 7eb1 aba7 ada8 eea7 d9b7 1da8 d6af 3da8 7ece 52a8 8c664443 0146 0000 0000 0000 0a00 0000 0200 0000 0d00 6f70 6f6c 6469 6c61 675f 6972 0064 0000 1600 e18d 0000 0d00 6f74 6f72 6469 6c61 675f 6972 0064 0000 0000 0100 08a8 b529 0ea8 f23d 0ca8 fe1f 05a8 5ace fda7 830e e6a7 2cc9 c4a7 9f67 a4a7 8b1e 9ea7 2319 bba7 5024 eba7 a392 0aa8 88a5 11a8 dabf 07a8 d82f e1a7 cd4b b8a7 e61a a8a7 f90b a9a7 9103 9fa7 8bea 81a7 50e5 49a7 0301 4ba7 403b 87a7 d083 ada7 535c bea7 7c76 c3a7 a44e d4a7 58da f7a7 e0a4 0ca8 4312 13a8 ca67 13a8 9c13 0fa8 825c 0aa8 8063 04a8 12fb fea7 fa19 eea7 dfa9 dca7 be74 cfa7 4a3e cea7 ed5f daa7 6762 f2a7 5024 0aa8 a0ec 1ea8 969a 29a8 3ed6 25a8 8963 15a8 4ec3 05a8 da0e eca7 725a c9a7 a614 9fa7 f449 7fa7 369f 77a7 c8cb 92a7 d261 b2a7 0036 caa7 a8fa d7a7 bad4 dda7 fa6d e2a7 3829 e9a7 e8da f6a7 ad90 04a8 4ea1 0fa8 1458 15a8 0cdf 11a8 ae02 fea7 90e8 d4a7 bebc b9a7 f8f1 aca7 5c54 92a7 38dc 47a7 1201 dfa6 4bdc bca6 b675 08a7 4fe9 2aa7 5c85 13a7 b0bd bea6 cdc5 b1a6 c617 36a7 4b80 b4a7 d05e 07a8 ac1e 1fa8 ec82 1fa8 a1f2 17a8 1e2e 13a8 e4ac 12a8 188e 04a8 b833 c2a7 9042 57a7 44df c6a6 f10f aaa6 c63a 1ca7 264e 88a7 b3b1 cea7 b369 08a8 9ef1 1ca8 c601 15a8 9337 f4a7 89f0 c1a7 dab6 b6a7 9c8b daa7 bb6a 09a8 e496 19a8 a13c 09a8 6489 bfa7 4a48 6ba7 fddd 66a7 40dd b2a7 ed0b f0a7 882e f8a7 f3d8 d5a7 c490 b1a7 ed63 9ba7 09ac 78a7 c561 06a7 1133 fca5 5cb3 39a6 d8e9 4ea7 64ac d1a7 c2a6 0ba8 fe47 0fa8 239c 04a8 62c6 00a8 d874 09a8 68a8 16a8 7082 19a8 109f 0da8 68ec f1a7 72c5 cba7 d8f3 bda7 17cc c7a7 9ed9 d5a7 8528 cca7 aae3 aba7 1828 85a7 0efb 69a7 1a55 6aa7 2031 64a7 fd1d 47a7 960a 38a7 2a9f 61a7 d49e 98a7 d49f b4a7 8e3b aea7 0837 8da7 a4c2 54a7 2a36 2ea7 dd1c 40a7 5400 89a7 b859 cba7 b691 07a8 4381 1aa8 d247 13a8 fc77 e8a7 9868 95a7 db78 2fa7 06c0 19a7 5378 60a7 4cd4 a3a7 a3c8 c7a7 f20d d1a7 a47c d1a7 0e2a d6a7 cb01 e1a7 05cc efa7 18b7 01a8 9b45 0ea8 1e9d 17a8 38aa 0da8 6963 d6a7 ac34 80a7 8c1e 09a7 73c7 d0a6 9c9d e2a6 e5a2 e2a6 8d18 c2a6 6e47 afa6 0015 b8a6 2ef2 cda6 ec73 dba6 3e67 d8a6 80c8 c9a6 a6ea dca6 9c7d 24a7 f455 7aa7 a2da 9ba7 946b 95a7 ee38 68a7 7cc3 32a7 06c8 3da7 6088 88a7 b062 c6a7 9aa4 00a8 b2f1 11a8 67df 0ea8 50b9 efa7 9051 aca7 5c24 5aa7 fe2b 01a7 86f3 bda6 818a 08a7 c512 81a7 9a22 c9a7 68ba f3a7 5dd9 f2a7 a462 d9a7 f6a3 c0a7 a064 b3a7 9ffa c2a7 5db7 f1a7 a6b1 11a8 38dc 12a8 62f0 e4a7 6334 84a7 c7a5 03a7 8b1d cda6 220b 13a7 daac 5fa7 9f6f 9ca7 1cb5 caa7 963d eda7 51e3 04a8 6152 14a8 080b 24a8 f8d8 26a8 89ff 0fa8 d325 d5a7 b45b a9a7 a1d3 b0a7 7bf3 caa7 6611 c9a7 2c80 a9a7 36ef 8ea7 6230 99a7 2917 c9a7 0a4e fda7 35e2 0da8 e0fc 13a8 48fa 19a8 8698 1ca8 06b4 10a8 2e1d daa7 a6ba 7ea7 8d08 caa6 c405 a9a5 5a05 01a5 bd80 f8a5 99db 89a6 80cf 02a7 90fc 6da7 8464 b6a7 af89 dca7 422e cfa7 04a5 a4a7 92e0 83a7 1a66 70a7 6cde 6aa7 efa4 6ca7 58f5 93a7 d8eb dfa7 72fd 20a8 a4b6 3fa8 3edc 3ba8 65bb 1ca8 880a eda7 31b2 bca7 ea91 b3a7 52de cda7 46bb efa7 0a44 f8a7 fc91 e3a7 3ee1 cba7 40e0 c9a7 aa9d d6a7 0850 d8a7 e89f c7a7 ed13 b1a7 bdf9 aea7 7a22 bda7 7d97 cda7 d092 c7a7 fffd a7a7 db5e 7ba7 3ac2 56a7 ca94 86a7 3a91 c6a7 be8e fea7 7cb4 00a8 b00d c9a7 5ac6 91a7 168f 91a7 4fb6 d1a7 b471 12a8 f444 2aa8 f6a9 23a8 8ac8 02a8 69b9 bba7 b458 99a7 8030 a6a7 a2ce bea7 0a15 bba7 fc9d aba7 8cf5 bba7 e280 fea7 d26e 29a8 ebd2 3fa8 d20f 2ba8 e88b eea7 5974 81a7 a780 1aa7 f24b 43a7 dd28 a1a7 0b51 e7a7 028e 12a8 2add 2da8 e2d6 41a8 2b5e 47a8 5716 3ea8 8bba 31a8 c8a4 2ba8 9243 2fa8 10da 3ba8 7626 44a8 0b40 40a8 db31 26a8 375d f3a7 2b26 95a7 a351 2ea7 ab9b 06a7 828a 1ca7 6152 5fa7 d61e a7a7 172e eda7 3a50 14a18d4 1fa8 3e28 0ea8 How about I “stream” my talk slide by slide? Thank You

  3. Outline • Scientific Data Management Issues. • Leadership Class Computing/Distributed Computing. • Scientific Data Management. • Workflow Automation. • Code coupling. • Ubiquitous and Transparent Data access. • Visualization Issues. • Conclusion

  4. DOE Computational Resources • What is leadership class computing? • 5K + Processors. • TB’s of data. • 100+ hours/simulation. • Good Science. • 22 Projects • What can you do with 80K Processors for 10 days? (19.2M CPU hours) • ORNL Cray XT3 • 5200 Procs, 25 TFlops, 10 TB RAM, 40TB Disk • 20,800 Procs, 100TFlop, 40TB RAM • ORNL Cray X1E 18 Tflops • NERSC IBM SP3 10 Tflops • NERSC IBM SP5 7.4 Tflops • NERSC Jacquard 3.1 Tflops • ANL BlueGene 5.7 Tflop • ORNL SGI 2TB RAM, 256 Procs • ORNL cluster (infiniband) 0.7Tflop • LANL Issues RFP For Petaflop Supercomputer Distributed Resources

  5. HPC Market Trends Report of the High-End Computing Revitalization Task Force, 2004 (Office of Science and Technology Policy, Executive Office of the President) “Make high-end computing easier and more productive to use. Emphasis should be placed on time to solution, the major metric of value to high-end computing users… A common software environment for scientific computation encompassing desktop to high-end systems will enhance productivity gains by promoting ease of use and manageability of systems.” Microsoft 2004 Systems 2004-9 CAGR Capability, Enterprise$1M+ 1,167 4.2% Divisional$250K-$1M 3,915 5.7% Top Challenges to Implementing Clusters(IDC 2004, N=229) Departmental$50-250K 22,712 7.7% Workgroup<$50K 127,802 13.4% <$250K – 97% of systems, 52% of revenue In 2004 clusters grew 96% to 37% by revenueAverage cluster size 10-16 nodes Source: IDC, 2005

  6. More Flops, More Data. • GTC is now producing field data, restart data, particle data. • Simulations producing TB’s of information. • Data sharing becoming a bigger issue for larger simulations. • Fast data access is becoming increasingly important! • Fast data analysis is becoming important! • 0.3 seconds to compute a PDF on 4,800P • Computing this in serial = 24 minutes! ParallelI/O

  7. Fusion Lab (MIT, NYU, PU,PPPL, ORNL, UCI) • Based on Planet Lab (Peterson PU) • http://www.planet-lab.org • 672 locations (including PKU). • Concept of virtualization. • Can put transp + Globus (fusiongrid) on a slice, other slice can be TSC, …. • Slices are development & production environments. • Slices can be MDS+ servers. • Slices can be Logistical Networking servers with MDS+. • Users access services from “local” FL nodes. • Services run on slices which are replicated from the main server for a slice. • Add high-level of security with OTP. Virtualization from Dell

  8. MDS+ & Data Management • MDS+ is the de facto standard in fusion experimental data acquisition system. • MDS+ was a great start, but • Data is not meant to be replicated. • 1 Shot is associated with 1 location. • Where is the parallelism? • No hyperslabbing. • Doesn’t support large data! • Made for short pulses. (Doesn’t support data streaming) • Standard interfaces to data (simulation/experimental) is essential for ITER.

  9. What is Scientific Data Management? • Data Management: “The control of data handling operations, such as acquisition, analysis, translation, coding, storage, retrieval, and distribution of data, but not necessarily the generation and use of data.” (ATIS). • Scientific Data Management : Science objectives Hypothesis Knowledge & Information Data archives Distributed (Grid) computing Data ownership/stewardship Simulation Experimentation Observation SDM Scientific models Data Analysis Data Exploration Data Visualization Scientific data formats Metadata Data provenance & Lineage Data processing pipelines

  10. What is a Scientific Workflow (SWF)? • Model the way scientists work with their data and tools • Mentally coordinate data export, import, analysis via software systems • Scientific workflows emphasize data flow (≠ business workflows) • Metadata (incl. provenance info, semantic types etc.) is crucial for automated data ingestion, data analysis, … • Goals: • SWF automation, • SWF, component reuse • SWF design & documentation  making scientists’ data analysis and management easier!

  11. Why not just a Python script? • Users who might be able to define, reuse, modify, specialize WFs might not be able to do the same for Python scripts • But wait, there’s more: • Modular reuse • Debugging and monitoring of WF execution • Automated Provenance Mgmt • Semantic types • From integrated WF modeling (ER + dataflow + co-registrations) to execution, optimization, archival …

  12. Workflow Requirements (Kepler-HPC) • Cray XT3 is our “target” platform. • 5212 (20K) compute nodes (catamount), 82 service nodes (login, I/O) • LUSTRE parallel filesystem: >5GB/sec r/w. • ‘Upgrade ‘late 2008’ to PFlop! Cray Baker. • Simulations run for long times. Machine can crash at any time. • Workflow must run on a server. • Automate Job submission, start daemons. • Large Data Movement (Ubiquitous and Transparent data access) • Code coupling. (Fault tolerant, low latency, NXM data movement.) • Runtime monitoring. (Web Portals to dashboards) • User definable (C/F90) Analysis code in workflow, via wizards.

  13. Commercial & Open Source Scientific Workflows Kensington Discovery Edition from InforSense Triana Microsoft Workflow Foundation Taverna

  14. Kepler workflow

  15. Code Coupling/Workflow Automation GTC runs on Tflop – Pflop Cray Large data analysis End-to-end system 160p, M3D runs on 64P Monitoring routines here 40Gbps Data replication User monitoring Data archiving Data replication Post processing

  16. Adaptive Workflow Automation Problem:Unique requirements of scientific WFs • Moving large volumes between modules • Tightly-coupled efficient data movement • Wide-area loosely coupled data movement • Specification of granularity-based iteration • e.g. In spatial-temporal simulations – a time step is a “granule” • Support for data transformation • complex data types (including file formats, e.g. netCDF, HDF) • Dynamic steering of workflow by user • Dynamic user examination of results • Adaptive workflow automation • detect and dynamically respond to changing requirements, state and execution in the workflow.

  17. Use workflow framework to couple codes in CPES Project. • Codes have different requirements. • Run XGC on 2,048 processors on Cray XT3. • Run M3D on 64 processors on end-to-end cluster. • 40Gbs link from machines. • Requirement is to move data from machines as fast as possible. (current times suggest 1 second is good enough). • Code coupling is lose, but requires fast NXM data movement. • Use Sandia Portals + sockets to move data off the XT3. • Workflow (Kepler) monitors this data movement, and allows users to visualize the coupling data. • Need to define clear interfaces (schema/data model) for actors.

  18. Parallel Data Redistribution A key aspect of the coupling problem is parallel data redistribution, which refers to the transfer of data from a parallel program running on M processors to another parallel program running on N processors Basic Approaches Component-based PRMI-based Component-based approach: Seine, MCT, InterComm (MetaChaos), PAWS, CUMULVS, etc. PRMI-based approach: DCA, XCAT, SciRun, etc.

  19. Monitoring Data through web portal • Users can interactively choose which variables to see. • Compare one simulation to another, or to experimental data. • Users update workflow during the simulation • Users “register” variables, and monitor interactively.

  20. Ubiquitous and Transparent Data Access • Logistical Networking being set for standard mechanism for accessing files. (Need fast data movement!) • http://loci.cs.utk.edu/ • Data gets replicated from LN to collaborating sites. • Share an eXnode which points to depots where actual data lives. • Data is replicated over many locations, (including tape). • Workflow keeps track of data movements, reports to logs on webpage. Tapes IDAVE e.g. HPSS Terabytes IDAVE Disks P bytes Disks IDAVE IDAVE

  21. Visualization • Most common is 1D visualization (x vs. f(x)). • Gnuplot, GRACE, IDL (GDL), Matlab, …. • 2D visualization. • IDL (GDL), GRACE, … • 3D Visualization • Many packages (AVS/Express, Ensight-Gold, OpenDX, ParaView, SciRun, Visit, VTK) • Parallel rendering, remote visualization, parallel analysis. • Feature Tracking. • High Dimensional Visualization (5D+time phase space).

  22. 3D Parallel Visualization • Visit (free from LLNL). • http://www.llnl.gov/visit • Ensight-Gold • http://www.ensight.com • Paraview • http://www.paraview.org • www.kitware.com • AVS/Express-PST • www.avs.com

  23. Conclusion • Computing Resources are distributed. • Collaborations in computing are becoming common. [although collaboration is small teams]. • Data Replication is a necessary part of computing in the modern world. • Metadata is crucial for longevity of data. • Parallel Visualization is important for large data! • Feature Identification/Tracking crucial for scientific discover. • Scientific Workflows are the crucial ‘framework’ for Scientific Discovery through Advanced Computing. • Thanks to all of my collaborators.

More Related