1 / 27

Front-end node

Front-end node. B ack-end node. Installation stage. Authenticator As Root. Job Scheduler. 1. Request. 2 . Spawn. 2 . Spawn. Job front-end As User1. 3 . Conversation. Job back-end As User1. 1. Request. Job front-end As User2. Job back-end As User2. 3 . Conversation. Backend node.

amena
Download Presentation

Front-end node

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Front-end node Back-end node Installation stage Authenticator As Root Job Scheduler 1. Request 2. Spawn 2. Spawn Job front-end As User1 3. Conversation Job back-end As User1 1. Request Job front-end As User2 Job back-end As User2 3. Conversation

  2. Backend node Frontend node Job Backend Process Running As A Job Frontend Process Running As A (First time) 3. Conversation 1. SSH 2. Spawn Job Launcher Daemon Job Scheduler Daemon 1. MAC message 2. Spawn Job Backend Process Running As A Job Frontend Process Running As A (Second time) 3. Conversation

  3. Job frontend process Job Scheduler Daemon Nn = Number of nodes Np = Number of processes Nt = Threads per process Request Job (username, Nn, Np, Nt) Cancel Job (errmsg) If insufficient resources Assign Job number (errmsg) If sufficient resources Assign required number of backend nodes. If a backend daemon is already running on the backend node, send “Assign Backend Daemon”. Else send “Create Backend Daemon” Assign Backend Daemon Create Backend Daemon Renew lease () Periodically while job is in progress Renew lease () Periodically while job is in progress Job finished () When job finishes Cancel Job (errmsg) If user cancels job Cancel Job (errmsg) If error in Job Scheduler, or admin cancels job Backend failed (name) If frontend detects backend has failed

  4. Server Client Time SSH_MSG_KEXINIT SSH_MSG_NEWKEYS SSH-transport SSH_MSG_SERVICE_REQUEST SSH_MSG_SERVICE_ACCEPT SSH_MSG_USERAUTH_REQUEST SSH-userauth SSH_MSG_USERAUTH_SUCCESS SSH_MSG_CHANNEL_OPEN SSH_MSG_CHANNEL_OPEN_CONFIRMATION . . . SSH-connection SSH_MSG_CHANNEL_CLOSE SSH_MSG_CHANNEL_CLOSE

  5. Server Client Time SSH_MSG_USERAUTH_REQUEST SSH_MSG_USERAUTH_FAILURE SSH_MSG_USERAUTH_REQUEST SSH_MSG_USERAUTH_INFO_REQUEST SSH_MSG_USERAUTH_INFO_RESPONSE SSH_MSG_USERAUTH_SUCCESS

  6. Server Client Time Establish TCP connection SSH_protoversion_softwareversion SSH_protoversion_softwareversion SSH_MSG_KEXINIT SSH_MSG_KEXINIT SSH_MSG_NEWKEYS SSH_MSG_NEWKEYS SSH_MSG_SERVICE_REQUEST SSH_MSG_SERVICE_ACCEPT

  7. Server Client Time Establish Authenticated Transport Layer Connection SSH_MSG_CHANNEL_OPEN SSH_MSG_CHANNEL_OPEN_CONFIRMATION SSH_MSG_CHANNEL_DATA SSH_MSG_CHANNEL_DATA . . . SSH_MSG_CHANNEL_CLOSE

  8. SSH protocol layering Connection Layer Protocol User Authentication Layer Protocol SSH Layers Transport Layer Protocol TCP Time IP

  9. Job frontend process Job Backend Daemon Challenge (random number A) Initiate authentication Authenticate (random number B, MAC1) One side authentication Authenticate ( MAC3, Job execution parameter) The other side authentication

  10. Job frontend process Job backend process mcg = middleware channel group wcg = world channel group fcg = frontend channel group Backend ready (rank, mcg, wcg, fcg) Commence job (mcg[], wcg[], fcg[], properties, mainclass, args) Sent to each backend when all backends are ready Request resource (name) To load a class from the user's program Report resource (name, bytecodes) Buffer goes to job frontend which writes it to file or stdout/stderr Write file (fd, buf, off, len) Renew lease () Periodically while job is in progress Renew lease () Periodically while job is in progress Backend finished () When main program finishes Job finished () Sent to each backend when all backend main programs have finished Cancel Job (errmsg) If job aborts Cancel Job (errmsg) If error in job backend

  11. Job backend Daemon Job Scheduler Daemon Ready (username, ip, port) When a job backend daemon is created Renew Lease(username, ip, port) Periodically while job is in progress Terminate (username, ip, port) When the job backend Daemon is about to terminate

  12. 6 Job Frontend Process Backend Process 7 5 1 2 3 Job Scheduler Daemon Backend Daemon 4

  13. RIT CS Paranoia 32-Processor Cluster Refresh Mon Feb 08 16:39:49 EST 2010 -- Parallel Java v20100121 Nodes Jobs

  14. Welcome, xxh2229 RIT CS Paranoia 32-Processor Cluster Logout Refresh Mon Feb 08 16:39:49 EST 2010 -- Parallel Java v20100121 Nodes Jobs upload Required CPUs Run Args

  15. Welcome, administrator RIT CS Paranoia 32-Processor Cluster Logout Refresh Mon Feb 08 16:39:49 EST 2010 -- Parallel Java v20100121 Nodes Jobs

  16. Frontend node Job Frontend Process Running As A Client A Spawn (SSH) Submit a job Job Scheduler Daemon Web Browse Secure Connection Cancel a Job Cancel (SSH) Job Frontend Process Running As A

  17. Backend nodes Frontend node Job Scheduler Daemon Web Browse Cluster status Spawn (SSH) stdin Job Backend Process Job Frontend Process Terminal Job result stdout stderr

  18. Backend nodes Frontend node Parent Process MPI_Init MPI_Com_spawn MPI_Finalize Child Process MPI_Init MPI_Finalize Spawn Message result

  19. Backend nodes Frontend node Status Job Scheduler Daemon Job Backend Daemon Web Browse Cluster status MAC message SSH Spawn stdin Job Frontend Process Terminal Job Job Backend Process result stdout stderr

  20. Client A Frontend node Submit a job Web Browse Secure Connection Web Server Cancel a Job SSH Agent

  21. Frontend node Web Server SSH Protocol SSH Agent SSH Daemon

  22. Web Server SessionManager SSL HttpRequest HttpServer SSL HttpResponse SSH Agent

  23. User B’s Process Web Server Submit Jobs User A Can not spawn User A’s Process Job Frontend Process

  24. User B’s Process Web Server Submit Jobs SSH Agent User A User A’s credential Root’s Process User A’s Process Spawn Job Frontend Process SSH Daemon

  25. Message M Authentication Key HMAC-SHA-256 Message M HMAC(M)

  26. Receiver Proxy Message (A,m1) m1(){ … } Channel Call A.m1() Object A Object B

  27. Know Hosts JSch JSch SSH Daemon Username Password Session Channel Channel Command …

More Related