Distributable Threads – An overview

Distributable Threads – An overview Venkita Subramonian Nov 20 2003 CS523 Fall 2003

Outline • CORBA and RTCORBA overview • Dynamic scheduling in RTCORBA • Distributable threads model • Preliminary implementation

Overview of CORBA • Common Object Request Broker Architecture (CORBA) • A family of specifications • OMG is the standards body • Over 800 companies • CORBA defines interfaces, not implementations • It simplifies development of distributed applications by automating/encapsulating • Object location • Connection & memory mgmt. • Parameter (de)marshaling • Event & request demultiplexing • Error handling & fault tolerance • Object/server activation • Concurrency • Security • CORBA shields applications from heterogeneous platform dependencies • e.g., languages, operating systems, networking protocols, hardware

RTCORBA 1.0 • RT CORBA adds QoS control to regular CORBA to improve application predictability, e.g., • Bounding priority inversions & • Managing resources end-to-end • Policies & mechanisms for resource configuration/control in RT-CORBA include: • Processor Resources • Thread pools • Priority models • Portable priorities • Communication Resources • Protocol policies • Explicit binding • Memory Resources • Request buffering • Limitations of CORBA • Lack of QoS specifications • Lack of QoS enforcement • Lack of real-time programming features • Lack of performance optimizations

RTCORBA 2.0 • RTCORBA 1.0 addresses static scheduling of resources • RTCORBA 2.0 intended for dynamically scheduled distributed real-time systems • Originally referred to as Dynamic Real-Time CORBA • Addresses the need to spread a real-time application across multiple physically dispersed computing nodes • Addresses end-to-end scheduling • Introduces a Distributable thread abstraction for an end-to-end computation

What is a Distributable Thread? • Based on the "distributed (trans-node) thread" abstraction first introduced in the Alpha distributed real-time OS kernel • An Architectural Overview Of The Alpha Real-Time Distributed Kernel (1993) - Raymond K. Clark, E. Douglas Jensen, Franklin D. Reynolds • Also influential in the Distributed Real-Time Specification for Java, proposed in Sun Java Specification Request JSR-000050 • A programming model abstraction for end-to-end control flow • A thread that can execute operations in objects without regard for physical node boundaries • A program may consist of multiple concurrent distributable threads

How is this relevant to CS523? • Extending concept of thread scheduling end-to-end • The distributable thread (not a local thread) is the schedulable entity • Pluggable dynamic schedulers • Scheduler interfaces standardized by RTCORBA 2.0 • Similar to the hierarchical scheduling framework in our project • Scheduling Parameter interfaces (some of them atleast) defined by RTCORBA 2.0 • Similar to concept of rms_attr_t, edf_attr_t (in our project)

<GUID, TID> <GUID, TID> Distributed system model DT carries scheduling parameters with it Host 1 Host 2 RTCORBA 2.0 Scheduler RTCORBA 2.0 Scheduler DT Physical binding of a single DT with two OS threads

DTs within a single node Scheduler invocation points • Programming model very similar to transaction programming model in databases begin_transaction() Do atomic stuff end_transaction()

DTs spanning multiple endsystems BSS – begin_scheduling_segment ESS – end_scheduling_segment Application call Interceptor call Scheduling parameters serialized at ORB interceptor points for migration

Distributable thread implementation • Vast area of research • Several issues to be addressed • Mutual exclusion and TSS semantics with same GUID mapped to two different TIDs • Scheduling parameter propagation • Heterogeneous schedulers • Focus of this discussion • Scheduler implementation

GUID=1, importance=1 GUID=1, importance=1 GUID=2, importance=2 GUID=2, importance=2 Condition Variable based implementation Middleware Scheduler/Dispatcher bss uss broadcast Most eligible Condition variable wait bss

CV approach - variation • Instead of a single CV, associate a CV with each thread • Each thread waits on its own CV • Only most eligible thread is signalled instead of a broadcast to the single CV • Less number of context switches • More resource (CV) consumption

Priority based approach • Assign thread priority based on scheduling parameters • Scheduling algorithm determines eligibility criterion • OS dispatcher takes care of actual dispatching of threads • Middleware Scheduler just maps from scheduling parameters to OS priority • Only coarse mapping from scheduling parameter to OS priority • T1 Deadline = 100 ticks from now – Assign priority 5 • T2 Deadline = 95 ticks from now – Assign priority 6 • T3 Deadline = 90 ticks from now – What priority do we assign? sched-params Middleware Scheduler OS prio OS Dispatcher

Solution – Use priority levels • Four priority levels • Scheduler thread runs at executive level • Active and Inactive levels to control thread execution based on eligibility • Blocked priority level to “park” threads that have blocked • This ensures that the thread gets a chance to be rescheduled when it unblocks

Issues with Level based approach • Threads have to explicitly inform the scheduler that they are going to block • One solution is to write wrappers for each blocking function • Very cumbersome • Error-prone • Scheduler activation concept might be useful • Portability of middleware code an issue • Idea of a “Block Catcher” thread (something like an idle thread) • The “Block catcher” thread just signals the scheduler thread to wakeup

CV vs Priority implementations

Conclusion • Lot of concepts from CS523 found applicable to this implementation • Implementation done as part of Kokyu framework and made available as part of ACE/TAO source. • A fertile area for new research • Future work • Application in Real-world scenarios (DARPA PCES project) • Group Scheduling abstraction (KU – Dr. Doug Niehaus) • Useful Links • http://www.omg.org/technology/documents/formal/RT_dynamic.htm • http://www.real-time.org/realtimecobra.htm • http://www.cs.wustl.edu/~schmidt/

Distributable Threads – An overview