1 / 40

Transparent migration and adaptation in a Graphical User Interface toolkit

Transparent migration and adaptation in a Graphical User Interface toolkit. Donatien Grolaux Thesis Public Defense September 4 h , 2007. Context. Ubiquitous computing UIs could take advantage of the ubiquity of devices Migration Adaptation

elisabeth-newton
Download Presentation

Transparent migration and adaptation in a Graphical User Interface toolkit

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Transparent migration and adaptation in a Graphical User Interface toolkit DonatienGrolaux Thesis Public Defense September 4h, 2007

  2. Context • Ubiquitous computing • UIs could take advantage of the ubiquity of devices • Migration • Adaptation • User centric view where the application follows the user in its mobility

  3. Demo

  4. Purpose • Main goal: find a way to provide migration and adaptation support to applications • Introduce as little extra complexity as possible • How could that be achieved ? • Ad hoc solution for each application • Do not touch the application, the migration/adaptation is provided externally • Introduce extra models for the migration/adaptation and have them executed at runtime • Provide the support for migration/adaptation in the toolbox that builds the UI to provide migration/adaptation • Create a new toolbox from scratch ? • Extend an existing toolbox No No No Yes No Yes

  5. Approach of this thesis • Give a tool to the developers that enables them to create multiplatform migratable & adaptable UIs. • While minimizing the impact on the interactive applications implementation • This tool is a graphical toolkit • That presents itself as a usual graphical toolkit (AWT, GTk, Tcl/Tk…) • Concepts of widgets, user events… • Multiplatform support • Migration support • Adaptation support • Some more features

  6. Approach of this thesis • Do not try to solve discovery problems, context awareness, meta-UI, tasks specification… • It is up to the application to solve these issues • However the tool allows for simple solutions

  7. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Overview • Goal: create applications supporting migratable and adaptable user interfaces. • Means: a graphical toolkit supporting these features, in an easy to use way. • Using a toolkit-agnostic middleware designed to provide these extensions • By extending an existing toolkit • Tools: the Oz programming language. • Support for distributed applications • Multi-paradigm programming language (O-O, symbolic programming, functional programming…)

  8. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Application • This thesis allows writing applications whose UI: • Is dynamically migratable • Is dynamically adaptable

  9. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Application • Low development cost • Extend classical graphical toolkit approach • Give control to the application itself • Dynamicity • Fault tolerance • Extensible

  10. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Mozart • Multi-paradigm programming language • Strongly dynamically typed • Procedural, Object-Orientation • Functional, Logic and Symbolic programming • Transparent distribution support • Distributed network protocols are attached automatically to some of the native data types of Oz

  11. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP EBL/Tk • Toolkit developped for this thesis • Complete graphical UI toolkit with widgets, fonts, windows, dialog boxes… • Use a mixed declarative/imperative approach • Declarative approach for creating UIs • Imperative approach for running UIs UI={Build window( name:window selector( name:selector items:["Radiobuttons" "Listbox" "Menu"] curselection:1 text:"Selector" action:proc{$} R={UI.selector get(curselection:$)} in {UI.selector setContext((default#listbox#menu).R)} end))} {UI.window show}

  12. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP EBL/Tk Adaptation • Adaptation is a special configuration parameter UI={Build window( name:window selector( name:selector items:["Radiobuttons" "Listbox" "Menu"] curselection:1 text:"Selector" action:proc{$} R={UI.selector get(curselection:$)} in {UI.selector setContext((default#listbox#menu).R)} end))} {UI.window show}

  13. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP EBL/Tk Migration • Capability based migration • Capability represents the authority to migrate a widget • Capability contains the Internet address of the widget • All widgets (except toplevel widgets) have the migration capability {Offer {UI.selector getRef($)}} UI2={Build window(name:window)} {UI2.window show} {UI2.window display({Obtain})}

  14. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Transparent Migration and Adaptation • Transparent migration • Widgets are used by the application indifferently of their actual situation (local/remote) as if they were always local. • The migration can happen at any time, and the protocol is executed independently of the concurrent running application. • Transparent adaptation • Widgets are used by the application indifferently of their actual visual representation as if there was only one representation. • The adaptation can happen at any time, and the protocol is executed independently of the concurrent running application.

  15. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Thesis main contribution: EBL • Middleware that has to be interfaced to an existing toolkit • Toolkit agnostic, could be interfaced to any • Infrastructure for migration  Distributed environment Device 2 Device 1 Device 3

  16. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Distributed Toolkit • Application is stationary, widget is migratory • Two parts: proxy & renderer Site A Site C Proxy A1 Site B Proxy C1 Renderer A1 Proxy A2 Renderer A2 Renderer C1

  17. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Fault Tolerance • Network fault tolerance • Proxy has the complete state • Renderer eventually notified of state update, reflected to the widget Site A Site C Proxy A1 Site B Proxy C1 Renderer A1 Proxy A2 Renderer A2 Site D Renderer C1 Renderer C2

  18. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Migration • Dynamic creation of a renderer • Linked to a proxy • Using local resources Site B Site A Proxy A1 Renderer A1 Local Resources

  19. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Geometry Management • Geometry is managed by hierarchy of container widgets • Root widget: toplevel • Toplevel is rooted at the application that creates it • td and lr widgets provide compact syntax for tables Site A Application A Display A Toplevel Proxy Toplevel Renderer Widget Renderer Site B Application B Widget Proxy

  20. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Migration Mechanism • Migration based on capability • Embeds the authority to migrate the widget (PULL) • Embeds the Internet address of the widget • Migration triggered by giving the migration capability to a container widget Application A Application B  Window Proxy   Widget Proxy Ref   Window Renderer  get Ref  Discovery Service Widget Renderer  Ref Ref

  21. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Adaptation • Alternate compatible renderers Widget X definition Proxy X Renderer X1 Widget Y definition Renderer Y1 Proxy Y Renderer Y2 Renderer Y3

  22. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Adaptation • Adaptation is supported by dynamically switching the renderer • Switching the renderer is implemented by migrating the widget into the place it already occupies, but using a different renderer definition Application A Application B Proxy Y Renderer Y1 Adaptation of Y, from renderer Y1 to Y2 Application A Application B Proxy Y Renderer Y2

  23. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Proxy-Renderer • Collaboration implemented by shared memory abstraction: store • StoreName X KeyName -> KeyValue • Type checking, transparent marshalling Application A Application B Widget Proxy - public API Widget Renderer - EBL compatible API EBL Manager EBL Manager EBL Store x EBL Store x EBL Store y EBL Store y EBL Store z EBL Store z

  24. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Delegation • In an ideal situation, the functional core of the widget is reproduced at its proxy • Require development cost • Functional core typically already present at the renderer • A delegation mechanism allows the proxy to rely on a renderer for a state update • The proxy sends the request to the renderer and waits for its response • The renderer uses the toolkit to apply the request • The renderer gets back the resulting state, and responds to the proxy • The proxy updates its state accordingly

  25. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Delegation Example Renderer meth set(I K V) case I#K of main#state then {Toolkit apply(state V)} end end meth ask(Q R) {Toolkit apply(state Q)} R={Toolkit getState($)} end Proxy meth change(Something) OldState={Store get(main state $)} NewState={Diff OldState Something} in {Store set(main state NewState)} end meth change(Something) NewState={Manager ask(Something $)} In {Store set(main state NewState)} end No Delegation Delegation

  26. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Transaction protocol for Delegation • Delegation is implemented by transactions • Transactions are resolved in the order they are submitted. • At the proxy, transaction requests blocks until they are committed. • For a transaction to commit, the renderer must survive long enough to give its answer. • When that is not the case, the transactions keeps on blocking until a new renderer comes in. • The state of the renderer is first synchronized to the current state of the proxy. • Transactions are then retried, still respecting the order in which they are submitted Proxy State | T1, T2, … Proxy State’ | T2, … State T2 T1 Renderer State Resp T1 Resp T2

  27. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Reflexivity • Build function is also available at the renderer • Allows the creation of compound widgets using the high expressivity level of EBL even when implementing the low level renderer. Frame={New Toolkit.frame tkInit(parent:Parent)} YesButton={New Toolkit.button tkInit(parent:Frame)} {YesButton tk(configure(text:“Yes”))} NoButton={New Toolkit.button tkInit(parent:Frame)} {NoButton tk(configure(text:”No”))} {Toolkit.pack YesButton} {Toolkit.pack NoButton} Widgets={Manager build(lr(name:frame button(text:”Yes”) button(text:”No”)) $)} Frame=Widgets.frame

  28. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Model based approach using EBL • Let’s consider an application specific model • Implemented by Oz data structures data(name:"Roger" surname:"Rabbit" address1:"Rue des toons" address2:"WB")

  29. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Model based approach using EBL • Let’s consider an application specific model • Implemented by Oz data structures • Functional paradigm of Oz manipulates these data structures data(name:"Roger" surname:"Rabbit" address1:"Rue des toons" address2:"WB") fun{Transform2 D} fun{Loop P} case P of I#E|Xs then label(text:I)| label(text:E)| newline| {Loop Xs} else nil end end in {List.toTuple lr {Loop {Record.toListInd D}}} end fun{Transform1 D} {List.toTuple td {List.map {Record.toListInd D} fun{$ I#E} lr(label(text:I) label(text:E)) end}} end + td(lr(label(text:address1) label(text:"Rue des toons")) lr(label(text:address2) label(text:"WB")) lr(label(text:name) label(text:"Roger")) lr(label(text:surname) label(text:"Rabbit"))) lr(label(text:address1) label(text:"Rue des toons") newline label(text:address2) label(text:"WB") newline label(text:name) label(text:"Roger") newline label(text:surname) label(text:"Rabbit") newline)

  30. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Model based approach using EBL • Let’s consider an application specific model • Implemented by Oz data structures • Functional paradigm of Oz manipulates these data structures • End result is a valid EBL/Tk description record td(lr(label(text:address1) label(text:"Rue des toons")) lr(label(text:address2) label(text:"WB")) lr(label(text:name) label(text:"Roger")) lr(label(text:surname) label(text:"Rabbit"))) lr(label(text:address1) label(text:"Rue des toons") newline label(text:address2) label(text:"WB") newline label(text:name) label(text:"Roger") newline label(text:surname) label(text:"Rabbit") newline) lr(label(text:address1) entry(text:"Rue des toons") newline label(text:address2) entry(text:"WB") newline label(text:name) entry(text:"Roger") newline label(text:surname) entry(text:"Rabbit") newline)

  31. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Model based approach using EBL • Let’s consider an application specific model • Implemented by Oz data structures • Functional paradigm of Oz manipulates these data structures • End result is a valid EBL/Tk description record • Alternate renderings are placed into alternate renderers • Because of the reflexivity, this is a straightforward process Data=data(name:"Roger" surname:"Rabbit" address1:"Rue des toons" address2:"WB") All={Build window(form(data:Data name:form) name:top)} {All.top show} {All.form setContext(unalignedview)} {All.form setContext(alignedview)} {All.form setContext(alignededit)}

  32. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Evaluation • Performance overhead • When no delegation is used, EBL runs in an asynchronous way • 328 ms for 20 000 operations • When delegation is used, EBL runs in a synchronous way • 1780 ms for 20 000 operations • Network overhead • Typically way greater than EBL overhead • Typically the throughput limitation can be felt only when sending huge amount of data in a small window of time, which does not happen often with EBL • For asynchronous operations: the application is not slowed down, the UI lags behind by the network delay time • For synchronous operations: the application is slowed down 2Xnetwork delay

  33. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Evaluation • Expressiveness • Full fledged graphical toolkit. • The declarative approach greatly reduces the size of the code required for building a UI (QTk experiment: down to 1/3 the size). • The mixing of declarative and imperative approaches allows the creation of adaptative UIs at reduced cost (FlexClock experiment: 300 lines of code for 17 different views). • The transparent migration approach allows migrating whole UIs with few lines of code. • The transparent adaptation approach allows adaptating whole UIs with few lines of code. • The delegation greatly lowers the actual toolkit binding development cost. • The extensibility of EBL allows reusability.

  34. Conclusion • We have a GUI toolkit • That runs, is available and is documented • Which is expressively equivalent to usual GUI toolkits • Which is often more expressive and simpler than usual GUI toolkits (imperative/declarative approach) • Runs on all platforms supported by Mozart (Mac OS X, Windows, Linux, Unix…) • Supports dynamic migration • Supports dynamic adaptation • Is easily extensible

  35. Shortcomings and Future work • Tcl/Tk is aging • Replace it by a more up to date toolkit (GTk) • Limitations of the current implementation • Concurrent multi-toolkit support • Limited multi-user functionality • Communications between proxy and renderer is restricted (store, delegation) • Expressivity is enough for EBL/Tk, but is it general enough? • Address security issues • Explore multi-user aspects • Explore usability aspects

  36. The End

  37. More slides just in case

  38. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Multi-paradigm & adaptation • Functional, Logic and Symbolic programming data(name:"Roger" surname:"Rabbit" address1:"Rue des toons" address2:"WB") F2 F1 td(lr(label(text:address1) label(text:"Rue des toons")) lr(label(text:address2) label(text:"WB")) lr(label(text:name) label(text:"Roger")) lr(label(text:surname) label(text:"Rabbit"))) lr(label(text:address1) label(text:"Rue des toons") newline label(text:address2) label(text:"WB") newline label(text:name) label(text:"Roger") newline label(text:surname) label(text:"Rabbit") newline)

  39. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Adaptation Protocol PM RM RC setContext(N) replace  replace  destroy  getClassDef classDef connectManager current state RC2 apply current state state update

  40. Application EBL/Tk EBL Tcl/Tk Mozart Distribution Layer TCP/IP Migration Protocol PC RC PM getRef  PM ref importHere(Ref PI)  getClassDef   classDef create env create manager  connectManager current state  create RM  RM apply current state   state update

More Related