A Model-Based Approach for Development of Multi-Agent Software Systems

1. A Model-Based Approach for Development of Multi-Agent Software Systems Haiping Xu, Ph.D. Computer and Information Science Department University of Massachusetts Dartmouth http://www.cis.umassd.edu/~hxu

2. Acknowledgments • Sol M. Shatz Thesis Advisor, Professor at UIC Co-Director, Concurrent Software Systems Laboratory • Tadao Murata UIC Distinguished Professor Director, Concurrent Software Systems Laboratory CIS Dept., UMass Dartmouth

3. Outline • Part 1: Background and Motivations • Part 2: Design of Agent-Based G-net Model • Part 3: Modeling Agent-Oriented Software • Part 4: Analysis of Agent-Oriented Model • Part 5: Design of Intelligent Mobile Agents • Part 6: Agent Development Kit (ADK) • Part 7: Conclusions and Future Work. CIS Dept., UMass Dartmouth

4. Part 1: Background and Motivations • The development of software systems starts with two main activities: • Software requirements analysis • Software design • Software requirements analysis: to reduce potential errors caused by incomplete and ambiguous requirements • Software design: to depict the overall structure of a system by decomposing the system into its logical components. CIS Dept., UMass Dartmouth

5. Formal Methods in Software Engineering • The purpose of software requirements analysis can be achieved in two ways: • Write a specification in natural languages • Choose a formal language, e.g., Petri nets • Ideally, formal methods can be applied in each phase of the software development life cycle, e.g., the design phase • However, to create a formal model in the design phase and to verify its correctness is rare. CIS Dept., UMass Dartmouth

6. Introduction to Petri Net • “Three-in-one” capability of Petri net models [Murata 1989] • Graphical representation • Mathematical description • Simulation tool • Definition: A Petri net is a 4-tuple, PN = (P, T, F, M0) where P = {P1, P2, …, Pm} is a finite set of places; T = {t1, t2, …, tn} is a finite set of transitions; F  (P x T)  (T x P) is a set of arcs (flow relation); M0: P --> {0, 1, 2, 3, …} is the initial marking. CIS Dept., UMass Dartmouth

7. An Example t2 P1 P2 t3 t1 P5 P3 t4 t5 P4 CIS Dept., UMass Dartmouth

8. G-Net: A High Level Petri Net • Defined to support modeling of systems as a set of independent and loosely-coupled modules [Deng et al. 1993] • Provides support for incremental design and successive modification • Is not fully object-oriented due to a lack of support for inheritance. CIS Dept., UMass Dartmouth

9. GSP(Buyer) GSP(Seller) buyGoods() askPrice() returnPrice() sellGoods() t1 t3 t5 t7 ISP(Seller, ISP(Seller, calculate_ sell_ sellGoods()) returnPrice()) goods price t2 t4 t6 t8 An Example CIS Dept., UMass Dartmouth

10. Introduction to Agents • The term “agent” comes from the greek word “agein”, which means to drive or to lead • A agent is a computer system that is situated in some environment, and that is capable of autonomous action • It is suitable to describe current trends in computer science: active instruments (to which work can be delegated) vs. passive tools • The term “agent” in computer science generally refers to a software agent. CIS Dept., UMass Dartmouth

11. Research Directions • Multi-agent systems (MAS) • Agents act as “active” objects (intelligence) • Collaborative or competitive • Generally use distributed but static (non-mobile) agents • Mobile agents (MA) • Model agent mobility and agent coordination • Generally assume very limited or even no intelligence. CIS Dept., UMass Dartmouth

12. Agent-Oriented Software Engineering • The agents can be considered as active objects, i.e., objects with a mental state • However, object-oriented methodologies do not address the following aspects: • asynchronous message-passing mechanism • autonomous behavior (mental states) • Agent-oriented approaches: provide guidelines for agent specification and design • AAII methodologies: based on BDI model • Gaia methodologies: based on role modeling. CIS Dept., UMass Dartmouth

13. Formal Methods in Agent-Oriented Software Engineering • Limited work on formal specification and design for agents • Use formal languages, such as Z, to provide a framework for describing the agent architecture [Luck and d’Inverno 1995] • Use temporal logic to specify agent behavior [Fisher 1995] • Design formal languages for specifying agent-based systems, e.g., DESIRE [Brazier et al. 1997] and SLABS [Zhu 2001] • Agent models based on Petri nets, such as [Moldt and Wienberg 1997][Merseguer et al. 2000] [Xu and Deng 2000] • However, they do not explicitly model agent interactions, and they do not address the issue of inheritance • Unlike the previous work, our proposed agent models: • Serve as a high-level design for agent implementaion • Support protocol-based agent interaction/communication • Support reuse of functional units of our agent class models. CIS Dept., UMass Dartmouth

14. Our Incremental Approach Object-based G-nets (the original G-nets) Standard G-nets (support class modeling) Agent-based G-nets (support agent modeling) Object-Oriented G-nets (support inheritance) Agent-Oriented G-nets (support inheritance) CIS Dept., UMass Dartmouth

15. Part 2: An Agent-Based G-Net Model • Multi-agent systems (MAS): one of the most important topics in distributed and autonomous decentralized systems • Key features: autonomous, reactive and internally-motivated agents • However, the G-net model is not sufficient for agent modeling because: • Does not support a common communication language and common protocols among agents • Does not directly support asynchronous message passing • Does not support modeling agent’s mental state, such as goals, plans and knowledge. CIS Dept., UMass Dartmouth

16. GSP(G) Goal Plan Knowledge-base Environment Planner utility method incoming message outgoing message action_m action_1 action_n utility_1 utility_p action_1 … … … message_ message_ message_ message_ utility_1 utility_p processing processing processing processing … … … MSP(G’.Aid) MSP(G’.Aid) Return Return MSP(self) MSP(self) An Agent-Based G-Net Model CIS Dept., UMass Dartmouth

17. GSP(G) Goal/Plan/KB Environment external internal from transition “update” dispatch_ incoming_ next_ start_a_ message action conversation ignore continue … private_utility dispatch_ outgoing_ update_ message goalplan/kb dispatch_ … utilities incoming messages update … … to place “goal” … … to place “knowledge base” outgoing messages utility methods A Template of Planner Module CIS Dept., UMass Dartmouth

18. Definitions of the Message Token struct Message{ int sender; // the identifier of the message sender int receiver; // the identifier of the message receiver string protocol_type; // the type of contract net protocol string name; // the name of incoming/outgoing messages string content; // the content of this message }; enum Tag {internal, external}; struct MtdInvocation { Triple (seq, sc, mtd); // as defined in Section 2.1 } if (mTkn.tag {internal, external}) thenmTkn.body = struct { Message msg; // message body } elsemTkn.body = struct { Message msg; // message body Tag old_tag; // to record the old tag: internal/external MtdInvocation miv; // to trace method invocations } CIS Dept., UMass Dartmouth

19. Formal Definitions of Agent-Based G-Net Model Definition 3.1Agent-Based G-Net An agent-based G-net is a 7-tuple AG = (GSP, GL, PL, KB, EN, PN, IS), where GSP is a Generic Switch Place providing an abstract for the agent- based G-net, GL is a Goal module, PL is a Plan module, KB is a Knowledge-base module, EN is an Environment module, PN is a Planner module, and IS is an internal structure of AG. Definition 3.2Planner Module A Planner module of an agent-based G-net AG is a colored sub-net defined as a 7-tuple (IGS, IGO, IPL, IKB, IEN, IIS, DMU), where IGS, IGO, IPL, IKB, IEN and IIS are interfaces with GSP, Goal module, Plan module, Knowledge-base module, Environment module and internal structure of AG, respectively. DMU is a set of decision-making unit, and it contains three abstract transitions: make_decision, sensor and update. Definition 3.3Internal Structure (IS) An internal structure (IS) of an agent-based G-net AG is a triple (IM, OM, PU), where IM/OM is the incoming/outgoing message section, which defines a set of message processing units (MPU); and PU is the private utility section, which defines a set of methods. Definition 3.4Message Processing Unit (MPU) A message processing unit (MPU) is a triple (P, T, A), where P is a set of places consisting of three special places: entry place, ISP and MSP. Each MPU has only one entry place and one MSP, but it may contain multiple ISPs. T is a set of transitions, and each transition can be associated with a set of guards. A is a set of arcs defined as: ((P-{MSP}) x T)  ((T x (P-{entry}). Definition 3.5U-Method A U-Method or method is a triple (P, T, A), where P is a set of places with three special places: entry place, ISP and return place. Each method has only one entry place and one return place, but it may contain multiple ISPs. T is a set of transitions, and each transition can be associated with a set of guards. A is a set of arcs defined as: ((P-{return}) x T)  ((T x (P-{entry}). CIS Dept., UMass Dartmouth

20. Buyer Seller Buyer Seller Buyer Seller request-price request-price request-price · refuse propose propose x propose reject-proposal accept-proposal accept-proposal x reject-proposal propose · confirm propose * accept-proposal accept-proposal x reject-proposal confirm · · confirm (a) (b) (c) Selling and Buying Agent Design CIS Dept., UMass Dartmouth

21. GSP(G) Goal Plan Knowledge-base Environment Planner incoming message outgoing message utility method request-price reject-proposal propose refuse confirm utility_1 utility_p accept-proposal … mesg_pr - mesg_pr - mesg_pr - mesg_pr - mesg_pr - mesg_pr - utili - utili - ocessing ocessing ocessing ocessing ocessing ocessing ty_1 ty_p t4 … MSP(G’.Aid) MSP(G’.Aid) MSP(G’.Aid) Return Return MSP(self) MSP(self) MSP(self) Buying Agent Class Design CIS Dept., UMass Dartmouth

22. Verifying the Agent-Based G-Net Model • L3-live: any communicative act can be performed as many times as needed • Concurrent: a number of conversations among agents can happen concurrently • Effective: an agent communication protocol can be correctly traced in the agent models. CIS Dept., UMass Dartmouth

23. (goa/plan/kb) (goa/plan/kb) (env) (env) a1 b1 c1 a2 b2 c2 G G GSP( ) GSP( ) (external) (internal) t3 (external) (internal) t21 t1 t2 t19 t20 (start_a_ (start_a_ (dispatch_ (dispatch_ conversation) conversation) (next_ incoming_ incoming_ (next_ d1 e1 d2 e2 action) message) message) action) (continue) (continue) (ignore) (ignore) t4 t5 t6 t7 t8 t22 t23 t24 t25 t26 (dispatch_ (dispatch_ f1 g1 h1 f2 g 2 h2 outgoing_ outgoing_ i1 i2 message) message) j1 j2 (update) (update) t9 t10 t27 t28 t29 t11 t12 t13 t14 t15 t30 t31 t32 t33 k1 l1 m1 k2 l2 m2 (request_price, accept_proposal, (propose, refuse, confirm) reject_proposal) (request_price, accept_proposal, t16 t17 t18 (propose, refuse, confirm) t34 t35 t36 reject_proposal) Buyer Seller Verifying the Agent-Based G-Net Model(continued) CIS Dept., UMass Dartmouth

24. Part 3: A Framework for Modeling Agent-Oriented Software • Core approaches that can be extended: • Object-oriented (OO) methodologies • Knowledge engineering (KE) methodologies • We follow the first approach, and separate traditional object-oriented features and reasoning mechanism to enhance reuse of communication components • Show the useful role of inheritance in agent-oriented software design. CIS Dept., UMass Dartmouth

25. GSP(G) Goal Plan Knowledge-base Environment Planner utility method incoming message outgoing message action_m action_1 action_n utility_1 utility_p action_1 … … … message_ message_ message_ message_ utility_1 utility_p processing processing processing processing … … … MSP(G’.Aid) MSP(G’.Aid) Return Return MSP(self) MSP(self) The Agent-Based Model (Revisited) CIS Dept., UMass Dartmouth

26. A Revised Planner Module • Abstract transition: represents abstract units of decision-making or mental-state-updating (with synchronization) • Autonomous unit: makes an agent autonomous and internally-motivated • Asynchronous Superclass switch Place (ASP): used to forward a MPU or a method call (token) to a “superclass” model in the case of inherited communication mechanisms. CIS Dept., UMass Dartmouth

27. Goa l Plan Knowledge-base Environment GSP(G) external internal from transition dispatch_ check_ “update” incoming_ primary autonomous unit message sensor <e> make_ new_ … private decision action bypass start_a_ dispatch_ <e> syn next_ automatic_ conver_ private_ … action update sation utility ASP(super) incoming messages ignore continue … update_ dispatch_ … outgoing_ goal/plan/kb <e> ASP(super) message to place “Goal” update utilit y methods … to place “Plan” t o place “Knowledge base” … ASP(super) outgoing messages A Template for the Planner Module(Initial Design) CIS Dept., UMass Dartmouth

28. Shopping agent Buying agent Selling agent Customer agent Retailer agent Auctioneer agent Examples of Agent-Oriented Design(class hierarchy) CIS Dept., UMass Dartmouth

29. shopping agent facilitator agent buying agent selling agent buying agent selling agent request - registration request - price request - price · refuse · refuse propose x x request - info propose reject - propos al accept - proposal supply - info x reject - proposal · propose refuse propose x * accept - info accept - proposal accept - proposal x · reject - proposal confirm · confirm · confirm (b) (c) (a) Examples of Agent-Oriented Design(contract net protocol) CIS Dept., UMass Dartmouth

30. SC GSP( ) Goal Plan Knowledge - base Environment Planner incoming message outgoing message u tilitiy method request - info refuse accept_info c onfirm request - registration supply - info utility_1 utility_p … mesg_pr - mesg_pr - mesg_pr - mesg_pr - mesg_pr - mesg_pr - utili - utili - ocessing ocessing ocessing ocessing ocessing ocessing ty_1 ty_p t4 … self self self self MSP(G’.Aid) MSP(G’.Aid) return return MSP( ) MSP( ) MSP( ) MSP( ) Examples of Agent-Oriented Design(shopping agent class) CIS Dept., UMass Dartmouth

31. BC) GSP( Goal Plan Knowledge - base Environment BC extends SC Planner outgoing message private utility method incoming message request - price accept - proposal reject - p roposal utility_1 utility _p propose … message_ message_ message_ message_ utility_1 utility_p processing processing processing processing … MSP(G’.Aid) MSP(G’.Aid) MSP(G’.Aid) MSP( self ) return return Examples of Agent-Oriented Design(buying agent class) CIS Dept., UMass Dartmouth

32. Part 4: Analysis of Agent-Oriented Models • To help ensure a correct design that meets certain specifications • To meet certain requirements such as liveness, deadlock freeness and concurrency • Use Petri net tool: INA (Integrated Net Analyzer) • verifying structural properties • verifying behavioral properties • modeling checking (CTL formulas). CIS Dept., UMass Dartmouth

33. Shopping Shopping (goal/plan/kb_1) (inhib_arc_1) (environment_1) : Shopping Subagent GSP( ) P3 P4 Buying : Buying Primary Subagent P1 P2 extends (Buying Agent Class Shopping agent Class) Selling_1 : Selling Agent_1 t1 Selling_2 : Selling Ag ent_2 (external_1) t2 (internal_1) List of message processing units (syn_1) (dispatch_ (check_ P5 ========================= P7 incoming_ P6 primary_1) message_1) P8: request_info P9: refuse (make_ t3 t4 t5 t6 t7 t8 (bypass_1) (sensor_1) P10: accept_info t9 decision_1) P11: confirm P8 P9 P10 P11 P16: request - registration (new_ (dispatch_ P14 P17: supply_info action_1) (next_ P12 outgoing_ P25: propose P13 action_1) message_1) P31: request - price (automatic_ P32: accept - proposal t10 t11 t12 t13 t14 update_1) P33: reject - proposal t15 t16 t17 (start_a_ (ignore_1) (continue_1) incoming me ssages P15 t18 t19 conversation_1) P16 P17 (update_ goal/plan/kb_1) t20 (update_1) t22 t21 GSP( Selling_1 ) outgoing messages Buying (goal/plan/kb_2) (inhib_arc_2) (environment_2) GSP( ) P20 P21 P35 P18 P19 (external_2) t45 t23 t24 (dispatch_ (internal_2) (syn_2) (check_ P24 incoming_ P22 primary_2) GSP( Selling_2 ) P23 message_1) t28 (bypass_2) (sensor_2) P36 (make_ t25 t26 t27 t29 (dispatch_ (new_ decision_2) P29 P25 P26 Super outgoing_ ASP( ) action_2) (next_ P28 t46 message_2) (automatic_ P27 action_2) update_2) t30 t32 (ignore_2) t33 t34 t35 t31 (start_a_ (continue_2) t37 t38 P3 0 t36 conversation_2) incoming messages t39 P31 P32 P33 (update_ P34 Super ASP( ) to superclass goal/plan/kb_2) t40 (update_2) t42 t43 t44 t41 outgoing messages to superclass A Transformed Model of One Buying Agent and Two Selling Agents CIS Dept., UMass Dartmouth

34. Experiment: Result -1 Computation of the reachability graph States generated: 8193 Arcs generated: 29701 Dead states: 484, 485,8189 Number of dead states found: 3 The net has dead reachable states. The net is not live. The net is not reversible (resetable). The net is bounded. The net is safe. The following transitions are dead at the initial marking: 7, 9, 14, 15, 16, 17, 20, 27, 28, 32, 33 The net has dead transitions at the initial marking. CIS Dept., UMass Dartmouth

35. G Goal Plan Knowledge - base Environment GSP( ) external internal from transition dispatch_ check_ “update” incoming_ primary autonomous unit message sensor <e> make_ new_ private … decision action bypass start_a_ dispatch_ <e> syn next_ automatic_ conver_ private_ … action update sation utility ASP( super ) incoming me ssages ignore continue … update_ dispatch_ … outgoing_ goal/plan/kb <e> super ASP( ) message to place “Goal” update private utilities … to place “Plan” to place “Knowledge base” … ASP( super ) outgoing messages Redesign of the Planner Module CIS Dept., UMass Dartmouth

36. Experiment: Result - 2 Computation of the reachability graph States generated: 262143 Arcs generated: 1540095 The net has no dead reachable states. The net is bounded. The net is safe. The following transitions are dead at the initial marking: 7, 9, 14, 15, 16, 17, 20, 28 The net has dead transitions at the initial marking. Liveness test: Warning: Liveness analysis refers to the net where all dead transitions are ignored. The net is live, if dead transitions are ignored. The computed graph is strongly connected. The net is reversible (resetable). CIS Dept., UMass Dartmouth

37. Property Verification Using Modeling Checking • Concurrency (e.g., process incoming and outgoing messages) EF(P5 &(P13 &(P22 &P28))) Result: The formula is TRUE • Mutual Exclusion (don’t want simultaneous updating of goals, plan, etc.) EF(P27 &P30) V (P29 &P30)) Result: The formula is FALSE • Inheritance Mechanism (decision-making in a superclass model) AG(-P12 &(-P14 &-P15)) Result: The formula is TRUE • Inheritance Mechanism (ASP message forwarding I) A[(P26 VP34)B(P5 VP6)] Result: The formula is TRUE • Inheritance Mechanism (ASP message forwarding II) A[P26 BP5]VA[P34 BP6] Result: The formula is FALSE CIS Dept., UMass Dartmouth

38. Part 5: Design of Intelligent Mobile Agents – A Generic Model • Two schemes for agent development: • Weak agent approach • Strong agent approach • Most of the existing work on mobile agents use weak agent approach (not flexible, security issues …) • In contrast, we propose a generic model for intelligent mobile agent. CIS Dept., UMass Dartmouth

39. a a MA: MA: (1) (2) (3) Agent World Architecture Host-A Host-B Q Q AVM: AVM: A B q b q g FA: MA: FA: MA: A B … … computer network (1) move-request (2) grant (3) notify (4) move CIS Dept., UMass Dartmouth

40. GSP(IMA) Goal Plan Knowledge-base Environment Planner incoming message outgoing message utility method ask - authCode refuse - move grant - move confirm-move move - request return - authCode notify move … ISP(FA, inform) action action action action action action utility … return return self self self self MSP(G’.Aid) MSP(G ’.Aid) MSP( ) MSP( ) MSP( ) MSP( ) Intelligent Mobile Agent (IMA) CIS Dept., UMass Dartmouth

41. confirm-move move() entry place entry place start_move begin_process migration message_processing fail succeed after_process retry change_CurIP ISP(self, notify) retry else begin_migration [retry ≤ MAX_TRIAL] ISP(self, move) ISP(FA, ISP(rFA, register) inform(FAILURE)) after_migration end MSP(G’, Aid) Return (a) (b) Refinement of Functional Units CIS Dept., UMass Dartmouth

42. Advantages of This Approach • Application-specific mobile agent class can be defined as a subclass of IMA • With further refinement of functional units, the design of mobile agents can be verified to meet certain requirements • The strong agent approach leaves adequate room for security modeling. CIS Dept., UMass Dartmouth

43. Part 6: Agent Development Kit (ADK) • Most of existing work on agent formal modeling defines what properties are to be realized by an agent system instead of how • In contrast, agent-oriented G-net model serves as a high-level design for agent development • Implement Agent Development Kit (ADK) • Use Jini middleware for agent communication • Provide a framework and a full class library to support development of application-specific agents for MAS. CIS Dept., UMass Dartmouth

44. ADK Agent Architecture Modularization Interface Definition: GSP Class Definition: Goal, Plan, Knowledge, Agent (Planner, Internal Structure) Jini Community: Environment Message Passing Mechanism MSP: new thread ISP: method invocation Functional Units (Inheritable) DMU: protected method MPU: protected method U-Method: protected method * DMUs are not inheritable in agent-oriented G-net model Overview of Agent Design Architecture Formal Agent Model Implementation Platform Modularization GSP (interface) Goal, Plan, Knowledge-base, Planner, Internal Structure Environment Middleware Jini/JavaSpaces/RMI Java Virtual Machine Java: OO Language Message Passing Mechanism Asynchronous: MSP Synchronous: ISP JVM: Thread, Java Swing etc. Functional Units (Inheritable) DMU* (decision-making unit) MPU (message processing unit) U-Method (utility method) Network Communication TCP/IP, UDP CIS Dept., UMass Dartmouth

45. Discovery Service Lookup Service Join Manager AirTicket Seller AirTicket Buyer Jini Community … … AirTicket Buyer GSP GSP AirTicket Seller GSP GSP The Jini Community CIS Dept., UMass Dartmouth

46. Action Component (agent B) Goal Plan Knowledge internal event message Planner from GSP decision- message sensor making units dispatcher external event Internal Structure incoming message outgoing message utility method (MPUs) (MPUs) (methods) message to GSP Jini Community (environment) SellerGSP … BuyerGSP ( agent B ) ( agent A ) … BuyerGSP SellerGSP The Agent Architectural Design CIS Dept., UMass Dartmouth

47. A Design Pattern for Intelligent Agents • Define the Agentclass to provide a pattern for agent implementation • Derive the Agent class from MiddlewareSupport, which deals with the Jini community • Separate agent communication capabilities and agent mental states to enhance reuse of agent design. CIS Dept., UMass Dartmouth

48. Agent Message Goal: myGoals ServiceID: senderID send/receive Plan: myPlans ServiceID: receiverID Knowledge: myKnowledge String: protocolName String: content initAgent() toString() autonousmousRun() communication capabilities mental states achieve initialize Goal Knowledge String: goalName AgentInfo: thisAgent use/update Goal: subGoal AgentInfo[]: remoteAgents Plan: associated Plans Protocol[]: protocols initKnowledge() achieveGoal() update() execute use/update Plan Protocol String: planName use/update String: protocolName Int: priority DataStore: protocolSequence Boolean: conditions Int: statusID S tring: associatedGoalName startPlan() initProtocol() stopPlan() callFo rProposal() Classes Diagram: Communication Capabilities and Mental States CIS Dept., UMass Dartmouth

49. GSP (interface) Protocol * Goal Plan MiddlewareSupport Knowledge Message * * * * A gent Send / Receive Class Library (ADK) Derived Classes Air Ticket Seller Air Ticket Buyer Classes defined in ADK and derived classes of the Agent class CIS Dept., UMass Dartmouth

50. A Design Pattern for Application-Specific Agents 1public class ApplicationSpecificAgent extends Agent { 2 3 /************************************************ 4 * Class Variables for Knowledge, Goal and Plan * 5 /************************************************/ 6 Goal myGoals; // committed goals, redefinition of Goal class is optional 7         Plan myPlans; // plans, redefinition of Plan class is optional 8         Knowledge myKnowledge; // knowledge-base, redefinition of Knowledge 9         // class is optional 10     … 11       12     /*********** 13     * Planner * 14     ***********/ 15     protected void dispatchMessage(Message message) {…} // refinement 16     // or redefinition 17     protected Message makeDecision(Message message) {…} // refinement 18     // or redefinition 19     protected void updateMentalState() {…) // refinement 20     // or redefinition 21     … 22       CIS Dept., UMass Dartmouth