Manufacturing Scheduling Using Agents

1. Manufacturing Scheduling Using Agents 2005.5.27. Kim Haejoong

2. “Artificial Intelligence – A Modern Approach second edition” • Prentice Hall (2003) • Stuart Russell, Peter Norvig • “Distributed Manufacturing Scheduling Using Intelligent Agents” • IEEE INTELLIGENT SYSTEMS (2002) • Weiming Shen • “A Multi-Agent Cell Controller Integrating Temporal and Precedence Constraints” • Proceedings of the 2004 IEEE (2004) • B.Bouzouiz, R.Bouchemma Agent의 개념 Agent의 Scheduling 적용 구현 사례

3. Agent and its structure “Artificial Intelligence – A Modern Approach second edition” Prentice Hall (2003) Stuart Russell, Peter Norvig

4. Think Action Rational Agent Human-like Rationally System that act rationally Artificial intelligence • Artificial intelligence (AI) • To understandhow we think? • To build intelligent entities • Four categories of AI System that think like humans System that think rationally System that act like humans System that act rationally Rationality A system is rational If it does the “right thing” given what it knows

5. Agent • Agent • Anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators. • Human agent, robotic agent, software agent…… Performance Sensors Percepts Agent function Environments Actuators Actions Task Environments PEAS: Performance, Environments, Actuators, Sensors

6. Examples of agents

7. Properties of task environments • Fully observable vs. Partially observable • An agent’s sensors give it access to the state of the environment at each point in time • Deterministic vs. Stochastic • The next state of the environment is completely determined by the current state • Episodic vs. Sequential • The agent’s experience is divided into atomic episodes (independent on the actions taken in previous episodes) • Dynamic vs. Static • The environment can change while an gent is deliberating • Discrete vs. Continuous • Discrete set of percepts and actions (ex) chess • Single agent vs. Multi-agent

8. Structures of agents • Structure of agents • Agent = Architecture (Hardware) + Program (agent program, software agent) • If the program is going to recommend actions lie Walk, the architecture had better have leg • Basic kind of agent program • Simple reflex agents • Model-based reflex agents • Goal-based agents • Utility-based agents

9. Simple reflex agents Percepts Sensors What the world is like now Environments What action I should do now Condition-action rules Actuators Actions Agents select actions on the basis of the current percept, ignoring the rest of the percept history Simple, but very limited intelligence On the basis of only fully observable. FunctionSimple-Reflex-Agent (percept) returns an action Static: rules, a set of condition-action rules state  Interpret-Input (percept) rule  Rule-Match (state, rules) action  Rule-Action [rule] Return action

10. Model-based reflex agents update State Percepts Sensors How the world evolves What the world is like now What my action do Environments Condition-action rules What action I should do now Actuators Actions Agents keep track of history  overcome partial observability Update statestate Update_State (state, action, percept) * Model – “how the world works”

11. Goal-based reflex agents Sensors Percepts State What the world is like now How the world evolves Environments What it will be like If I do action A Recent actions What action I should do now Goals Actuators Actions Knowing about the current state is not always enough Search/Planning Consieration of the future “what will happen if I do such-and-such?”

12. Utility-based reflex agents Percepts Sensors State What the world is like now How the world evolves Environments What it will be like If I do action A Recent actions How happy I will be in such a state Utility What action I should do now Actuators Actions Goals just provide a crude binary distinction, happy/unhappy Utility function maps a state onto a real number, degree of happiness

13. Learning agents Performance standard Percepts Critic Sensors feedback Environments changes Learning element Performance element knowledge Learning goals Problem generator Actuators Actions It allows the agent to operate in initially unknown environments

14. Distributed Manufacturing Scheduling Using Intelligent Agents Weiming Shen National Research Council IEEE INTELLIGENT SYSTEMS (2002)

15. Contents 1. Manufacturing scheduling’s complexity 2. Manufacturing scheduling solutions 3. Majors issues of agent-based manufacturing scheduling 4. The research literature 5. Research opportunities and challenges

16. 1. Manufacturing scheduling’s complexity • Manufacturing scheduling • An optimization process that allocates limited manufacturing resources over time among parallel and sequential manufacturing activities. • This allocation must obey a set of rules or constraints that reflect the temporal relationships between manufacturing activities and the capacity limitations of a set of shared resources. • The allocation also affects a schedule’s optimality with respect to criteria such as cost, lateness, or throughput. • Agent-based manufacturing scheduling • The globalization of manufacturing  complex, distributed environment

17. Scheduling problems are typically NP-hard • Finding an optimal solution is impossible without using an essentially enumerative algorithm, and the computation time increases exponentially with the problem size. • Manufacturing scheduling (Job shop scheduling) is one of the most difficult scheduling problems. • The purpose is to find a best schedule • An allocation of the operations to time intervals on the machines that has the minimum duration required to complete all jobs. (ex) n jobs and m machines

18. Other variables • Additional resources • In a real manufacturing enterprise, production managers or human schedulers must consider all kinds of manufacturing resources, not just jobs and machines. • Simultaneous planning and scheduling : integrated planning and scheduling • Traditional approaches: Separating process planning and scheduling • Global optimization of a manufacturing system  Only possible when process planning and scheduling are integrated. • Unforeseen dynamic situations : dynamic scheduling/real-time scheduling • Scheduled jobs canceled and new jobs are inserted • Certain resources become unavailable and additional resources are introduced. • A scheduled task takes more or less time than anticipated, and tasks arrive early or late. • Power system failures, machine failures, operator absence, and unavailability of tools and materials. • An optimal schedule, generated after considerable effort, might become unacceptable because of unforeseen dynamic situations on the shop floor. (ex) n jobs and m machines, k operators

19. 2. Manufacturing scheduling solutions • Traditional approaches • Methods • Generate-and-test • Neighborhood search methods • Constraint satisfaction • Metaheuristics • GAs, Tabu search, and simulated annealing • Problems • Great difficulties when dealing with real situations. Simplified theoretical models Essentially centralized—that is, a central computing unit performs all computations.  Inflexible, expensive, and slow to satisfy real-world scheduling problems.

20. Agent-based approaches • Agent-based manufacturing scheduling system • Manufacturing resources (for example, machines, operators, robots, and setup stations) can be treated and represented as intelligent agents. • They are connected through a local network (for example, an Ethernet). • An agent-based manufacturing scheduling system supports distributed scheduling such that each agent can locally handle the schedule of its machine, operator, robot, or station. • However, the participating agents can collectively perform global scheduling through some negotiation mechanism and protocol (for example, a contract net protocol or an auction protocol, which I describe later). • Several potential advantages for manufacturing scheduling • The agent paradigm uses parallel computation through a large number of processors, which could provide high efficiency and robustness. • The agent paradigm makes integrating process planning and manufacturing scheduling easy, so as to realize the simultaneous optimization of manufacturing process planning and scheduling. • You can connect resource agents directly to their represented physical devices, so as to realize real-time dynamic rescheduling. • You can combine other techniques with agent-based approaches at certain levels for learning and decision-making

21. 3. Majors issues of agent-based manufacturing scheduling 1) Encapsulation • Functional decomposition • Functions such as order acquisition, planning, scheduling, material handling, transportation management, and product distribution. • No explicit relationship between agents and physical entities. • Functional decomposition tends to share many state variables across different functional agents. This can lead to inconsistency and unintended interactions. • However, functional decomposition is useful in integrating existing systems (for example, CAD tools, materials requirements planning systems, and databases), so as to resolve legacy problems. • Agents, responsible for scheduling orders, perform local incremental searches for their orders and might consider multiple resources. • The system merges the local schedules to produce a global schedule. • This is similar to centralized scheduling.

22. Physical decomposition • Represent entities in the physical world, such as operators, machines, tools, products, parts, and operations. An explicit relationship exists between an agent and a physical entity. • Physical decomposition naturally defines distinct sets of state variables that individual agents with limited interactions can manage efficiently. • However, it needs a large number of resource-related agents, which can lead to other problems (such as communication overhead) and complex agent management. • an agent represents a single real-world resource (for example, a work cell, a machine, a tool, or an operator) and maintains this resource’s schedule. • This agent might negotiate with other agents to carry out overall scheduling. • Most agent-based manufacturing scheduling systems use this approach.

23. 2) Coordination and negotiation protocols • Functional decomposition is similar to traditional integrated systems; they usually use a predefined coordination mechanism. • Contract net protocol (CNP) or its modified versions • CNP • Each agent (manager) having work to subcontract broadcasts an offer and waits for other agents (contractors) to send their bids. • After some delay, the manager retains the best offers and allocates its contracts to one or more contractors, which then process the subtask. • The CNP coordinates task allocation, providing dynamic allocation and natural load balancing. R.G. Smith, “The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver,” IEEE Trans. 1980

24. CNP problems • The basic CNP is quite simple and can be efficient. However, when the number of nodes (agents) is large, the number of messages on the network increases. • This increase can lead to a situation where agents spend more time processing messages than doing the actual work or, worse, where the system stops because messages have flooded it. • CNP improvements • Sending offers to a limited number of nodes (agents) instead of broadcasting them; • Anticipating offers; that is, contractors send bids in advance; • Varying the time when commitment is decided; • Allowing decommitment (breaking commitments); • Allowing several agents to answer as a group (coalition formation); • Introducing priorities for solving tasks.

25. 3) Architectures • Architectures for agent-based manufacturing scheduling systems • Hierarchical agent • Federated agent • Autonomous agent

26. Hierarchical • Functional decomposition • Each agent represents a function or a department in a traditional manufacturing system. • Hierarchical architectures suffer serious problems due to centralization • Federated • Federated multi-agent architectures increasingly are considered a compromise solution for industrial agent-based applications, especially for large-scale engineering applications. • A fully federated agent-based system has no explicit shared facility for storing active data. Rather, the system stores all data in local databases and handles updates and changes through message passing.

27. Federated architectures

28. Autonomous agent • An autonomous agent usually is not controlled or managed by any other software agent or human being. • Autonomous agents can communicate and interact directly with other agents in the system and with other external systems, • An autonomous agent has knowledge about other agents and its environment, and has its own goals and an associated set of motivations. • The autonomous agent architecture is well suited for developing distributed manufacturing systems consisting of a small number of agents.

29. 4) Decision schemes (Physical decomposition) • In most agent-based manufacturing scheduling systems, individual agents need to reply to all kinds of offers. • They also sometimes need to compete, negotiate, or bargain with other agents. • Knowledge • Rich knowledge andpowerful learning and reasoning mechanisms are important. • Each agent should have at least knowledge about the capability, availability, and cost of the physical resource (for example, a machine) that it represents. • A more sophisticated agent can have knowledge about other agents in its system, knowledge of the products to be manufactured, a knowledge base of previously successful cases, and so on. • The coordination or negotiation mechanism • The agent’s local decision-making mechanism

30. 4. The research literature • Early attempts • Michael Shaw proposed that a manufacturing cell could subcontract work to other cells through a bidding mechanism.(1991). • Methodologies and techniques • CORTES • Market-driven contract net • Logistics Management System • Constraint Partition and Coordinated Reaction • Autonomous Agents at Rock Island Arsenal • Case-Based multiagent Planning/Scheduling • Approaches and architectures • Distributed Asynchronous Scheduling architecture • ADDYMS (Architecture for Distributed Dynamic Manufacturing Scheduling) • Market-like control model for adaptive resource allocation and distributed scheduling • Multi-agent architecture for dynamic job shop scheduling • Dynamic-scheduling system architecture composed of holons • Test beds and industrial applications • Only a few test beds and real industrial applications have been developed. IBM LMS

31. 5. Research opportunities and challenges • Negotiation mechanisms and protocols • First, the increasing use of bidding- or market-based negotiation protocols necessitates research and development of more sophisticated negotiation mechanisms and protocols. Combinatorial market-based negotiation protocols are of much interest for the near future. • Integrating planning, scheduling, and control • Agent-based approaches provide a natural way to integrate manufacturing process planning, scheduling, and execution control. • However, they significantly increase the problem’s complexity. This topic requires much more research, including formal modeling of such integration. • Integrating agent-based and traditional approaches • Because bidding- or market-based approaches emphasize flexibility and responsiveness, rather than the optimality of solutions, they are more suitable for dynamic rescheduling. • Approaches to search such as GAs and simulated annealing focus more on the optimality of solutions, so they are more suitable for advance scheduling. • Shop floors that require both advance and dynamic scheduling could combine some of these approaches. • I have proposed combining agent-based negotiation with GAs for scheduling optimization • Also, integrating agent-based approaches with other traditional approaches (such as fuzzy logic- and artificial neural network-based learning, Petri net- or color Petri net-based coordination, or constraint satisfaction) will likely produce sophisticated manufacturing scheduling systems.

32. Combining individual solving and coordination-negotiation • Obviously, a trade-off exists between solving at the individual agent level and the coordination–negotiation scheme at the system level. • The challenge is how to combine them using integration such as I mentioned earlier. • Benchmarks • We need benchmarks to compare different agent-based systems and to compare these systems with others using traditional approaches. • Theoretical investigation of methodology • Weneed theoretical investigation of methodology, including implementation methodology, to consolidate current research results and facilitate implementation of real industrial applications.

33. 6. Conclusion • Agent-based approaches offer many advantages for distributed manufacturing scheduling systems: Quick responsiveness, flexibility, scalability, and robustness, dynamic manufacturing scheduling problems. • Combinations of agent-based and traditional approaches look even more promising • Whether implementations of such systems realize these potential advantages will depend on selecting a suitable system architecture and on effective mechanisms and protocols for communication, cooperation, coordination, negotiation, adaptation, and other capabilities.

34. A Multi-Agent Cell Controller Integrating Temporal and Precedence Constaints B.Bouzouiz, R.Bouchemma Division Robotique et Productique, Centre de Developpement des Technologies Avacees, Hassen, Algerie Proceedings of the 2004 IEEE (2004)

35. Abstract • A flexible Cell Controller based on the Multi-Agent approach • The total distribution of the scheduling and control functions • Contract Net protocol • Scheduling and rescheduling process • A multi-agent architecture for flexible Cell control based on Oulhdj architecture with integration of temporal and precedence constraints during scheduling • LAN, Visual C# language, OPC(OLE for process control) • Test Bed • The Flexible Manufacturing Cell, used as test bed, contains the following machines and tools: • GE FANLJC PLC Controller, Series 90-30, EMCO TURN 140 • CNC Machine. Six axes Robot GT6A, SNT Conveyor type • RMI 40 FR and finally one rotation table. • The PLC controls Conveyor and the rotation Table. GT6A • Robot is controlled through Serial Porl communication with • its own software. CNC Machine can be controlled by sending • command (G-Code File) on its Serial or Parallel Port.

36. Architecture • Software Architecture • The multi-agent architecture proposed for real-time monitoring and task scheduling Task Manager Agent (TMA): the Cell supervisor Cell resource (Robot, CNC Machine or PLC) Resource Driver Agent (RDA) Each RDA controls one Cell resource and contributes with other Cell agents in scheduling process TMA role is the tasks and system management. All RDA agents make registration into TMA in order to contribute in Control process.

37. Agent Architecture

38. Implementation

40. Discussion • Agent의 한계 • 아직까지 실제 적용한 사례가 많지 않다. • Traditional approach와의 비교 • Agent의 Intelligence의 문제 • Coordination problem • Agent의 가능성 • 복잡한 문제, 시스템에 대한 해결책 • Intelligence의 발달

41. Q&A • Agent structure 에서 goal-based structure에서의 goal과 agent의 performance의 차이는? • Goal은 performance의 상위개념이다. 예를 들어 어떤 agent의 goal의 목표가 “happiness”라면 performance는 “health”, “safeness”, “wealth” 등이 된다. • Agent가 실제 application으로 적용되지 못하는 이유는? • Agent는 AI에서 처음 소개되어 광범위한 개념으로 사용되었습니다. 굳이 Agent라고 칭하지 않더라도 넓은 범위에서 agent라고 할 수 있는 것은 광범위하게 사용되고 있습니다. 다만 manufacturing scheduling에서는 dynamic, distributed environment 하에서 20여 년 정도 연구도 진행되고 실제 application 구현을 위한 시도가 있었지만 좋은 결과를 낳지는 않았습니다. 이는 scheduling 문제의 복잡도로 인해 놓은 수준의 Intelligence와 negotiation이 필요하지만 아직까지 그 수준에 도달하지 못했다. • Agent기반 시스템의 가능성? • Intelligence와 negotiation protocol의 한계가 극복되면 다양한 분야에서 사용될 것이다.