multi-agent system

A multi-agent system (MAS or "self-organized system") is a computerized system composed of multiple interacting intelligent agents.Yoav Shoham, Kevin Leyton-Brown. Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, 2009. http://www.masfoundations.org/H. Pan; M. Zahmatkesh; F. Rekabi-Bana; F. Arvin; J. Hu "[https://ieeexplore.ieee.org/document/10965835 T-STAR: Time-Optimal Swarm Trajectory Planning for Quadrotor Unmanned Aerial Vehicles]" IEEE Transactions on Intelligent Transportation Systems, 2025. Multi-agent systems can solve problems that are difficult or impossible for an individual agent or a monolithic system to solve.Hu, J.; Turgut, A.; Lennox, B.; Arvin, F., "[https://ieeexplore.ieee.org/abstract/document/9409965 Robust Formation Coordination of Robot Swarms with Nonlinear Dynamics and Unknown Disturbances: Design and Experiments]" IEEE Transactions on Circuits and Systems II: Express Briefs, 2021. Intelligence may include methodic, functional, procedural approaches, algorithmic search or reinforcement learning.Stefano V. Albrecht, Filippos Christianos, Lukas Schäfer. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024. https://www.marl-book.com/ With advancements in large language models (LLMs), LLM-based multi-agent systems have emerged as a new area of research, enabling more sophisticated interactions and coordination among agents.{{cite journal | last =Li | first =Guohao | title =Camel: Communicative agents for "mind" exploration of large language model society | journal = Advances in Neural Information Processing Systems | volume = 36 | pages = 51991–52008 | year = 2023 | url = https://proceedings.neurips.cc/paper_files/paper/2023/file/a3621ee907def47c1b952ade25c67698-Paper-Conference.pdf | arxiv = 2303.17760 | s2cid = 257900712}}

Despite considerable overlap, a multi-agent system is not always the same as an agent-based model (ABM). The goal of an ABM is to search for explanatory insight into the collective behavior of agents (which do not necessarily need to be "intelligent") obeying simple rules, typically in natural systems, rather than in solving specific practical or engineering problems. The terminology of ABM tends to be used more often in the science, and MAS in engineering and technology.{{cite journal |first1=Muaz |last1=Niazi |first2=Amir |last2=Hussain |year=2011 |title=Agent-based Computing from Multi-agent Systems to Agent-Based Models: A Visual Survey |journal=Scientometrics |volume=89 |issue=2 |pages=479–499 |doi=10.1007/s11192-011-0468-9 |url=https://www.researchgate.net/publication/220365334 |format=PDF|arxiv=1708.05872 |hdl=1893/3378 |s2cid=17934527 }} Applications where multi-agent systems research may deliver an appropriate approach include online trading,{{cite journal |first1=Alex |last1=Rogers |first2=E. |last2=David |first3=J. |last3=Schiff |first4=N.R. |last4=Jennings |url=http://eprints.ecs.soton.ac.uk/12716/ |title=The Effects of Proxy Bidding and Minimum Bid Increments within eBay Auctions |journal=ACM Transactions on the Web |volume=1 |issue=2 |pages=9–es |year=2007 |doi=10.1145/1255438.1255441 |citeseerx=10.1.1.65.4539 |s2cid=207163424 |access-date=2008-03-18 |archive-date=2010-04-02 |archive-url=https://web.archive.org/web/20100402101304/http://eprints.ecs.soton.ac.uk/12716/ |url-status=dead }} disaster response,{{cite web |first1=Nathan |last1=Schurr |first2=Janusz |last2=Marecki |first3=Milind |last3=Tambe |first4=Paul |last4=Scerri |first5=Nikhil |last5=Kasinadhuni |first6=J.P. |last6=Lewis |url=https://aaai.org/papers/0002-ss05-01-002-the-future-of-disaster-response-humans-working-with-multiagent-teams-using-defacto |title=The Future of Disaster Response: Humans Working with Multiagent Teams using DEFACTO |year=2005 |access-date=8 January 2024 |archive-date=2013-06-03 |archive-url=https://web.archive.org/web/20130603165342/http://teamcore.usc.edu/papers/2005/SS105SchurrN.pdf| url-status=live}}{{cite book |last1=Genc|first1=Zulkuf |title=Intelligent Systems for Crisis Management |chapter=Agent-Based Information Infrastructure for Disaster Management |chapter-url=http://www.gdmc.nl/gi4dmdocs/Gi4DM_2012_Genc.pdf |pages=349–355 |date=2013|display-authors=etal|doi=10.1007/978-3-642-33218-0_26 |isbn=978-3-642-33217-3 |series=Lecture Notes in Geoinformation and Cartography }} target surveillance{{cite journal |last1=Hu |first1=Junyan |last2=Bhowmick |first2=Parijat|last3=Lanzon |first3=Alexander |title=Distributed Adaptive Time-Varying Group Formation Tracking for Multiagent Systems With Multiple Leaders on Directed Graphs |journal=IEEE Transactions on Control of Network Systems |date=2020 |volume=7 |pages=140–150 |doi=10.1109/TCNS.2019.2913619 |s2cid=149609966 |doi-access=free }} and social structure modelling.{{cite journal |first1=Ron |last1=Sun|author-link1=Ron Sun |first2=Isaac |last2=Naveh |url=http://jasss.soc.surrey.ac.uk/7/3/5.html |title=Simulating Organizational Decision-Making Using a Cognitively Realistic Agent Model |journal=Journal of Artificial Societies and Social Simulation|date=30 June 2004}}

Concept

Multi-agent systems consist of agents and their environment. Typically multi-agent systems research refers to software agents. However, the agents in a multi-agent system could equally well be robots, humans or human teams. A multi-agent system may contain combined human-agent teams.

Agents can be divided into types spanning simple to complex. Categories include:

Passive agents{{citation |first1=Yoann |last1=Kubera |first2=Philippe |last2=Mathieu |first3=Sébastien |last3=Picault |url=http://www.lifl.fr/SMAC/publications/pdf/aamas2010-everything.pdf |title=Everything can be Agent! |journal=Proceedings of the Ninth International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS'2010) |pages=1547–1548 |year=2010 }} or "agent without goals" (such as obstacle, apple or key in any simple simulation)
Active agents with simple goals (like birds in flocking, or wolf–sheep in prey-predator model)
Cognitive agents (complex calculations)

Agent environments can be divided into:

Virtual
Discrete
Continuous

Agent environments can also be organized according to properties such as accessibility (whether it is possible to gather complete information about the environment), determinism (whether an action causes a definite effect), dynamics (how many entities influence the environment in the moment), discreteness (whether the number of possible actions in the environment is finite), episodicity (whether agent actions in certain time periods influence other periods),{{Russell Norvig 2003}} and dimensionality (whether spatial characteristics are important factors of the environment and the agent considers space in its decision making).{{cite book | last1 = Salamon | first1 = Tomas | title = Design of Agent-Based Models | location = Repin | publisher = Bruckner Publishing | year= 2011 | page = 22 | isbn = 978-80-904661-1-1 | url=http://www.designofagentbasedmodels.info/}} Agent actions are typically mediated via an appropriate middleware. This middleware offers a first-class design abstraction for multi-agent systems, providing means to govern resource access and agent coordination.{{ cite journal |first1=Danny |last1=Weyns |first2=Amdrea |last2=Omicini |first3=James |last3=Odell |year=2007 |title=Environment as a first-class abstraction in multiagent systems |journal=Autonomous Agents and Multi-Agent Systems |volume=14 |issue=1 |pages=5–30 |doi=10.1007/s10458-006-0012-0 |citeseerx=10.1.1.154.4480 |s2cid=13347050 }}

= Characteristics =

The agents in a multi-agent system have several important characteristics:{{cite book |first=Michael |last=Wooldridge |title=An Introduction to MultiAgent Systems |publisher=John Wiley & Sons |year=2002 |pages=366 |isbn=978-0-471-49691-5}}

Autonomy: agents at least partially independent, self-aware, autonomous
Local views: no agent has a full global view, or the system is too complex for an agent to exploit such knowledge
Decentralization: no agent is designated as controlling (or the system is effectively reduced to a monolithic system){{cite journal |first1=Liviu |last1=Panait |first2=Sean |last2=Luke |url=http://cs.gmu.edu/~eclab/papers/panait05cooperative.pdf|title=Cooperative Multi-Agent Learning: The State of the Art |journal=Autonomous Agents and Multi-Agent Systems |volume=11 |issue=3 |pages=387–434 |year=2005 |doi=10.1007/s10458-005-2631-2|citeseerx=10.1.1.307.6671 |s2cid=19706 }}

= Self-organisation and self-direction =

Multi-agent systems can manifest self-organisation as well as self-direction and other control paradigms and related complex behaviors even when the individual strategies of all their agents are simple.{{citation needed|date=December 2016}} When agents can share knowledge using any agreed language, within the constraints of the system's communication protocol, the approach may lead to a common improvement. Example languages are Knowledge Query Manipulation Language (KQML) or Agent Communication Language (ACL).

= System paradigms =

Many MAS are implemented in computer simulations, stepping the system through discrete "time steps". The MAS components communicate typically using a weighted request matrix, e.g.

Speed-VERY_IMPORTANT: min=45 mph,

Path length-MEDIUM_IMPORTANCE: max=60 expectedMax=40,

Max-Weight-UNIMPORTANT

Contract Priority-REGULAR

and a weighted response matrix, e.g.

Speed-min:50 but only if weather sunny,

Path length:25 for sunny / 46 for rainy

Contract Priority-REGULAR

note – ambulance will override this priority and you'll have to wait

A challenge-response-contract scheme is common in MAS systems, where

First a "Who can?" question is distributed.
Only the relevant components respond: "I can, at this price".
Finally, a contract is set up, usually in several short communication steps between sides,

also considering other components, evolving "contracts" and the restriction sets of the component algorithms.

Another paradigm commonly used with MAS is the "pheromone", where components leave information for other nearby components. These pheromones may evaporate/concentrate with time, that is their values may decrease (or increase).

= Properties =

MAS tend to find the best solution for their problems without intervention. There is high similarity here to physical phenomena, such as energy minimizing, where physical objects tend to reach the lowest energy possible within the physically constrained world. For example: many of the cars entering a metropolis in the morning will be available for leaving that same metropolis in the evening.

The systems also tend to prevent propagation of faults, self-recover and be fault tolerant, mainly due to the redundancy of components.

Research

The study of multi-agent systems is "concerned with the development and analysis of sophisticated AI problem-solving and control architectures for both single-agent and multiple-agent systems."{{cite web |url=http://mas.cs.umass.edu/ |title=The Multi-Agent Systems Lab |publisher=University of Massachusetts Amherst |access-date=Oct 16, 2009}} Research topics include:

agent-oriented software engineering
beliefs, desires, and intentions (BDI)
cooperation and coordination
distributed constraint optimization (DCOPs)
organization
communication
negotiation
distributed problem solving
multi-agent learning{{citation|first1=Stefano |last1=Albrecht |first2=Peter |last2=Stone |year=2017 |contribution=Multiagent Learning: Foundations and Recent Trends. Tutorial |title=IJCAI-17 conference |url=http://www.cs.utexas.edu/~larg/ijcai17_tutorial/multiagent_learning.pdf}}
agent mining
scientific communities (e.g., on biological flocking, language evolution, and economics){{ cite journal | last =Cucker | first =Felipe | author2=Steve Smale | year =2007 | title =The Mathematics of Emergence | journal = Japanese Journal of Mathematics | volume = 2| pages = 197–227| url =http://ttic.uchicago.edu/~smale/papers/math-of-emergence.pdf | access-date = 2008-06-09 | doi =10.1007/s11537-007-0647-x | s2cid =2637067 | author2-link =Stephen Smale }}{{ cite journal | last =Shen | first =Jackie (Jianhong) | year =2008 | title =Cucker–Smale Flocking under Hierarchical Leadership | journal =SIAM J. Appl. Math. | volume =68 | issue =3 | pages = 694–719| url =http://scitation.aip.org/getabs/servlet/GetabsServlet?prog=normal&id=SMJMAP000068000003000694000001&idtype=cvips&gifs=yes | access-date = 2008-06-09 | doi =10.1137/060673254 | arxiv =q-bio/0610048| s2cid =14655317 }}

dependability and fault-tolerance
robotics,{{citation|last1=Ahmed |first1=S. |pages=459 |last2=Karsiti |first2=M.N. |url=http://eprints.utp.edu.my/320/|title=2007 IEEE International Conference on Electro/Information Technology |year=2007|doi=10.1109/EIT.2007.4374547|contribution=A testbed for control schemes using multi agent nonholonomic robots |isbn=978-1-4244-0940-2 |s2cid=2734931 |url-access=subscription }} multi-robot systems (MRS), robotic clusters
multi-agent systems also present possible applications in microrobotics,{{cite journal |last1=Yang |first1=Lidong |last2=Li |first2=Zhang |year=2021 |title=Motion control in magnetic microrobotics: From individual and multiple robots to swarms |journal=Annual Review of Control, Robotics, and Autonomous Systems |volume=4 |pages=509–534 |doi=10.1146/annurev-control-032720-104318|s2cid=228892228 }} where the physical interaction between the agents are exploited to perform complex tasks such as manipulation and assembly of passive components.
language model-based multi-agent systems

Frameworks

Frameworks have emerged that implement common standards (such as the FIPA and OMG MASIF standards).{{Cite web|url=https://www.omg.org/cgi-bin/doc?orbos/97-10-05|title=OMG Document – orbos/97-10-05 (Update of Revised MAF Submission)|website=www.omg.org|access-date=2019-02-19}} These frameworks e.g. JADE, save time and aid in the standardization of MAS development.{{cite web |first1=Salman |last1=Ahmed |first2=Mohd N. |last2=Karsiti |first3=Herman |last3=Agustiawan |title=A development framework for collaborative robots using feedback control| url=https://www.researchgate.net/publication/238431442| year=2007| access-date=8 January 2024}}

Currently though, no standard is actively maintained from FIPA or OMG. Efforts for further development of software agents in industrial context are carried out in IEEE IES technical committee on Industrial Agents.{{Cite web|url=https://tcia.ieee-ies.org/|title=IEEE IES Technical Committee on Industrial Agents (TC-IA)|website=tcia.ieee-ies.org|access-date=2019-02-19}}

With advancements in large language models (LLMs) such as ChatGPT, LLM-based multi-agent frameworks, such as CAMEL,{{Cite web |title=CAMEL: Finding the Scaling Law of Agents. The first and the best multi-agent framework. |url=https://github.com/camel-ai/camel/ |website=GitHub}} have emerged as a new paradigm for developing multi-agent applications.

== Applications ==

MAS have not only been applied in academic research, but also in industry.{{Cite book|title=Industrial agents : emerging applications of software agents in industry|others=Leitão, Paulo,, Karnouskos, Stamatis|isbn=978-0128003411|location=Amsterdam, Netherlands|oclc=905853947|last1 = Leitão|first1 = Paulo|last2 = Karnouskos|first2 = Stamatis|date = 2015-03-26}} MAS are applied in the real world to graphical applications such as computer games. Agent systems have been used in films.{{cite web |publisher=MASSIVE |url=http://www.massivesoftware.com/film.html |title=Film showcase|access-date=28 April 2012}} It is widely advocated for use in networking and mobile technologies, to achieve automatic and dynamic load balancing, high scalability and self-healing networks. They are being used for coordinated defence systems.

Other applications{{Cite journal|last1=Leitao|first1=Paulo|last2=Karnouskos|first2=Stamatis|last3=Ribeiro|first3=Luis|last4=Lee|first4=Jay|last5=Strasser|first5=Thomas|last6=Colombo|first6=Armando W.|date=2016|title=Smart Agents in Industrial Cyber–Physical Systems|journal=Proceedings of the IEEE|volume=104|issue=5|pages=1086–1101|doi=10.1109/JPROC.2016.2521931|s2cid=579475|issn=0018-9219|url=http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-128744|hdl=10198/15438|hdl-access=free}} include transportation,Xiao-Feng Xie, S. Smith, G. Barlow. [http://www.wiomax.com/team/xie/paper/ICAPS12.pdf Schedule-driven coordination for real-time traffic network control]. International Conference on Automated Planning and Scheduling (ICAPS), São Paulo, Brazil, 2012: 323–331. logistics,{{Cite journal | last1 = Máhr | first1 = T. S. | last2 = Srour | first2 = J. | last3 = De Weerdt | first3 = M. | last4 = Zuidwijk | first4 = R. | title = Can agents measure up? A comparative study of an agent-based and on-line optimization approach for a drayage problem with uncertainty | doi = 10.1016/j.trc.2009.04.018 | journal = Transportation Research Part C: Emerging Technologies | volume = 18 | pages = 99–119 | year = 2010 | issue = 1 | bibcode = 2010TRPC...18...99M | citeseerx = 10.1.1.153.770 }} graphics, manufacturing, power system,{{cite book| chapter=Generation Expansion Planning Considering Investment Dynamic of Market Participants Using Multi-agent System| author1=Kazemi, Hamidreza| author2=Liasi, Sahand| author3=Sheikh-El-Eslami, Mohammadkazem| title=2018 Smart Grid Conference (SGC)| chapter-url=https://www.researchgate.net/publication/334765661| date=November 2018| pages=1–6| access-date=8 January 2024| doi=10.1109/SGC.2018.8777904| isbn=978-1-7281-1138-4}} smartgrids,{{cite journal| title=Distributed Multi -Agent System Based Load Frequency Control for Multi- Area Power System in Smart Grid| author1=Singh, Vijay| author2=Samuel, Paulson| url=https://www.researchgate.net/publication/312304154| journal=IEEE Transactions on Industrial Electronics| volume=64| issue=6| pages=5151–5160| date=6 June 2017| access-date=8 January 2024| doi=10.1109/TIE.2017.2668983}} and the GIS.

Also, Multi-agent Systems Artificial Intelligence (MAAI) are used for simulating societies, the purpose thereof being helpful in the fields of climate, energy, epidemiology, conflict management, child abuse, ....{{Cite web|url=https://www.newscientist.com/article/mg24332500-800-ai-can-predict-your-future-behaviour-with-powerful-new-simulations/|title=AI can predict your future behaviour with powerful new simulations|website=New Scientist}}

Some organisations working on using multi-agent system models include Center for Modelling Social Systems,{{Cite web |title=Center for Modeling Social Systems - Norce |url=https://www.norceresearch.no/en/research-group/cmss |access-date=2025-04-13 |website=NORCE Norwegian Research Centre |language=en}} Centre for Research in Social Simulation,{{Cite web |title=Centre for Research in Social Simulation – A multidisciplinary centre bringing together the social sciences and agent-based modelling to promote and support the use of social simulation in research in the human sciences. |url=https://cress.soc.surrey.ac.uk/cresswp/ |access-date=2025-04-13 |language=en-GB}} Centre for Policy Modelling, Society for Modelling and Simulation International.

Vehicular traffic with controlled autonomous vehicles can be modelling as a multi-agent system involving crowd dynamics.{{cite journal |last1=Gong |first1=Xiaoqian |last2=Herty |first2=Michael |last3=Piccoli |first3=Benedetto |last4=Visconti |first4=Giuseppe |title=Crowd Dynamics: Modeling and Control of Multiagent Systems |journal=Annual Review of Control, Robotics, and Autonomous Systems |date=3 May 2023 |volume=6 |issue=1 |pages=261–282 |doi=10.1146/annurev-control-060822-123629 |language=en |issn=2573-5144|doi-access=free }}

Hallerbach et al. discussed the application of agent-based approaches for the development and validation of automated driving systems via a digital twin of the vehicle-under-test and microscopic traffic simulation based on independent agents.{{cite journal |last1=Hallerbach |first1=S. |last2=Xia |first2=Y. |last3=Eberle |first3=U. |last4=Koester |first4=F. |title=Simulation-Based Identification of Critical Scenarios for Cooperative and Automated Vehicles |journal=SAE International Journal of Connected and Automated Vehicles |volume=1 |issue=2 |page=93 |date=2018 |publisher=SAE International |doi=10.4271/2018-01-1066 |url=https://www.researchgate.net/publication/324194968}} Waymo has created a multi-agent simulation environment Carcraft to test algorithms for self-driving cars.{{cite news |last1=Madrigal |first1=Story by Alexis C. |title=Inside Waymo's Secret World for Training Self-Driving Cars |url=https://www.theatlantic.com/technology/archive/2017/08/inside-waymos-secret-testing-and-simulation-facilities/537648/ |access-date=14 August 2020 |work=The Atlantic}}{{cite journal |last1=Connors |first1=J. |last2=Graham |first2=S. |last3=Mailloux |first3=L. |title=Cyber Synthetic Modeling for Vehicle-to-Vehicle Applications |journal=In International Conference on Cyber Warfare and Security |date=2018 |page=594-XI |publisher=Academic Conferences International Limited}} It simulates traffic interactions between human drivers, pedestrians and automated vehicles. People's behavior is imitated by artificial agents based on data of real human behavior.