NDLI: An agent structure for evaluating micro-level MAS performance

Please wait, while we are loading the content...

Evaluation of navigation of an autonomous mobile robot

Assessing the impact of bi-directional information flow in UGV operation: a pilot study

A common operator control unit color scheme for mobile robots

How DoD's TRA process could be applied to intelligent systems development

A brief history of PRIDE

Autonomy levels for unmanned systems (ALFUS) framework: safety and application issues

Evaluation of autonomy in recent ground vehicles using the autonomy levels for unmanned systems (ALFUS) framework

A methodology for testing unmanned vehicle behavior and autonomy

Standardizing measurements of autonomy in the artificially intelligent

Assessment of man-portable robots for law enforcement agencies

Performance metrics and evaluation of a path planner based on genetic algorithms

The evolution of performance metrics in the RoboCup Rescue Virtual Robot Competition

Robot simulation physics validation

Design and validation of a Whegs robot in USARSim

Maze hypothesis development in assessing robot performance during teleoperation

Human system performance metrics for evaluation of mixed-initiative heterogeneous autonomous systems

Concepts of operations for robot-assisted emergency response and implications for human-robot interaction

Multimodal displays to enhance human robot interaction on-the-move

Autonomy (what's it good for?)

Definitions and measures of intelligence in Deep Blue and the Army XUV

Automotive Turing Test

Autonomous robots with both body and behavior self-knowledge

A cognitive-based agent architecture for autonomous situation analysis

Evaluation of an integrated multi-task machine learning system with humans in the loop

Survey measures for evaluation of cognitive assistants

Development of tools for measuring the performance of computer assisted orthopaedic hip surgery systems

Haptic feedback system for robot-assisted surgery

Prototype rover field testing and planetary surface operations

Planning to fail: reliability as a design parameter for planetary rover missions

A decision space compression approach for model based parallel computing processes

Physically-proximal human-robot collaboration for air and space applications

Analyzing the performance of distributed algorithms

An agent structure for evaluating micro-level MAS performance

Information management for high performance autonomous intelligent systems

Efficient Monte Carlo computation of Fisher information matrix using prior information

Performance of 6D LuM and FFS SLAM: an example for comparison using grid and pose based evaluation methods

Smart assembly: industry needs and challenges

Science based information metrology for engineering informatics

Evaluating manufacturing machine control language standards: an implementer's view

Interoperability testing for shop floor measurement

Virtual Mentor: a step towards proactive user monitoring and assistance during virtual environment-based training

Report on panel discussion on (Re-)Establishing or Increasing Collaborative Links Between Artificial Intelligence and Intelligent Systems

An agent structure for evaluating micro-level MAS performance

Content Provider	ACM Digital Library
Author	Mitkas, Pericles A. Symeonidis, Andreas L. Dimou, Christos
Abstract	Although the need for well-established engineering approaches in Intelligent Systems (IS) performance evaluation is urging, currently no widely accepted methodology exists, mainly due to lack of consensus on relevant definitions and scope of applicability, multi-disciplinary issues and immaturity of the field of IS. Even existing well-tested evaluation methodologies applied in other domains, such as (traditional) software engineering, prove inadequate to address the unpredictable emerging factors of the behavior of intelligent components. In this paper, we present a generic methodology and associated tools for evaluating the performance of IS, by exploiting the software agent paradigm as a representative modeling concept for intelligent systems. Based on the assessment of observable behavior of agents or multi-agent systems, the proposed methodology provides a concise set of guidelines and representation tools for evaluators to use. The methodology comprises three main tasks, namely metrics selection, monitoring agent activities for appropriate measurements, and aggregation of the conducted measurements. Coupled to this methodology is the Evaluator Agent Framework, which aims at the automation of most of the provided steps of the methodology, by providing Graphical User Interfaces for metrics organization and results presentation, as well as a code generating module that produces a skeleton of a monitoring agent. Once this agent is completed with domain-specific code, it is appended to the runtime of a multi-agent system and collects information from observable events and messages. Both the evaluation methodology and the automation framework are tested and demonstrated in Symbiosis, a MAS simulation environment for competing groups of autonomous entities.
Starting Page	229
Ending Page	236
Page Count	8
File Format	PDF
ISBN	9781595938541
DOI	10.1145/1660877.1660910
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2007-08-28
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Automated evaluation Autonomous agents Performance evaluation methodology Multi-agent systems
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in