Induction of environment and goal models by an adaptive agent in deterministic environment ; Adaptyvaus agento aplinkos ir tikslo modelių indukcija deterministinėje aplinkoje

Owner

Sujets

• Informatics
• Planning
• Reinforcement learning

Informations

• Publié par: vytautas_magnus_university
• Publié le: 01 janvier 2011
• Nombre de lectures: 27

Extrait

VYTAUTAS MAGNUS UNIVERSITY
INSTITUTE OF MATHEMATICS AND INFORMATICS
OF VILNIUS UNIVERSITY












Jurgita Kapo či ūt ė-Dzikien ė


INDUCTION OF ENVIRONMENT AND GOAL MODELS
BY AN ADAPTIVE AGENT IN DETERMINISTIC
ENVIRONMENT





Summary of Doctoral Dissertation
Physical Sciences, Informatics (09P)

















Kaunas, 2010
Dissertation was prepared at Vytautas Magnus University in 2005 – 2010.

Scientific supervisor:
doc. dr. Gailius Raškinis (Vytautas Magnus University, Physical Sciences,
Informatics, 09P).

The dissertation is to be defended in the Joint Council on Informatics of
Vytautas Magnus University and Institute of Mathematics and Informatics of
Vilnius University.

Chairman:
prof. habil. dr. Vytautas Kaminskas (Vytautas Magnus University, Physical
Sciences, Informatics, 09P).

Members:
prof. habil. dr. Gintautas Dzemyda (Institute of Mathematics and Informatics of
Vilnius University, Physical Sciences, Informatics, 09P),
prof. habil. dr. Rimantas Šeinauskas (Kaunas University of Technology,
Technological Sciences, Informatics Engineering, 07T),
prof. habil. dr. Edmundas Kazimieras Zavadskas (Vilnius Gediminas Technical
University, Technological Sciences, Informatics Engineering, 07T),
prof. habil. dr. Antanas Žilinskas (Institute of Mathematics and Informatics of atics, 09P).

Opponents:
prof. habil. dr. Henrikas Pranevi čius (Kaunas University of Technology, Physical
Sciences, Informatics, 09P),
prof. habil. dr. Laimutis Telksnys (Institute of Mathematics and Informatics of
Vilnius University, Physical Sciences, Informatics, 09P).

The defense of dissertation is scheduled for 10 a.m. on January 24, 2011, at Vincas
Čepinskis reading room, at the Faculty of Informatics of Vytautas Magnus University.
Address: Vileikos str. 8, LT-44404, Kaunas, Lithuania.
Summary of dissertation was mailed on December , 2010.

The dissertation is available at the National M. Mažvydas Library, the Library of
Vytautas Magnus University, and the Library of Institute of Mathematics and
Informatics of Vilnius University.

VYTAUTO DIDŽIOJO UNIVERSITETAS
VILNIAUS UNIVERSITETO MATEMATIKOS IR INFORMATIKOS
INSTITUTAS












Jurgita Kapo či ūt ė-Dzikien ė


ADAPTYVAUS AGENTO APLINKOS IR TIKSLO
MODELI Ų INDUKCIJA DETERMINISTIN ĖJE APLINKOJE





Daktaro disertacijos santrauka
Fiziniai mokslai, informatika (09P)


















Kaunas, 2010
Disertacija rengta 2005 – 2010 metais Vytauto Didžiojo universitete.

Mokslinis vadovas:
doc. dr. Gailius Raškinis (Vytauto Didžiojo universitetas, fiziniai mokslai,
informatika, 09P).

Disertacija ginama jungtin ėje Vytauto Didžiojo universiteto ir Vilniaus
universiteto Matematikos ir informatikos instituto informatikos mokslo krypties
taryboje.

Pirmininkas:
prof. habil. dr. Vytautas Kaminskas (Vytauto Didžiojo universitetas, fiziniai
mokslai, informatika, 09P).

Nariai:
prof. habil. dr. Gintautas Dzemyda (Vilniaus universiteto Matematikos ir
informatikos institutas, fiziniai mokslai, informatika, 09P),
prof. habil. dr. Rimantas Šeinauskas (Kauno technologijos universitetas,
technologijos mokslai, informatikos inžinerija, 07T),
prof. habil. dr. Edmundas Kazimieras Zavadskas (Vilniaus Gedimino technikos
universitetas, technologijos mokslai, informatikos inžinerija, 07T),
prof. habil. dr. Antanas Žilinskas (Vilniaus universiteto Matematikos ir informatikos
institutas, fiziniai mokslai, informatika, 09P).

Oponentai:
prof. habil. dr. Henrikas Pranevi čius (Kauno technologijos universitetas, fiziniai
mokslai, informatika, 09P),
prof. habil. dr. Laimutis Telksnys (Vilniaus universiteto Matematikos ir
informatikos institutas, fiziniai mokslai, informatika, 09P).

Disertacija bus ginama viešame gynimo tarybos pos ėdyje, kuris įvyks 2011 m.
sausio 24 d., 10 val. Vytauto Didžiojo universiteto Informatikos fakulteto Vinco
Čepinskio skaitykloje.
Adresas: Vileikos g. 8, LT-44404, Kaunas, Lietuva.
Disertacijos santrauka išsiuntin ėta 2010 12 .

Disertacij ą galima perži ūr ėti nacionalin ėje M. Mažvydo, Vytauto Didžiojo
universiteto, Vilniaus universiteto Matematikos ir informatikos instituto bibliotekose.
INTRODUCTION
The artificial intelligence model creators refused the opinion dominating for a long
time – that the artificial intelligence can be created by aggregation of effective but
narrowly specified modules (such as speech recognition, automatic translation, proof of
theorems, machine learning, etc.) into one intellectual system. These specialized
modules are often designed by considering different fundamentals and operational
theories. Instead, the paradigm of adaptive agent was established by denoting that the
intelligence (or adaptation) arises from agent’s permanent interaction with the
environment. Through this interaction adaptive agent performs actions that change the ent; changes of the environment in turn influence its action selection. Usually
adaptive agent seeks for the optimal action that could approach it to the beneficial
situation in the environment. The relevance of the action depends on agent’s learned
knowledge. Therefore the agent can be demanded to achieve only such effectiveness as
in practice it is possible by processing and integrating available information incoming
from the environment. According to this approach it is not important that agent’s
effectiveness could be similar to human effectiveness when solving narrowly specified
tasks. The main accent is the range of solvable tasks, integrated learning and decision
making mechanisms.
Interaction between agent and its environment can be simulated in many different
ways. Compared with other similar researches, this dissertation differs in that it presents
three novel solutions for three learning problems.
The objective and tasks
The objective of this work is to create an adaptive agent, able to interact with the
grid-world environment (that is the analogue of space), functioning as the deterministic
st1 order Markov decision process. The agent must be capable to solve broader set of
adaptation tasks, compared with other known architectures of adaptive agents.
Tasks, necessary for the objective achievement are:
1. To extend the capabilities of existing adaptive agents by solving the problem of
knowledge transferability from one environment into the others (environments differ
in objects arrangements, but not in the laws). The problem has to be solved using
proposed algorithm (implemented into adaptive agent), able to learn approximation
of the laws of environment (defined as environment model).
2. To extend the capabilities of existing adaptive agents by solving the problem of goal
percepts generalization. The problem has to be solved using proposed algorithm
(implemented into adaptive agent), able to determine the features that goal situations
have (defined as goal model).
3. To extend the capabilities of existing adaptive agents, by solving perceptual aliasing
thproblem (when percepts change under the deterministic n order Markov decision
process) and allowing agent to operate in partially observable environment. The
problem has to be solved by complementation of environment model induction
thalgorithm with the capability of transforming the n order Markov decision process
stinto the 1 order.
Statement for the defense
If the laws independent from changes of states exist in observable or partially
observable environment, functioning as deterministic Markov decision process, it is
possible to create such logical induction methods (and implement into adaptive agent)
5capable to discover those laws (by learning environment and goal models) only through
interaction between agent and environment and having no initial knowledge; learned
knowledge can be transferred and applied in the unknown environments thus indicates
better adaptation of the agent.
Object of investigation
The main objects of investigation in this dissertation are: interaction between
adaptive agent and environment; the possibility of integration of learning, recognition,
prediction and planning processes; learning process of adaptive agent; the knowledge
representation.
In this dissertation the following capabilities of adaptive agent are investigated:
st• Capability of operating in the environment, functioning as the deterministic 1 order
Markov decision process and learning environment model, corresponding
deterministic finite state automaton and approximating deterministic Markov
decision process.
• Capability of learning goal model, enabling agent to search for goal situations
purposively in unknown environments (when goal situations are represented via
reinforcement).