TY  - JOUR
PY  - 2022//
TI  - Advanced artificial agents intervene in the provision of reward
JO  - AI magazine
A1  - Cohen, Michael K.
A1  - Hutter, Marcus
A1  - Osborne, Michael A.
SP  - ePub
EP  - ePub
VL  - ePub
IS  - ePub
N2  - We analyze the expected behavior of an advanced artificial agent with a learned goal planning in an unknown environment. Given a few assumptions, we argue that it will encounter a fundamental ambiguity in the data about its goal. For example, if we provide a large reward to indicate that something about the world is satisfactory to us, it may hypothesize that what satisfied us was the sending of the reward itself; no observation can refute that. Then we argue that this ambiguity will lead it to intervene in whatever protocol we set up to provide data for the agent about its goal. We discuss an analogous failure mode of approximate solutions to assistance games. Finally, we briefly review some recent approaches that may avoid this problem.<p />  <p>Language: en</p>
LA  - en
SN  - 0738-4602
UR  - http://dx.doi.org/10.1002/aaai.12064
ID  - ref1
ER  -