The paper proposes knowledge representation and machine learning framework for multi-agent systems. The framework combines ontology based knowledge representation and knowledge structure update approach with rules and priority based reasoning as well as the user given feedback based machine learning algorithm. The research enables to update agents’ knowledge and also knowledge structures after launching the system. The knowledge base is provided to agents as a cloud, in particular, all these mechanisms are implemented in a centralised server that is available for all agents. The aim is to enable multi-agent systems, including multi-robot systems as one kind of multi-agent systems, to express adaptivity to changes in the environment and the system itself. The framework is the first step to the long term adaptivity and decision autonomy of multi-robot systems that is also called viability in Systems Theory. It is developed for domains where the main problem is to choose the most appropriate capability for a particular action. A heterogeneous multi-robot system for cleaning tasks is used as a case study in this paper. The multi-robot system is implemented in a simulated environment where each robot is simulated by a software agent developed in JADE agent development framework.