Group contributions: theme definition
The sessions realized in the second week focused on setting up the game’s theme and clarifying the strategy and work organization. I focused on developing a basic AI model to gain some knowledge related to how to work with AI/ML agents in Unity.
At the definition of the theme, we presented our personal ideas and tried to sell them to the other member. In the end, we selected what had the best fit with our tastes, and we got to the definition of the tone of the game: in a future dystopian society, every individual is controlled by chips that model the behavior and shape the world. For an unknown reason, the controllers of the protagonist are malfunctioning, and they can attempt to escape from his reality. This concept was strongly influenced by Lukes, the art specialist team member.
Focusing on the usage of Machine Learning and how to implement/add it to the core mechanics, I suggested the use of auto-generated pieces of memory represented by haiku (reference). However it this idea didn’t contribute to the gameplay or the structure of the proposed game. By the end of the session, after some divagations, I proposed the usage of improved AI applied to the Enemies using Machine Learning generated models. It would bring some value to the project and its practical implementation.
As a group, we also defined our backlog, at least the first view of it, and build and started our first sprint using Jira. It was a different experience working with a much more open approach. Usually, in my daily routine, we have a huge amount of limitations, and our team is specialized in a couple of technologies that are adherent to the business. And at this team, the responsibilities are related to a broader knowledge spectrum. Linda started building a mood board with the tones that we are going to use. It will be fundamental to another part of the game, which we didn’t start to discuss yet. Nural was totally had much more concern with the narrative. I felt much more comfortable focusing completely on Machine Learning and on the direct possibilities of application.
Besides the positive aspects, the group interaction seems to be at the earlier stages. We are learning about each other and building trust. At the moment, no clear mistakes were made or discussions were triggered. I wonder if, with our available time, we are going to move forward in the leverage of the group relationships.
Besides all the discussions, I also shared some specific content related to 2D with Dan and set up our repository in GitHub: https://github.com/orgs/The-Unknown-Team-gpd30/.
Personal development: Machine Learning adventures
I got deeply interested in the applications. I’m focused at the moment on the role of AI as Actor applied to game development. Therefore I’ve studied the construction of an agent that can be used as an enemy in the game. All-time that I invested was doing tutorials and reading the documentation of Unity ML-Agents Toolkit, which enables games to serve as environments for training intelligent agents.
The usage is initially fairly simple, although the initial expectations I had to spend a couple of hours only configuring the local environment and understanding the core/related concepts.
Backing to the basics, Machine Learning is a branch of artificial intelligence and computer science focused on the usage of algorithms to imitate the way humans learn. When applied to the role of Actor in Game development, it can be used to mimic the human behavior in Non-Playable Character or even to update the game difficulty as the player interact with the environment.
Exists three main classes of ML algorithms: unsupervised learning, supervised learning, and reinforcement learning. Each class used a different strategy to learn and create a model.
Models are the data structure used to describe the learned behavior and used to replicate and improve it over time.
The training is the phase where multiple environments are build and given some levels or randomization is collected data related to the efficiency of the multiple attempts of the algorithms used to achieve a proposed goal.
In Unity, I’ve been using the class of ML reinforced learning. This particular class proposes a policy that drives the training phase named the reinforced learning circle:
It makes clear how we should the execution steps should be organized to be possible the correct execution and training of a model.
I first created an agent using the toolkit ML-Agents provided by Unity. This is a simple application where the agent should find a goal. Each time this agent finds the goal, it receives a reward. I used the basic example provided as a reference by the toolkit (Unity-Technologies/ml-agents, 2021).
The main reward is collecting a ball or touching it. Each time it happens, the environment is restarted in a different configuration. The agent is aware of the position of the ball, and the ball position is fixed, so it reduces the number of variables that are going to be needed for this particular session.
I also build an execution environment with multiple environments in order to reduce the amount of time spent in the training sessions.
Follow an example, after a couple of hours learning about all the configurations:movie_001.mp4 – 4MB
And after that applying the randomization effect.movie_002.mp4 – 2MB
My next step is to increase the number of variables and introduce the usage of guided learning so I can also interfere and help to improve the performance of the model.
At the moment, I found some limitations related to the memory of the agent, which I believe can be solved with the usage of a different type of Neural Network. Besides that, my initial expectation is that the usage of ML can leverage the level of the enemies and allies in the game without the implementation of extensive rule conditions with enormous chains of if/else in the code.
Code Monkey (2020) How to use Machine Learning AI in Unity! (ML-Agents). Available at: https://www.youtube.com/watch?v=zPFU30tbyKs&list=PLzDRvYVwl53vehwiN_odYJkPBzcqFw110&index=5&ab_channel=CodeMonkey (Accessed: 13 June 2021).
Guzdial, M. et al. (2018) ‘Explainable PCGML via Game Design Patterns’, arXiv:1809.09419 [cs]. Available at: http://arxiv.org/abs/1809.09419 (Accessed: 13 June 2021).
Rieder, B. (no date) ‘Using Procedural Content Generation via Machine Learning as a Game Mechanic’, p. 108.Samuel, A. L. (no date) ‘Some studies in machine learning using the game of checkers’, p. 13.Traditional Game AI vs. ML-Agents & Hummingbird Course Preview (no date) Unity Learn. Available at: https://learn.unity.com/tutorial/traditional-game-ai-vs-ml-agents-hummingbird-course-preview (Accessed: 18 June 2021).
Unity-Technologies/ml-agents (2021). Unity Technologies. Available at: https://github.com/Unity-Technologies/ml-agents/blob/59211621fc23b8b0064c527d4a1363acb9f59adb/docs/Background-Machine-Learning.md (Accessed: 19 June 2021).
Unity-Technologies/ml-agents: Unity Machine Learning Agents Toolkit (no date). Available at: https://github.com/Unity-Technologies/ml-agents (Accessed: 18 June 2021).
What is Machine Learning? (2021). Available at: https://www.ibm.com/cloud/learn/machine-learning (Accessed: 18 June 2021).