As some of you might know, I have spend some of my time into getting an API for AI running for UP 1.5. It allows programmers to control all AI commands found in regular AI scripts outside of those regular AI scripts, line in C# or Python. So its not Machine Learning, but an application that makes it a lot easier for people with less, more basic, programming experience (basically no Assembly and C++ knowledge) to make something with Machine Learning or any other AI technique (I think Behaviour Trees could do wonders for some next level micro-ing).
This part works, and I'm currently testing if it works across different machines, Windows versions and UP releases. And making it a bit more friendly to use, most importantly enable Python integration, the main language used for Machine Learning. When that is done I will release it, with most of the source opened. (Some parts can be easily modified for use in multiplayer cheats, so I intent to keep that (small) part closed-source but free "as in beer") When this is done I will post a demo and a getting started. Here, on AIscripters and on discord (see below).
If needed this is the moment where you can have a minor Hype or Let's Go! moment. Because now I would like to discuss some technical things with you. I will first give you the crashiest of ML crash courses, which could hurt your brain. (Oh, and this is also the part, if you haven't done so already, where you realize there is no way to check if my previous claims are truthful.)
There are many approaches to Machine Learning, the most prominent are several variations of Deep Learning, going many layers deep. It's very basically coming up with one or several lines separating points on a multi-dimensional piece of paper, and perfecting it until you have a line that does: "if this point is above all these lines and below those we must build a barracks". Say we have that multi-dimensional point represents advancing to Feudal, having plenty of wood and making scouts versus the opponent is a good idea, we build a barracks. (Although probably the AI has different criteria than the ones I use for building a barracks.)
So how do we learn the AI this stuff? There are two ways I want to discuss.
The first one being trial and error; The AI tries stuff until it works. We rate every action he does and reward it for doing good. This means the AI might come up with new strategies we have never seen, and maybe even abuse a bug or two we didn't find yet. Could be exciting, but takes extremely long, and we will have to figure out how to reward the AI for what he does. And by doing so we might even limit new ideas, if we would reward him for making a villager, he will never come up with a strategy with a low amount of villagers. If we just use score he will scout the map 100% all the time. Or that is what I think, ML often works in mysterious ways and simple solutions often work best. Another (bigger) downside of this is that the AI will have to play millions of games to get remotely good at the game. This is a Reinforcement Learning approach, which can be combine with deep learning.
The second approach is to use recorded games. From there I think we have two options. Either (a) turn the recorded game into the available API actions or (b) use the raw data and let the AI learn what those actions do by itself. For (a) it requires a lot of extra processing for the AI engineers, but it think it will be faster at coming up with the right strategies that might be able to defeat an Easy classic AI. The other option (b) will require the AI to do the processing the AI engineers do for (a) by itself, figuring out how each command from the API should be used and what effect it has, and how those commands can mimic the recorded game it is given. The latter is the more prominent solution, that works by throwing a lot of computation power to the problem. (In this case also computation power for running AoE2)
Now for a bit more more practical stuff I need some advice for.
I hope both approaches, and the two options make at least some sense. Since I will be releasing the API, you can try both by yourself. I however will be focusing on the easiest approach (2a) which doesn't require me to buy very expensive hardware.
Let's say we want to have AI's competing with each other, but at the same time keep the things they have to learn at a minimum and have a lot of recorded games for. This is where I lack some knowledge:
I was thinking about Huns wars (both sides are Huns so AI only have to learn 1 Civ). They used to be very popular (a lot of recorded games) in the 1.0c patch (so they don't have to cope with a lot of different balances - 1.0c is a single balance right?). They are usually played on Arabia (1 map type). And they tend to be reasonably fast paced while still going through most stages of the game. (Learn a lot in less time.)
Can someone confirm that these assumptions are true? And perhaps point me to a trove of recorded game of high ranked 1.0c Huns wars?
I think a good first challenge for this approach would be getting to Feudal Age. If we can make that work I think we can do a lot more with the API.
And I would like to have some feedback. Are you interested in working on a ML AI? What do you think about the approach I am taking? Are you interested in the Machine Learning topic? Any other thoughts?
I'm not really interested in if you think it will work or not, though. I personally don't think the AI's will ever be better than what we currently have, but that doesn't make it less educational for me (or you). Currently Machine Learning isn't the just add water solution to all of life's problems.
Do keep in mind that this is a personal project I do in my free time because I think it's fun, and I have many other things competing for that time. So "soon TM" has another meaning for this project. So at some point I will be asking for help.
I created a discord channel for this. It's probably pointless at this point because it's empty and I might not actively participate until the first version of the API is done (which I think is in the next few weeks), so it's more of a waiting room. Although I think I will use it for casual development updates, and maybe some basic education, I don't expect anyone to join (yet). But if you are interested in this project you are very welcome to join, and it shows you being interested in this project, which motivates me: https://discord.gg/njAsNuD
Edit: Added some highlights to reduce the amount of people commenting that clearly didn't read that this is not the place to discuss the viability of ML. I do it's because it's fun, and comments like that are not, and we already had them in another thread. Also this API allows for other AI methods as well, like Behaviour Trees which are used in most modern games that don't utilize ML techniques yet.
This part works, and I'm currently testing if it works across different machines, Windows versions and UP releases. And making it a bit more friendly to use, most importantly enable Python integration, the main language used for Machine Learning. When that is done I will release it, with most of the source opened. (Some parts can be easily modified for use in multiplayer cheats, so I intent to keep that (small) part closed-source but free "as in beer") When this is done I will post a demo and a getting started. Here, on AIscripters and on discord (see below).
If needed this is the moment where you can have a minor Hype or Let's Go! moment. Because now I would like to discuss some technical things with you. I will first give you the crashiest of ML crash courses, which could hurt your brain. (Oh, and this is also the part, if you haven't done so already, where you realize there is no way to check if my previous claims are truthful.)
There are many approaches to Machine Learning, the most prominent are several variations of Deep Learning, going many layers deep. It's very basically coming up with one or several lines separating points on a multi-dimensional piece of paper, and perfecting it until you have a line that does: "if this point is above all these lines and below those we must build a barracks". Say we have that multi-dimensional point represents advancing to Feudal, having plenty of wood and making scouts versus the opponent is a good idea, we build a barracks. (Although probably the AI has different criteria than the ones I use for building a barracks.)
So how do we learn the AI this stuff? There are two ways I want to discuss.
The first one being trial and error; The AI tries stuff until it works. We rate every action he does and reward it for doing good. This means the AI might come up with new strategies we have never seen, and maybe even abuse a bug or two we didn't find yet. Could be exciting, but takes extremely long, and we will have to figure out how to reward the AI for what he does. And by doing so we might even limit new ideas, if we would reward him for making a villager, he will never come up with a strategy with a low amount of villagers. If we just use score he will scout the map 100% all the time. Or that is what I think, ML often works in mysterious ways and simple solutions often work best. Another (bigger) downside of this is that the AI will have to play millions of games to get remotely good at the game. This is a Reinforcement Learning approach, which can be combine with deep learning.
The second approach is to use recorded games. From there I think we have two options. Either (a) turn the recorded game into the available API actions or (b) use the raw data and let the AI learn what those actions do by itself. For (a) it requires a lot of extra processing for the AI engineers, but it think it will be faster at coming up with the right strategies that might be able to defeat an Easy classic AI. The other option (b) will require the AI to do the processing the AI engineers do for (a) by itself, figuring out how each command from the API should be used and what effect it has, and how those commands can mimic the recorded game it is given. The latter is the more prominent solution, that works by throwing a lot of computation power to the problem. (In this case also computation power for running AoE2)
Now for a bit more more practical stuff I need some advice for.
I hope both approaches, and the two options make at least some sense. Since I will be releasing the API, you can try both by yourself. I however will be focusing on the easiest approach (2a) which doesn't require me to buy very expensive hardware.
Let's say we want to have AI's competing with each other, but at the same time keep the things they have to learn at a minimum and have a lot of recorded games for. This is where I lack some knowledge:
I was thinking about Huns wars (both sides are Huns so AI only have to learn 1 Civ). They used to be very popular (a lot of recorded games) in the 1.0c patch (so they don't have to cope with a lot of different balances - 1.0c is a single balance right?). They are usually played on Arabia (1 map type). And they tend to be reasonably fast paced while still going through most stages of the game. (Learn a lot in less time.)
Can someone confirm that these assumptions are true? And perhaps point me to a trove of recorded game of high ranked 1.0c Huns wars?
I think a good first challenge for this approach would be getting to Feudal Age. If we can make that work I think we can do a lot more with the API.
And I would like to have some feedback. Are you interested in working on a ML AI? What do you think about the approach I am taking? Are you interested in the Machine Learning topic? Any other thoughts?
I'm not really interested in if you think it will work or not, though. I personally don't think the AI's will ever be better than what we currently have, but that doesn't make it less educational for me (or you). Currently Machine Learning isn't the just add water solution to all of life's problems.
Do keep in mind that this is a personal project I do in my free time because I think it's fun, and I have many other things competing for that time. So "soon TM" has another meaning for this project. So at some point I will be asking for help.
I created a discord channel for this. It's probably pointless at this point because it's empty and I might not actively participate until the first version of the API is done (which I think is in the next few weeks), so it's more of a waiting room. Although I think I will use it for casual development updates, and maybe some basic education, I don't expect anyone to join (yet). But if you are interested in this project you are very welcome to join, and it shows you being interested in this project, which motivates me: https://discord.gg/njAsNuD
Edit: Added some highlights to reduce the amount of people commenting that clearly didn't read that this is not the place to discuss the viability of ML. I do it's because it's fun, and comments like that are not, and we already had them in another thread. Also this API allows for other AI methods as well, like Behaviour Trees which are used in most modern games that don't utilize ML techniques yet.
Last edited: