More Varied AI Research Paths

[harpy eagle]

I've been thinking of starting a mod that varies the AI research paths a little bit. Currently it seems like the AI always researches the same things in roughly the same order (also it was really bugging me how the Ashdar Colonials and Imperials don't research Escort or Light Carriers, but instead only research Fleet Carriers much later in the game). I took a look through the code and found that indeed, the AI always researches techs according to the same predefined list.

While the list seemed kind of outdated, I thought it would make the AI more interesting if varied what parts of the tech tree it explored each game. So I created a bunch of pre-defined lists of techs representing various tech focuses that I could think of e.g. kinetic weapons, industrial production, pop growth, etc. And made the AI randomly choose to prefer various focuses over others each game.

This isn't really be aimed at improving the AI's performance so much as making the AI a bit more interesting. Instead of always facing lasers+missiles, then ions, then disruptors, your AI opponent might choose to research a different part of the tech tree in a different order than they always do in vanilla.

So for example, in some games the an AI empire might decide it wants to focus on kinetic weapons instead of energy beams. Or go for carriers before heavy cruisers, instead of always going for heavy cruisers first.

I'm thinking the approach I'd use is to create a bunch of canned lists of techs to research, that the AI switches between based on a set of desires that are randomly set at the beginning of the game. Although I guess there could be some context involved as well, like e.g. weapon/shield/armour techs being more desired if the empire is at war.

Any thoughts or suggestions? I'll post some of the tech lists I'm thinking of when I have them.

EDIT: I've implemented the research choice logic. As you can see, the AI can now actually design ships with coilguns
Since there's a lot more variety in the selection of weapons available to the AI, this mod enables the game to make much fuller use of the autodesign code.



The AI's research desires aren't totally random. I've implemented some logic so to ensure that e.g. phidi always value economy techs to at least a certain degree, and the same for orthin and energy weapons/shields etc.

If an empire doesn't have any research desires, it will initialize them the next time the empire tries to decide on a tech to research, no need to start a new game to use the mod. Now I'm just hoping that I can store each empire's research desires inside the empire table and it will persist across saved games.

The AI also seems to make stronger research choices, just due to the fact that it used to completely ignore certain branches of the tech tree.

Update v1.1

I've made some significant improvements to how the AI chooses to spread it's science income into different research paths, that will allow the AI to more clearly express it's strongest research desires, and help the AI make better choices by taking into account research paths that have techs in common.

LAST UPDATED: 2018/04/29

[harpy eagle]

Here you can see the AI Yoral have researched and built an advanced destroyer. I don't think I've ever seen them do that in vanilla. It's equipped with nuclear torpedoes too. This is turn 1160. Usually the AI never goes for torpedoes until at least fusion.

[sven]

The AI's research desires aren't totally random. I've implemented some logic so to ensure that e.g. phidi always value economy techs to at least a certain degree, and the same for orthin and energy weapons/shields etc.

I think going this route with 'static', entirely canned paths is probably sufficient. In theory, we could imagine the AI being really adaptive, and, say, pushing into pd weapons if it's fighting someone using a missile heavy strat. But, I suspect that might actually be over-optimizing things.

If an empire doesn't have any research desires, it will initialize them the next time the empire tries to decide on a tech to research, no need to start a new game to use the mod. Now I'm just hoping that I can store each empire's research desires inside the empire table and it will persist across saved games.

It should persist if you stick in an empire key -- though putting it in empire.ai_state might be more 'correct'.

[harpy eagle]

It should persist if you stick in an empire key -- though putting it in empire.ai_state might be more 'correct'.

Thanks, I'm doing that instead now.

Found and fixed a few bugs (392bac9). Forgot to filter out impossible techs when evaluation one_of() groups, and weapon research paths and their heavy variants shouldn't clobber each other any more. However, the AI desires for those paths will still be linked.

I've also decided to loosen their relationship. Instead of using the same desire for both e.g. lasers and heavy lasers, the desire for the heavy variants will be limited if the desire for the former is low. So for example, the AI will not desire heavy lasers unless it has at least a marginal desire for lasers, however it can have a high desire for lasers without wanting heavy lasers.

[zolobolo]

The AI sometimes produces Advanced Destroyers in Vanilla but it is very rare and without the use of the Strategic AI mod it is very difficult to catch that - only saw it when playing against 3 other Yoral

Also noticed that the AIs all seem to take the same research route - so will try it out with the mod next

[harpy eagle]

There might be some issues with the AI not always picking the best weapons available. For example, in a recent game, I saw the ashdar colonials research heavy drivers, but continued to equip their heavy cruisers with regular lasers

Kind of beyond the scope of this mod though.

[sven]

There might be some issues with the AI not always picking the best weapons available. For example, in a recent game, I saw the ashdar colonials research heavy drivers, but continued to equip their heavy cruisers with regular lasers

Yeah, that sounds kinda like a bug in the autobuild mechanic. If you see this happening again, would you upload the game?

[sven]

Found and fixed a few bugs (392bac9). Forgot to filter out impossible techs when evaluation one_of() groups, and weapon research paths and their heavy variants shouldn't clobber each other any more. However, the AI desires for those paths will still be linked.

Did you ever experiment with different exponential spreads? i.e., tweaking the formula in:

Code: Select all

    desired_research_spread[path] = 2^desire

That the target spread is at root exponential makes sense (because the tech costs are roughly exponentially increasing), but, there's plenty of different exponential spreads one might use here. Are you confident 2 is a large enough base?

[zolobolo]

Yeah, that sounds kinda like a bug in the autobuild mechanic. If you see this happening again, would you upload the game?

It is not exactly the same as when using an inferior version of the same weapon (like using nuclear instead of fusion missile) but the below example shows the Orthin building Gunships with Missiles instead of the Ion Battery they already have researched:

Save: https://www.dropbox.com/s/fbcglyngz7gcy ... g.rar?dl=0

[harpy eagle]

Did you ever experiment with different exponential spreads? i.e., tweaking the formula in:

Code: Select all

    desired_research_spread[path] = 2^desire

That the target spread is at root exponential makes sense (because the tech costs are roughly exponentially increasing), but, there's plenty of different exponential spreads one might use here. Are you confident 2 is a large enough base?

Well, I am pretty confident though that it shouldn't be made any smaller. The value there was chosen more as a reasonable starting point so I could continue with development. It's definitely a parameter that is open for tweaking.

But, now that we're thinking about it:

That value basically controls the "dynamic range" between the most desired and least desired research paths for the AI.

However, this is also pretty closely tied with how we go about initializing the AIs desires for the different paths, and there's probably a variety of methods we could use there as well (more on that later).

There's some logic to handle species tech preferences, but for the most part it just assigns a random, independent number for each desire between -1 and +1. According to Wikipedia the the standard deviation interval for this is -0.577 to +0.577.

So if we plug that in, we find that the AI will want to spend about twice the number of beakers on a typical "desired" tech path vs a typical "undesired" tech path (2^(0.577+0.577) ~ 2.2).

I guess this is kind of low. Maybe four times, five times would be more reasonable? For a different angle, we could also think about how much spread would be too much. e.g. would ten times be too extreme?

Also, if the goal is to give the AI distinct "personalities", at least as reflected by their tech choices, maybe setting the desires using independent, random numbers isn't be best approach.

For example, we could instead shuffle the list of possible tech paths, and assign them increasing desires. This would ensure that there is always some research paths that are strongly desired, and others that are undesired.

But maybe I'm starting to overthink the issue.

[sven]

There's some logic to handle species tech preferences, but for the most part it just assigns a random, independent number for each desire between -1 and +1. According to Wikipedia the the standard deviation interval for this is -0.577 to +0.577.

So if we plug that in, we find that the AI will want to spend about twice the number of beakers on a typical "desired" tech path vs a typical "undesired" tech path (2^(0.577+0.577) ~ 2.2).

I guess this is kind of low. Maybe four times, five times would be more reasonable? For a different angle, we could also think about how much spread would be too much. e.g. would ten times be too extreme?

I'm not sure either, though my *hunch* is that if there's a tech you really want priced at 1000, and one you don't particularly want priced at 250, you're probably going to go for the 1000 tech most of the time. So, 5-8x could maybe be not wrong? That said, I did some testing with the mod last night, and from what I can tell, the current values are in fact working pretty well, in the sense that the AI will noticeably favor the tech paths it's supposed to be favoring.

Also, if the goal is to give the AI distinct "personalities", at least as reflected by their tech choices, maybe setting the desires using independent, random numbers isn't be best approach.

For example, we could instead shuffle the list of possible tech paths, and assign them increasing desires. This would ensure that there is always some research paths that are strongly desired, and others that are undesired.

But maybe I'm starting to overthink the issue.

I've been thinking the same thing, and I actually don't think it's a case of over designing The core issue, I think, is that to really get the AI's behaving as if they're pursuing different strategies, it's not enough to just change their research priorities (though that's a necessary first step), we also need to change their target navy compositions. Right now, the navy composition target rules are overly simple, but, I think an ideal system would take into account the "build strategy" that the AI is going for. So, for example, if you roll carriers=1, you really should be biased towards *building* carriers, as well as researching them.

Here some kind of card system starts to feel natural. I'm not quite sure how it would work, but, at a high level, the main thing it needs to do is allow us to derive ship build priorities and research priorities from the same underlying master plan. At the simplest level, we could simply hardcode a large number of different "strategic plans" that would include a full research priority set + some similar sort of build priority set. A slightly more clever approach might be to draw a couple different "cards" that would somehow, when taken together, imply a complete set of planning data. So, for example, you might draw "carriers+kinetics+fast expand", or "industry+missiles+heavy armor" or "destroyer spam+energy weapons+research", and that card set would then dictate how the more detailed research and build priority arrays were assigned.

[sven]

Here some kind of card system starts to feel natural.

Right now I'm daydreaming about how, exactly, such a multi-card system might work. We could imagine drawing 1 card from 3 different decks, called, say, "economy", "hulls", and "weapons" (where each card would themselves serve as some kind of abstractions bundling together some of the more specific research path definitions you've created). But, just doing that naively might lead to weird combinations; like the afore mentioned "research"+ "destroyers"+"heavy energy". A slightly more sophisticated approach might be to let hulls and economy cards include special rules for "pruning" the weapon tech draws. So, for example, if you're anyone but Yoral, and you draw "destroyers" your weapon deck is pruned so you can't draw "heavy energy". We could also stack the decks differently for different races -- so, Yoral would have more missile/destroyer cards in their weapons/hulls decks, Orthin would perhaps *always* be limited to a relatively small set of thematically appropriate weapon and economy choices, etc.

(All of this is not dissimilar to things you're already doing with the research path 'flags' and 'contingent_on' variables, but, it would lean a bit harder into the idea of using deck compositions to give the game designers more control over just what sorts of strategies tend to emerge from the system.)

[sven]

(where each card would themselves serve as some kind of abstractions bundling together some of the more specific research path definitions you've created)

Maybe each high-level card includes some sort of rule for how it would modify the initial underlying random roll for each research path? So, you'd have something like:

Code: Select all

energy_weapons = {
	lasers = {.5,1},
	heavy_lasers={.5,1},
	missiles = {-1,0},
	heavy_missiles={-1,0},
	...
}

[harpy eagle]

Right now I'm daydreaming about how, exactly, such a multi-card system might work. We could imagine drawing 1 card from 3 different decks, called, say, "economy", "hulls", and "weapons" (where each card would themselves serve as some kind of abstractions bundling together some of the more specific research path definitions you've created).

The cards sound like a good idea. It would give you a lot more control over how things are randomized, which would then allow you to ensure that the AI empires get distinct and coherent, yet not entirely predictable, personalities.

I'm not sure either, though my *hunch* is that if there's a tech you really want priced at 1000, and one you don't particularly want priced at 250, you're probably going to go for the 1000 tech most of the time. So, 5-8x could maybe be not wrong? That said, I did some testing with the mod last night, and from what I can tell, the current values are in fact working pretty well, in the sense that the AI will noticeably favor the tech paths it's supposed to be favoring.

Well, I have a hunch that a little more specialization won't hurt, so I'll go with 5x.

I'm also thinking of adding a research path for planetary defenses and/or troop upgrades, and nudging the Gremak a little bit towards combat boarders and battleships.

[harpy eagle]

Yeah, that sounds kinda like a bug in the autobuild mechanic. If you see this happening again, would you upload the game?

I had another look at the save. It wasn't a bug after all. The AI was determining correctly that lasers in a heavy mount with the RF mod did more damage on average than heavy coilguns.