Question about modding the preBuiltArt

[Beetlezombie]

Hello

Just a small question about the difference of the file in the preBuiltArt folder and the files in the luastate\drawers folder (emperors, ships etc).

Is the preBuiltArt for the locked races only?

[sven]

Hello

Just a small question about the difference of the file in the preBuiltArt folder and the files in the luastate\drawers folder (emperors, ships etc).

Is the preBuiltArt for the locked races only?

The art asset caching system is complicated. Um, I think it may have been discussed in more detail inside other threads, but, basically, the game includes all sorts of tools for "building" the art assets that will appear. The data inside Lua state\Drawers usually contains exported Photoshop layers for Arioch's master art files. But, there's often only 1 master file for ship graphics that may need to appear with a dozen different recolor schemes over the course of the game. Thus, the game includes a lot of tools to "build" ship graphics with different color settings. Building a new color scheme is relatively slow, so the built ship graphics are cached in Lua sate\ArtCache everytime they're built.

The PreBuiltArt folder contains recolors that we expect to see relatively frequently, so they're distributed "pre built" so that players won't need to wait for asset build times the first time they see a particular ship. In theory, you can safely delete both your PreBuiltArt and ArtCache directories, and as you play the game, the art rendering engine will slowly populate ArtCache with the assets that it needs.

Now, what's strange is that it's not only the ship drawers that work like this -- population art, character art, and even certain UI frames are all "built" from master files in much the same way as ships. There's not a great reason for this -- other than the fact that I had a system in place for getting Arioch's ship art into the game, and modifying it slightly to work with character and race art as well was easier than building a new system from scratch.

The bottom line is that all this complexity makes doing something that may seem like it should be simple (like, say, adding new character or race art), harder than it probably should be. But, it also means that introducing new ship color schemes can be surprisingly easy.

What are you trying to do in your mod?

[sven]

See also this thread.

[Beetlezombie]

Thank you for the fast reply.

Basically I'm trying to go Total Conversion on SiS. I'm dabbling in the prebuilt emperors and ship art to get a feel on things. I've modded Space Empires 4 over and over in the past, and I recently got SiS in a Steam sale because it had that mix between SE4 and MoO2.

Just finished reading the thread you linked ... phew.

The bottom line is that all this complexity makes doing something that may seem like it should be simple (like, say, adding new character or race art), harder than it probably should be. But, it also means that introducing new ship color schemes can be surprisingly easy.

Basically I'm trying to add static png art. And replacing the prebuilt files kinda does that.

This means, from a modding perspective, all you should ever need to distribute is the Drawers assets; as long as they exist, the ArtCache files will be created implicitly for anyone who has them.

Think this will be the best but slow option ... but ...

What is constructed from Drawers is built into ArtCache/. PrebuiltArt is a copy of the ArtCache/ dir on my dev machine. The key difference between ArtCache and PrebuiltArt is that art files that exist in PrebuiltArt *never* get entirely rebuilt, instead we just assume that the metadata in Drawers/ will match the output images in PrebuiltArt. ArtCache files, on the other hand, are implicitly rebuilt anytime the source Drawers data changes.

* Does the art in prebuiltart need to be rebuilt? Ever? I would guess not right?

* Why wouldn't the metadata in drawers not match the output images in prebuiltart? Is the assumtion that the prebuiltart is static for the 8 stock empires/races wrong?

* Artcache files are rebuilt anytime the source drawers data changes. When does that happen without modding? When the game randomly generates color schemes for lesser races. Or when you generate a large galaxy with max empires?

Sorry for the barrage of questions

o7
BZ

[sven]

* Does the art in prebuiltart need to be rebuilt? Ever? I would guess not right?

Hypothetically, it should never need to be rebuilt by a player. And if you're a modder, distributing a modified copy of PreBuiltArt is probably unnecessary. (Unless, perhaps, you're doing something like a total conversion mod, in which case... replacing people's PreBuiltArt is maybe a good idea?)

* Why wouldn't the metadata in drawers not match the output images in prebuiltart? Is the assumtion that the prebuiltart is static for the 8 stock empires/races wrong?

The contents of PreBuiltArt are generated from source data in Lua state\Drawers. If I update the source art, and push a patch, that patch should also include an update to PreBuiltArt that corrects any impacted assets to match the new specs.

However, the scripts I use to rebuild PreBuiltArt aren't perfect, and they do miss a few things (usually weapon recolors and certain ship submodule art). Because of those "holes", players will also have an ArtCache dir that holds whatever stuff they've encountered in-game that PreBuiltArt is missing.

* Artcache files are rebuilt anytime the source drawers data changes. When does that happen without modding? When the game randomly generates color schemes for lesser races. Or when you generate a large galaxy with max empires?

Technically ArtCache files are generated any time the game needs an asset that it can't yet find in PreBuiltArt. So, you could trigger it while live-coding, if you tried to display a ship with a color scheme that isn't included in the PreBuiltArt files. Or during normal gameplay, anytime you encounter a place where my caching scripts have overlooked an asset that the game needs. (Usually, that happens when captured ships need to be displayed with unusual color schemes.)

Note that the engine will always trust that any asset in PreBuiltArt is up to date with the source art, but, it won't make the same assumption about assets in ArtCache. So, if you're going to be actively changing source art in Drawers, you may need to delete your copy of PreBuiltArt, so that it won't inhibit auto-generation of assets.

As I said in that other thread, I don't run the game using a PreBuiltArt folder, instead I rely on auto-generation to keep my copy of ArtCache up to date, and then my dev machine's ArtCache becomes everyone else's PreBuiltArt anytime I roll out a patch.

If you're going to do a total conversion, the way to do it might be to imitate that setup. Delete PreBuiltArt from your dev machine entirely, just work with the source assets by changing the data in Lua state/Drawers, rely on autogeneration to keep a relatively complete ArtCache directory on your own machine, and then setup your mod installer to replace people's PreBuiltArt dir with a copy of your own ArtCache.

Um, yes, it is all probably more awkward and confusing than it needs to be. But if you start really digging deep, you'll discover that there are Photoshop scripting utilities built into the game that you can use to convert PSD files into assets for Lua state\Drawers. So... if you have source art in Photoshop, there are actually tools included inside the game itself that you can use to import that art into the game.

[sven]

Delete PreBuiltArt from your dev machine entirely, just work with the source assets by changing the data in Lua state/Drawers, rely on autogeneration to keep a relatively complete ArtCache directory on your own machine, and then setup your mod installer to replace people's PreBuiltArt dir with a copy of your own ArtCache.

The other option, of course, is to just ignore the auto-asset generation stuff entirely, and hack in images based on raw PNG files that you provide. In some cases, this will be easier. But, it's not always going to be easy -- and if you really want to mod a lot of the game's art using this technique, you may end up being forced to write a collection of relatively low-level utility functions and other special case hooks to help with the process. This is also... probably possible? But how hard it is would depend on exactly what parts of the game you're interested in modding. (Ships would get hairy. Most of the character and race art might be relatively easy though.)

[Beetlezombie]

If you're going to do a total conversion, the way to do it might be to imitate that setup. Delete PreBuiltArt from your dev machine entirely, just work with the source assets by changing the data in Lua state/Drawers, rely on autogeneration to keep a relatively complete ArtCache directory on your own machine, and then setup your mod installer to replace people's PreBuiltArt dir with a copy of your own ArtCache.

Thanks for that great explanation. I think I'm going to follow your suggestion, and imitate your setup.

[Beetlezombie]

The other option, of course, is to just ignore the auto-asset generation stuff entirely, and hack in images based on raw PNG files that you provide. In some cases, this will be easier. But, it's not always going to be easy -- and if you really want to mod a lot of the game's art using this technique, you may end up being forced to write a collection of relatively low-level utility functions and other special case hooks to help with the process. This is also... probably possible? But how hard it is would depend on exactly what parts of the game you're interested in modding. (Ships would get hairy. Most of the character and race art might be relatively easy though.)

I started that way, but like you said, when you look at the ships that is where it becomes ... complex.

[Beetlezombie]

I've been playing with adding static images, and that is working just fine and dandy. However I decided to take a look at the way you created the art system, all the different layers that ultimately create a ship. And now I'm stuck in limbo, because it is so awesome and I am lost ...

The problem is that my LUA knowledge is lacking severly ... I can find my way, kinda, learning as I go, but could you give me some pointers as to how the engine places the layers on top of each other in layer_info.table.

I'm taking the HawkColony as an example. I get how the layer_info.table and the layers.table handle the snapping and the placing and the opacity etc etc. I get that the higher you go up in the folder structure the .lua files there ( like HawkColony.lua, ShipDrawerConfig.lua etc) handle the bigger background stuff.

I get that if you edit the following you can alter the location on the x and y axis.

Code: Select all

classic_blue_palette = {
    yellow = {
      629,
      149,
    },
    canvas_size = 1500,
    white_markings = {
      442,
      165,
    },
  },

But what is the loadorder for the lack of a better word? Which png is the master that dictates the 0,0 ?

Sorry for the obvious newby questions. In the meantime, I'm off to https://www.amazon.com/exec/obidos/ASIN/8590379868/lua-pilindex-20.

[sven]

but could you give me some pointers as to how the engine places the layers on top of each other in layer_info.table.

The thing that's thrown you off here is that HawkColony is a "paletted" layer asset. That means that while it's in some ways just one asset, it actually includes 2 different recipes for combining layers to create the final output art. One of these is "classic_blue" and the other is "anasazi_purple". If you go into either of those subfolders, you'll find a layers.table file, which has the ordering info you're looking for.

(This complexity comes out of the fact that Arioch sometimes creates different "skins" for some of the ship art, and each skin can require it's own ordering info.)

In simpler, non-paletted assets (like BoxCA), there's no palette specialization, and so layers.table is at the root of BoxCA/layers.

[sven]

Which png is the master that dictates the 0,0 ?

It's been a while, (for the last many years I've only intereacted with this system through my own high-level asset building scripts), but, I believe all the positioning variables assume a canvas_size x canvas_size canvas. (But some other analysis is done during the rendering process so that, insomuch as the real art only covers some subset of that canvas, we don't draw a bigger area than we need to.)

[sven]

In the meantime, I'm off to https://www.amazon.com/exec/obidos/ASIN/8590379868/lua-pilindex-20.

So, as noted on the modding page, SiS uses a modified version of Lua 5.2. Which means the best reference is, perhaps, the 3rd edition of PiL. (That said, the 4th edition of PiL is probably fine too -- just understand that 5.2 Lua doesn't include any 64bit integer math -- it's a pure 64bit floating point-based number system.)

[Beetlezombie]

The thing that's thrown you off here is that HawkColony is a "paletted" layer asset...

Yep the more time I spend in the drawers, the better I understand what's going on. It's a very neat system you built. Curious how you came up with it. It's different, that's for sure. At first glance, I thought the ship assets were more like Distant Worlds or Space Empires 4 (Looking at the prebuilt or Artcache).

But now that I've messed with the drawers, and I see the assets being built in the Artcache as I go along, dang, pretty impressive. Some very nice programming there. But it's definitely going to take a while to get familiar with it.

It's been a while, (for the last many years I've only intereacted with this system through my own high-level asset building scripts), but, I believe all the positioning variables assume a canvas_size x canvas_size canvas. (But some other analysis is done during the rendering process so that, insomuch as the real art only covers some subset of that canvas, we don't draw a bigger area than we need to.)

I found http://stars-in-shadow.com/forum/viewtopic.php?f=5&t=598, and the person in question pretty much had the same issue I'm having. That really helped me out

So, as noted on the modding page, SiS uses a modified version of Lua 5.2. Which means the best reference is, perhaps, the 3rd edition of PiL. (That said, the 4th edition of PiL is probably fine too -- just understand that 5.2 Lua doesn't include any 64bit integer math -- it's a pure 64bit floating point-based number system.)

Thanks for the heads up. I'm going through the free version until I get my copy.

[akkamaddi]

Hi!
So, I'm the person in that other post.
If you have used Photoshop or GIMP, that's the easiest way to imagine how the layers work. It actually gives you very good control over shading. Like I mentioned there, once I figured it out, it was clear that the system was designed for someone who does comic art work.

Another big advantage is that it allows for animated layers. The Ashdar emperor will blink, if you haven't noticed. The first time I was soaking in the cool ship design for one of the new space harpy ships, it wiggled its mouth, which I was not expecting and it knocked me back in my chair.

Everything I did was with single layers, which wasn't that difficult. You just have to make sure the measurements in the lua file match the size of the image.

[Beetlezombie]

Hi!
So, I'm the person in that other post.
If you have used Photoshop or GIMP, that's the easiest way to imagine how the layers work. It actually gives you very good control over shading. Like I mentioned there, once I figured it out, it was clear that the system was designed for someone who does comic art work.

Another big advantage is that it allows for animated layers. The Ashdar emperor will blink, if you haven't noticed. The first time I was soaking in the cool ship design for one of the new space harpy ships, it wiggled its mouth, which I was not expecting and it knocked me back in my chair.

Everything I did was with single layers, which wasn't that difficult. You just have to make sure the measurements in the lua file match the size of the image.

Hi Akkamaddi! Thanks for your input as well.

Yep I'm there now as well. The more you take the system apart the more you realize it is actually a very powerful, smart way of doing things as a programmer.

That is exactly the moment when I got intrigued by it all. Why was the Ashdar character different then the others etc. And then I found the layers upon layers that 'make' the animation.

Yep going single layer is not that hard. A bit tedious work, especially if you going to do the ship art.

Um, yes, it is all probably more awkward and confusing than it needs to be. But if you start really digging deep, you'll discover that there are Photoshop scripting utilities built into the game that you can use to convert PSD files into assets for Lua state\Drawers. So... if you have source art in Photoshop, there are actually tools included inside the game itself that you can use to import that art into the game

I have Photoshop collecting dust atm on my system, so now is a great time to figure all this out. Definitely not enough hours in a day