Simulations of One-On-One Battles: Original Starfire Book 1

Home of the 1st and 2nd editions of STARFIRE

Moderators: SDS Members, SDS Owner

Forum rules
Forum rules
1. Nothing obscene.
2. No advertising or spamming.
3. No personal information. Mostly aimed at the posting of OTHER people's information.
4. No flame wars. We encourage debate, but it becomes a flame when insults fly and tempers flare.

Try to stick with the forum's topic. Threads that belong to another forum will be moved to that forum.

Simulations of One-On-One Battles: Original Starfire Book 1

Postby Graywulffe on Fri 10 Jun 2016 15:59

This will be a long post. If you are strongly interested in the mechanics of the Starfire combat system, then you may find this text worth the time.

I have been developing a Python program that pits two spaceships against each other using the original Starfire Book 1 rules. Currently, I have implemented enough weapons systems, and point defense, to allow for many different aspects of the combat system to be tested. Below are the relevant details so that you can interpret the preliminary results from the 20-odd trials that I have done so far. It is important to keep in mind the limitations of the testing method when interpreting the results.

The main conditions are:

1) This is a one-dimensional simulation, with the ships moving back and forth on a number line. The purpose of having movement is to create a situation with varying ranges. This provides a more dynamic test of weapon systems.
2) Both players decide to do a full-frontal assault.
3) Both ships use all of their movement points to cover the maximum distance (e.g. six intact engines = move six spaces).
4) Once the ships pass each other (if they survive to this point) and reach a specified distance (12 hexes from zero), they turn around and resume the fight.
5) It takes one turn to spin a ship 180 degrees.
6) All available weapons will be fired each turn, if the opponent is in range.
7) For each ship, weapons are fired in this sequence:
1) all G; 2) all R; 3) all W; 4) all L; and 5) all F.
8) Ships can fire directly behind (as with the original rules).
9) If all weapons are destroyed on a vessel, the ship will run.
10) The other ship will chase the running vessel.
11) Game over when:
a) a ship becomes a floating hulk (no weapons or engines) or;
b) if one ship leaves the board (48 hexes) after losing all its weapons or;
c) if one ship is destroyed.
12) Gun/Missile launcher considerations:
The cutoff for gun verses missile is a balance between the probability that a gun will hit at a given range, verses a missile hitting and then getting past point defense. At range six, a gun has a 41.7% chance of hitting and causing damage, and at range seven, a 27.7% chance. A missile has an 83.3% chance of hitting at range six or seven, but when combined with point defense only a 13.8% chance of success on a single-shot volley, a 27.7% chance for the second missile on a double volley and a 41.7% chance for the third missile on a triple volley. Guns are set to fire at range <7 and missiles >6. This, of course, assumes that the enemy ship has point defense.

Available technology for V1.1 of this simulator:

Technology Level I:
- Gun (G).
- Missile (R).
- Laser (L).
Technology Level II:
- Gun-missile launcher (W).
Technology Level III:
- Point defense (D).
Technology Level IV:
- Force Beam (F)

During each simulation 100,000 games are played. This takes the computer 5-30 minutes to complete, depending on many factors but most especially on how much damage can be delivered in a given turn (higher damage = shorter runs). These simulations could be done faster, but I have much data being printed out during the simulation so I can check back on interesting battles to see what happened. The total number of turns during a simulation has ranged from 0.3 to 4.7 million.

Selected Key Results:

1. Control Battles

I started off with some controls. I pitted two ships of exactly the same design against each other. This was repeated for different designs using different weapons and point defense configurations. The expected outcome of these battles is Ship1 50% and Ship2 50% wins. These simulations are run to make sure the program is working well. All battles between the same ship designs have resulted in a range of 49.8% to 50.2% wins for each vessel (e.g. Ship1 49.8% and Ship2 50.2% or Ship1 50.1% and Ship2 49.9% and so on). This outcome is expected, allowing for some noise due to the random nature of the battle interactions. Amazingly, one short 1,000 trial simulation returned exactly 50/50. The key information here is simply that if you have an encounter between two ships of exactly same design, all you need to do is roll one dice to determine the winner. Boom!

2. Moving System Modules Around on the Same Ship

Looking specifically at two ship designs:

Resolute - ES1 (2) - SSAIIWDII (4) MC 108 / 97
Reliant - FG1 (2) - SSSAAHIIWWDII (4) MC 178 / 160

The names are the ship class, as in "Resolute Class Escort". Escort, or ES, is the hull type. As I am using it, the class name stays the same even when system modules are moved around. If a different system is added (e.g. an R for a W), then the class name is changed. The main reason for using this system is to make it clear when system modules are just being moved around verses new systems being added.

When using carefully considered system module location changes in the Resolute Class ES and then pitting it against the much stronger Reliant Class FG, a pattern begins to emerge. Starting with the control runs:

Ship1 vs. Ship2:
SSAIIWDII vs. SSAIIWDII: Ship1, 50.02% & Ship2, 49.98%
SSSAAHIIWWDII vs. SSSAAHIIWWDII: Ship1, 50.01% & Ship2, 49.99%

Now pitting the FG against the ES:

SSSAAHIIWWDII vs. SSAIIWDII: Ship1, 100.00% & Ship2, 0.00%

The Resolute had very little chance! In this case, out of 100,000 games, the Resolute actually won 2 times, and the Reliant 99,998 times. The rounding makes it seem like 100% / 0%. Thus, the Resolute appears to have approximately a 1/50,000 chance of winning each battle. Now, make a slight configuration change to the Resolute: move the WD to the right by one place:

SSSAAHIIWWDII vs. SSAIIWDII: Ship1, 99.9% & Ship2, 0.01%

This time, the Resolute won 12 times, suggesting a probability of success of 1/8,333. The Reliant is still the safer ship to be in, but the odds of the Resolute winning have been markedly improved by putting one more engine in front of the WD cluster. Now, make another small change, moving the last engine forward and reversing the WD to DW because you may as well be getting shots off until the last moment:

SSSAAHIIWWDII vs. SSAIIIIDW: Ship1, 99.88% & Ship2, 0.12%

The Resolute won 120 times, indicating an improvement in the probability of success to around 1/833. Though the FG is still by a large margin the strongest ship, small system module location changes in the ES have resulted in orders-of-magnitude improvements in its success rate. Now we go even more extreme, and bring in the new Stinger Class of ES (MC 112/101), what I would consider an exotic or non-traditional (strongly outside of the deigns in the Original Starfire scenarios) ship design:

SSSAAHIIWWDII vs. SIIWDW: Ship1, 98.21% & Ship2, 1.79%

With 1,793 wins, the Stinger's chances of success are around 1/56, yet another order-of-magnitude leap in capability. Now, in 100 battles, the little ES can be expected to win roughly 1 or 2 times. Note that the Stinger has very low movement. In a real 2D game between human opponents, the gains over other ship designs may not be as high as in the full frontal assaults being simulated here.

To further drive home the message, when pitting two Reliant Class FGs against each other (the control run is shown above):

SSSAAHIIWWDII vs. SSSAAHIIIIWDW: Ship1, 19.68% & Ship2, 80.32%

With the same system modules, simply putting the weapons at the right-most end of the list tipped the odds in favor of Ship2 from 1:1 to 5:1. And just for fun, bring in a new class of FG, called the Viper, like the Stinger, an exotic ship design:

SSSAAHIIWWDII vs. SAIIWWWDW: Ship1, 9.80% & Ship2, 90.20%

The odds in favor of the Viper are around 10:1. The increased number of weapons is likely the main contributor to the higher frequency of wins, though weapon placement is also likely adding some advantage. Like with the Stinger, the Viper has slow movement, and this could become a detriment in a real game. However, I note that with four missile launchers, I could see the ship plodding along at the edges of a battle pestering ships from afar, and then being able to deliver a biting barrage of gunfire if anyone dared move in to close range. One note about ships with all their engines ahead of the weapons: the frequency of the opponent escaping after all of its weapons are destroyed goes up noticeably.

The main message here is that the placement of systems modules can be very important to outcomes. Also, it appears that putting as many systems (e.g. S, A, H, I) in front of your key (all) weapons is advantageous. The advantage looks to be strong enough that I suspect it is the main reason primary beams (P) were implemented in the game. Given that a P can strike deeply (and randomly) into a ship's systems, the advantage shown here is reduced.

Currently, I do not have P implemented. The primary beam rules are more challenging to code than other systems (coding D and L was tough enough!). I will attempt to create a P simulator before trying to code it into the game, and this will take some time.

3. The Big Question: Gun Verses Missile

For those who encounter someone who brought missiles to a gun fight, who is likely to be the winner?

For this analysis, we have two new spaceships:

Reach - ES1 (2) - SSAAIIIIRII (6) MC 85 / 77
Gunner - ES1 (2) - SSAAIIIIGII (6) MC 80 / 72

The control runs for both of these ships were around 50/50, so all is good.

SSAAIIIIRII vs. SSAAIIIIGII: Ship1, 99.30% & Ship2, 0.70%

Missiles win hands-down! And this is in a simulation where the ships are charging each other, giving guns an opportunity to fire at 0 range. Missiles have no chance of hitting at 0 range. Likely, if I slowed the movement to 4, which gives guns a chance to fire one time before range 0, the outcome would be a bit different. I plan to do this simulation soon.

Here is what happens if you bring in a new class of gunship that has a point defense system, the Defiant (MC 93/84):

SSAAIIIIRII vs. SSAIIGDII: Ship1, 3.87% & Ship2, 96.13%

The situation is literally reversed! Point defense is effective. Also, the Defiant Class ES only moves at a 4, allowing for an extra shot or two during a pass. However, zero range is also prevented due to the movement differential, giving some favor to missiles.

Applying what we learned about putting weapons at the end of the system module list, I also did a simulation with the Defiant configured like so:

SSAAIIIIRII vs. SSAIIIIDG: Ship1, 0.74% & Ship2, 99.26%

The results were even more decisive against missiles. Point defense systems are very useful when you are faced with missile attacks.

That is it for now. I am continuing with new simulations, and plan to report back when I find more interesting outcomes. To be brought into the mix with my own creations, I plan to use some of the ship designs from the Original Starfire Book 1 scenarios—there are several from the early scenarios that fit right in with the current technology implementations. If you have any one-on-one battles you want to test, feel free to post some suggestions.

Thanks for reading.

-best
Graywulffe
Lieutenant JG
Lieutenant JG
 
Posts: 23
Joined: Sat 28 May 2016 08:35
Location: Vancouver, BC

Re: Simulations of One-On-One Battles: Original Starfire Book 1

Postby Cralis on Sat 11 Jun 2016 00:23

It's absolutely no surprise that the same design against itself is a statistical 50%-50% split. With that many combats, all outlier events will be lost in the noise.

And while the old timers know that system placement makes a huge difference, it's interesting to see just how much of a difference it makes.

But I find it more fascinating the counter-systems, like D in this example, are so effective. I knew it was a necessity, but I didn't realize it would be that powerful. It's the biggest reason why it (and other active defenses) were changed in Galactic Starfire and later versions, as empires that never developed them, or fell behind, were mathematically toast.

(Yes, I know that we aren't testing overwhelming the defense, but that wouldn't be possible without a fleet simulator)

Are you eventually planning on extending this to later versions of the game? It would be a handy tool for basic testing of the relative power of ship weapons and defenses.
Image
User avatar
Cralis
SDS Member
SDS Member
 
Posts: 10197
Joined: Tue 30 Jun 2009 19:27
Location: Oregon, USA

Re: Simulations of One-On-One Battles: Original Starfire Book 1

Postby Graywulffe on Sat 11 Jun 2016 15:13

Hi Cralis,

With regards to the 50/50 on same-ship face-offs, exactly! The main reason why I do those tests is to check the programming. Certain bugs might show up as deviations from the expected 50/50 outcome.

Yes, the system module placement appears critical, and there is another big consideration that I just deciphered from the simulations that I will write about later (gotta run right now). With regard to the point defense, what happens to ships without D is that they get picked to pieces by missiles being fired at long range. Missiles only do 1 point of damage, but it adds up rather quickly during battle, especially for smaller ships. One point on a CT with 10 system modules = 10% damaged.

Right now my plans are to incorporate more Original Starfire rules into the system I have. I have not played, nor in fact have I had a chance to look at the latest version of Starfire. I am planning on buying the full package at some point soon, though. When I have a chance to pour over the rules, I may consider writing a new program. It will depend on how much time I can apply to the project. Believe me, the Book I rules to the point that I have taken them took much effort and time. Right now I can justify the time as I am using it to learn Python. On the plus side, I have thought of more efficient ways to make this combat simulation program and perhaps tackling the latest rules will be an opportunity to try out a different approach--more practice. But it would be awhile before I have anything running.

-best
Graywulffe
Lieutenant JG
Lieutenant JG
 
Posts: 23
Joined: Sat 28 May 2016 08:35
Location: Vancouver, BC

Re: Simulations of One-On-One Battles: Original Starfire Book 1

Postby Graywulffe on Mon 13 Jun 2016 08:15

Part II of Combat Simulations: Large Hull Types Vs. Small

As I continued testing my simulation program, I designed this light cruiser (CL) to throw into the meat grinder:

Javelin – CL1 – (3) SSSSSAAAAHIIWWWDWWDII (4) MC 396/356

Before continuing, it is important to note that for many of my tests I tend to have just one weapon type on the ship. This is another form of test control. It helps with interpretation of the outcomes. I plan to do more mixed weapon examinations later.

In some ways, the Javelin Class CL is a monster. Five W systems can deliver quite a punch from both long distance and at close range.

I tested this frigate (introduced in my previous post) against the Javelin:

Reliant – FG1 – (2) SSSAAHIIWWDII (4) MC 178/160

As expected, after 100,000 games, the Javelin completely outclassed the Reliant, with 100.00% of wins. The Reliant succeeded not once (~< 1/100,000 chance of victory). I started keeping track of total system modules lost in each battle to compute an average % of damage from the entire simulation (more recently, I have also started keeping track of internal systems damage). On average, the Reliant damaged the Javelin 11.8%, or 2.5 system modules, meaning that the majority if not all battles resulted in no internal systems being damaged, just two-to-three S. The Reliant, of course, was always reduced to a rapidly expanding debris cloud.

Moving up to the next hull type thinking that some extra system modules including another W and D would make a significant contribution:

Guardian – DD1 – (2) SSSSAAAHIIWWDWDII (4) MC 291/262

The results were not much better than the reliant. After 100,000 games, the Javelin completely outclassed the Guardian, with 99.99% of wins. The Guardian succeeded only 15 times (~1/6,667 chance of winning). On average, the Guardian damaged the Javelin 30.7%, or 6.4 system modules, meaning that many battles resulted in no internal systems being damaged, just S and a few A. The Guardian, of course, was typically destroyed.

I thought the DD would do better. Alas, no. Some other simulations I did later on make this outcome a bit more interesting, but that is for another post. I started wondering how a flotilla of two Reliant Class FGs would do against the Javelin, and also two Guardians. Comments by Cralis on my earlier post also stimulated thinking about testing larger assemblages of ships. I began brainstorming how fleet encounters could be simulated without additional programming.

It quickly occurred to me that at least a small fleet could be represented attacking a single ship in the one-on-one simulator that I have been working on. For a two-ship mini-fleet, the ship systems could be entered in tandem as a single ship into the program. This would simulate the opposing ship concentrating fire on one ship until it has been destroyed, then moving on to the other. To simulate fire spread equally between the two ships, the system modules could be arranged in the manner of a single large ship, but spreading fire does not necessarily make sense in a two-on-one situation. The opponent would want to take out weapons systems as quickly as possible and this means concentrating fire. One could argue for a switch to the second ship after the last weapons module was destroyed on the first, thus ignoring the last few systems on the first ship. This does reflect a small weakness to my flotilla simulation approach.

Other problems with the small fleet kludge approach included dealing with the engines of the two vessels. I had to set up dummy engines, using a "Q", so that movement would not be double what it should be. I spread the Qs equally between the ships—this works ok because speed in this simulator is not particularly important (though I wanted to keep it a 6 maximum anyway). I also had to represent the second ship's shields with "V" due to the nature of the programming. This means that the two-ship kludge cannot work with laser attacks. I changed the first point defense to an "N" to prevent these two ships from having the benefit of two Ds when the opponent fired its missiles (this approach is not perfect, but is close depending on where the D is located on the ship). Since the two attacking ships would fire in a single volley, I gave them each a data-link (Z) to fit the game rules.

At some point, I will try to make a more comprehensive fleet simulator, but it will be awhile before I get there.

This is what the system modules for the two FGs looked like when I entered them into the program:

SSSZAAHQIWWNQIVVVZAAHQIWWDQI

I threw this at the Javelin… And the Javelin won. With 96.22% of the wins, the battle was not even close. The two FGs had some success, with 3,780 wins (1/26) making them more effective than a single Guardian Class DD. On average, the FGs also caused 48.51% damage to the CL, or 10.2 systems. This means that in the average battle, the frigates punched through the S and A and damaged internal systems. However, of importance is the cost of two Reliant-Class FGs relative to the cost of one Javelin-Class CL: 338 vs. 396 MC (note that I am using the lower mass-production cost for the second FG). This is without the data-links. With Z, the cost is 415 vs. 396 MC. The expense of building two Reliant-Class FGs is comparable to one Javelin-Class CL, and the latter appears to be far more effective in a full-on combat situation. Now, this does not test things like turn modes and applying different tactics, but it is nevertheless informative.

What this suggests is that it may be better to put your funds into a single large ship than in a fleet of smaller vessels (for campaigns where both opponents have the same or similar monetary resources). Part of the reason for this is that the weapon systems of a single big ship can be buried behind a large array of systems, whereas for the smaller vessels there is not as much shielding. Thus, the larger ship can begin picking off weapon systems sooner than the smaller vessels can.

I then tested two Guardian class DDs against the Javelin:

SSSSZAAAHQIWWNWQIVVVVZAAAHQIWWDWDQI

(Note I forgot to enter the second "N" in the first ship when I ran this simulation.)

Two DDs produce a completely different outcome, besting the Javelin 99.78% of the time. The CL only managed 222 wins (~1/450) and only caused 36.26% damage to the DDs on average, or the equivalent of 12.7 system modules. The first DD survived, with heavy damage, the average battle and the second remained unscathed. The cost of the two DDs is 553 MC without data-links and 629 MC with. This is 1.4 to 1.6 times the cost of one CL. So one would hope the DDs would be victorious against a single CL. The difference in cost is enough to build a CT or FG that could accompany the CL and provide enough firepower (two more W) to swing the game the other way.

I ran a 100,000 game simulation using a CT with the configuration SSAIIWWDII added in front of the CL and the combination had an advantage of 78.4%/21.6% over the two DDs. The DDs caused 71.2% damage to the enemy on average, or just barely getting into the internal systems of the CL. Given that the CT and CL are different hull types, I also ran a 10,000 game simulation with the CL in front of the CT, depicting a situation where the DDs target the larger ship first, and the CL+CT combo still had a 69.3%/30.6% advantage (these numbers hint at the idea that it may be better to target the bigger ship first, which makes sense for the bigger hull types tends to hold the majority of weapons—I plan to test this more later). The average percent of damage that the two DDs caused to the CL+CT was 67.7%, or ~ 22 system modules, slightly lower than the other arrangement.

The outcomes of these simulations support this basic conclusion: It appears better to concentrate funds in larger hull types than spread out money in a fleet of smaller vessels. At least this seems true of the Original Starfire system.

-best,
Graywulffe
Lieutenant JG
Lieutenant JG
 
Posts: 23
Joined: Sat 28 May 2016 08:35
Location: Vancouver, BC

Re: Simulations of One-On-One Battles: Original Starfire Book 1

Postby Cralis on Mon 13 Jun 2016 10:04

This type of testing is very basic numbers. You're missing the one part of combat that makes all the difference: tactics. Without the ability to move into the blindspot, the program can't replicate the advantage of multille ships (especially vs one enemy).

But unless you make an actual battle simulator with AI... but it's hard and that's why it hasn't been done yet. So don't sweat it. Having a numerical analysis will help us avoid pure imbalances!

One thing I do find fascinating is that your results are almost always 99+% one way...

Also, you might experiment with percentages. The "standard" is 20% passive defenses, 10% active defenses, and the remaining available (non I, Q, etc.) space as weapons. What if you go passive heavy? Passive light? etc. Though I think this will make more of a difference with BB and larger ships.
Image
User avatar
Cralis
SDS Member
SDS Member
 
Posts: 10197
Joined: Tue 30 Jun 2009 19:27
Location: Oregon, USA

Re: Simulations of One-On-One Battles: Original Starfire Book 1

Postby Graywulffe on Mon 13 Jun 2016 10:16

Cralis wrote:This type of testing is very basic numbers. You're missing the one part of combat that makes all the difference: tactics. Without the ability to move into the blindspot, the program can't replicate the advantage of multille ships (especially vs one enemy).


Indeed. I am very good at losing Starfire. I need to expand my skills in tactics. ;^) It is always good to keep in mind the limitations of any simulation when interpreting the results.

Cralis wrote:But unless you make an actual battle simulator with AI... but it's hard and that's why it hasn't been done yet. So don't sweat it. Having a numerical analysis will help us avoid pure imbalances!


I can see making one with simple AI rules, but would have to go into deep study to allow for more complicated decisions. I am not about to tackle that yet.

Cralis wrote:One thing I do find fascinating is that your results are almost always 99+% one way...


Yes, I have noticed this, too. Small changes are tipping the balance rather dramatically. It is difficult to find situations where there are splits below 10:1, though they do occur. And this leads into testing some of the ship designs in the Book 1 scenarios... I am finding evidence that they were well considered in terms of balance.

Cralis wrote:Also, you might experiment with percentages. The "standard" is 20% passive defenses, 10% active defenses, and the remaining available (non I, Q, etc.) space as weapons. What if you go passive heavy? Passive light? etc. Though I think this will make more of a difference with BB and larger ships.


This is a very good idea and I will certainly do some trials with these considerations. Thank you much for this input!

-best
Graywulffe
Lieutenant JG
Lieutenant JG
 
Posts: 23
Joined: Sat 28 May 2016 08:35
Location: Vancouver, BC

Re: Simulations of One-On-One Battles: Original Starfire Book 1

Postby Graywulffe on Wed 15 Jun 2016 20:08

Part III of Combat Simulations: Using Proportional Allocation of System Modules Based On Their Function to Test Ship Designs

1. Allocating System Modules

As Cralis suggested a few days ago, one method for choosing system modules in a consistent manner between and within ship types involves proportional allocation based on function (e.g. Defensive, Offensive). System module functionality can be broken down in different ways. Here I will largely use the categories that Cralis listed: 1) passive defense including S and A; 2) active defense including D; 3) offense including G, R, W, L and F; and 4) my own contribution that I term incidentals such as H and X, items that may be necessary for campaigns and specific missions. A standard allocation may be something like 20% passive defense, 10% active defense, 60% offense and 10% incidentals, or 20/10/60/10 for 100%.

I apply these allocations to the remaining hull spaces after engines (I) are installed. For a cruiser (CA), if four engines are installed, this leaves 42. With this number, 5% is equivalent to approximately 2 hull spaces. There is enough "resolution", so to speak, in a CA to test different allocation ratios. Superdreadnaughts (SD), by the way, allow one to get even finer as 1% = ~1 hull space. I plan to explore SDs soon.

2. First Simulation: Standard Vs. Offense-Heavy

Starting with a "standard allocation", 20/10/60/10, I developed this cruiser:

Stonewall - CA1 (3) - SSSSAAAAHXWHIWWIWWDWWWWDII (4) MC 598/538

Again, I stick with a single weapon type on these platforms to simplify interpretation of outcomes. Incidentally, in the Original Starfire Book 1, ships loaded with W are a recommended addition to fleets.

The Stonewall is then pitted against a CA with an offense-heavy allocation of 10/5/80/5:

Antietam - CA1 (3) - SSAAHXWHIWWWWIWWWWWWDII (4) MC 601 / 541

To create this allocation, the defense was simply cut in half relative to the Stonewall. With 11 "W" systems, the Antietam looks like a monster!

After running these ships through 100,000 full-frontal charges, the Stonewall, with a more even ("even" being relative) allocation between the different system function types, came out on top about 3:1, or 74.9% to 25.1% (74.9%/25.1%). A result that is not 99% in favor of one ship! Another way of comparing outcomes is with the amount of damage each ship takes in the average battle: Stonewall lost 59.5% of all system modules and 43.3% internal systems on average with the Antietam losing 89.0% and 86.6% respectively.

When considering the outcomes from the one-on-on simulator, it is important to think about what happens in a single battle. The ships charge towards each other with a speed of four. Both ships end the movement phase of turn one at 12 hexes from zero, so they have a distance of 24. No shots are fired. On turn two, they close to eight hexes from zero. With a distance of 16, both ships fire missiles. Point defense takes care of most hits, with perhaps a few strikes reaching each ship. With nine W systems, the Stonewall's fire can overwhelm the Antientam's point defense, whereas the 11 W systems on the latter cannot quite overwhelm the former's D. The Antietam is more likely to take damage on turn two. Turn three is a similar situation, with the ships at distance of eight. The hits accrued during these two turns are important as they tended to just reach the internal systems—exposing each ship to serious damage. The strikes rarely reached an engine, so speed tended to remain constant. These two outcomes make turn four very important. Both ships move to zero range, and blast away with guns that have a very high probability of hitting their target. With the potential to destroy nine or 11 systems depending on the vessel, the outcome of this exchange could determine the outcome of the battle. Both ships have enough firepower to eliminate some of the opponent's weapon systems on this turn, which means the one with initiative has a chance of reducing the potential amount of damage that they will take on this critical round.

We know that the turn four initiative roll does not determine all outcomes, as this would mean 50%/50% when what happened was approximately 75%/25%. However, in the case of the two CAs being tested, turn four may have determined a large proportion of outcomes—but who knows how many? After turn four, the ships zip past each other and the situation may return to a distance battle with missiles. Turn four damage may include engines, meaning that distance on turn five is probably less than eight. So there is some chance of another round of guns. Turn six will be missiles. Keep this sequence in mind with all the simulations presented in this post, as all ships used have an initial movement of four.

3. Second Simulation: Standard Vs. Defense-Heavy

The opposite of weapons-heavy is defense-heavy. For this case, the defense allocations are doubled, resulting in 40/20/30/10 and this ship:

Monitor - CA4 (3) - SSSSSSSSAAAAAAAAHXIDHWIDWWDWWDII (4) MC 592 / 533

Note that with all three of these ships, effort has been made to arrange internal systems in a similar manner to help with the interpretation of results.

The outcome of 100,000 games between the Stonewall and Monitor resulted in the former being favored about 4:1, or 81.0%/19.0%. During the average game, the Stonewall suffered about 63.0% damage with 46.6% to the internal systems, and the Monitor 94.6% and 89.2% respectively. Interestingly, the defense-heavy Monitor caused slightly more damage to the Stonewall than the offense-heavy Antietam, by about one system module on average.

The Stonewall's heavier firepower appears to win over the heavy-duty defense of the Monitor. This is probably in part due to the use of guns at the critical turn-four and also during later close-encounters as the ships usually get through the fourth turn in good fighting condition.

To test the above idea, I changed the gun-missile launchers (W) to missile launchers (R), with considerable savings in MC for each ship, and pitted the CAs against each other in a 10,000 game simulation. Using R systems eliminates the gun impact on turn four and puts the focus on the distance battle. In this case the Stonewall still wins on average, but at the reduced rate of about 2:1, or 67.1%/32.9%. It appears that the guns definitely contributed during the simulation using the W systems, but they were not entirely the deciding factor.

In considering the outcome with straight R systems, though the Monitor has four point defense systems compared to the Stonewall's two, the D systems of both ships were nearly equally effective, 75% for former and 74% for the latter. This is because of the reverse difference in number of weapons, with the Stonewall having the higher number. Simply by having more shots per turn, the Stonewall had more opportunities to cause damage, 9/5 = 1.8:1, perhaps the main reason for the vessel having the upper hand.

4. Third Simulation: Defense-Heavy Vs. Offense-Heavy

Now to the final test of these cruisers: The Monitor vs. Antietam, or defense-heavy vs. offense heavy. Who will be the victor?

In this case, stronger defense won over more guns and missiles. The Monitor bested the Antietam by about 2:1, or 64.0%/36.0%. During the average battle, the Monitor was damaged 76.9% with 54.4% of internal systems lost and the Antietam suffered 84.5% and 81.2% respectively. These outcomes are quite interesting given the large difference in number of weapons, five in the former and 11 in the latter, and also the fact that the Stonewall bested the Monitor 4:1. The Monitor's success against the Antietam may simply be in the vessel's much larger number of system modules. The Monitor's first weapon is 22 systems down the line, while slot 22 on the Antietam is the second to the last engine! Likely, point defense also played a role, with the 4 Ds on the Monitor having an average success rate of 74% while the single D on the Antietam a relatively low 58%.

5. Summary and Conclusions

Only ships with gun-missile launchers were tested (save for the smaller trial using missile launchers only), thus the following conclusions are likely best applied to such ships. Different weapons systems, and ships with multiple weapon types, may produce different outcomes than the ones described here. Nevertheless, ships loaded with W systems probably point to some general trends, with the specifics such as system module allocation percentages perhaps different for other weapon types.

In the standard vs. offense-heavy test, the standard allocation of the Stonewall had a higher number of successes than the weapons-loaded Antietam. The Stonewall's standard allocation also proved successful against the defense-heavy Monitor. The Monitor survived more battles when pitted against the Antietam.

Given the Stonewall's success, the outcomes of the experiments support the idea that a more conservative allocation of defense and offense in a ship leads to a higher success rate. Going extreme, either offense-heavy or defense-heavy, tends to result in a lower chance of success against a more standard allocation of ship systems. This does not mean that extreme designs do not have a place. They may be useful in specific situations. However, ships for general duty—"ship-of-the-line"—are probably better off with something akin to a standard allocation.

The outcome of the third trial, that of a strong defense being more victorious over strong offense, suggests that extra defense be given a little more priority than extra offense. It appears that the main reason for this is simply that defensive systems only take 1 hull space and therefore provide much more padding than weapons that take 3 hull spaces each.

Given that the offense-heavy ship faired a little better against the standard allocation than the defense-heavy ship, the 20/10/60/10 used here for the "standard" may not be exactly in the middle. There may be a slightly better distribution. I plan to use superdreadnoughts to fine-tune the splits to better identify what might be termed the "golden allocation". This will take some time, as the larger the ship, the longer the simulation takes: about one hour for two SDs head-to-head.

-best
Graywulffe
Lieutenant JG
Lieutenant JG
 
Posts: 23
Joined: Sat 28 May 2016 08:35
Location: Vancouver, BC

Re: Simulations of One-On-One Battles: Original Starfire Book 1

Postby thebard on Thu 16 Jun 2016 02:18

Interesting, but seeing as you (apparently) have lots of time to spare . . let me ask a question.


Given that S and A are much cheaper than weapons systems (even allowing for 3*S for a W), have you any thoughts
about success as a function of cost?
thebard
Commander
Commander
 
Posts: 139
Joined: Mon 08 Sep 2014 08:47

Re: Simulations of One-On-One Battles: Original Starfire Book 1

Postby Whitecold on Thu 16 Jun 2016 16:20

Just a question out of curiosity, how does W work? I take these are the box launchers, but they have been cut from the Solar version of the game.
Whitecold
Rear Admiral
Rear Admiral
 
Posts: 446
Joined: Fri 19 Sep 2014 15:03

Re: Simulations of One-On-One Battles: Original Starfire Book 1

Postby Graywulffe on Thu 16 Jun 2016 16:39

thebard wrote:Interesting, but seeing as you (apparently) have lots of time to spare . . let me ask a question.

Given that S and A are much cheaper than weapons systems (even allowing for 3*S for a W), have you any thoughts about success as a function of cost?


Time? Not as much as you think. I run the simulations when I know the computer is going to be idle, and do the write-ups piecemeal when I can find a few minutes here an there. The programming and combat trials that I am currently doing are a unique moment in time for me. When the tides of life shift, my effort to explore the Starfire combat system may get put on the backburner, at least for a while.

Your question is interesting and well worth exploring as it addresses game balance. Since I am still pursuing avenues of the Starfire combat system, I cannot study the cost considerations in-depth right now. That said, I have been paying attention to the cost of different ships as I have been blowing them up, and your question brought up a few thoughts/ideas. What follows below is largely stream-of-consciousness as opposed to a cohesive narrative.

As a first approximation, the more megacredits (MC) you put in a ship, the more likely it will succeed against other vessels. A 1500 MC superdreadnought is virtually guaranteed to succeed against a 150 MC frigate. Given that all beam weapons require the same number of hull spaces (4), and the more destructive weapons tend to cost the most, higher funding for weapon systems is likely to bear fruit. Of course, lots of relatively cheap armor can offset the damage taken by heavy weapons--but these defenses only last so long under a hail of attacks, and some weapons zap right through them.

Because S and A cost less both in MC and hull spaces, they are not as problematic to add to a given spaceship. Thus, what happens is that a vessel with fewer weapons tends to end up with more defensive system modules and much of the potential cost differential is erased. This is in part because there is a tendency to use up all available hull space. One S costs 4 MC, meaning that three hull spaces of S costs 12 MC. This is less than the 20 MC that a gun-missile launcher (W), which requires three hull spaces, costs. Armor is even less expensive than shields. In combination you can get something like 3xS and 3xA for 18 MC, which is less expensive than six hull spaces of missile launchers (R). Of course, the S and A give you six points of padding and the R systems only two, an important trade-off. That extra padding, however, is only really useful if you can back it up with some weapons, making ship design the tension between defensive and offensive capability that makes the game so interesting (and sometime frustrating).

The costs of the three cruisers I presented in my Part III post above reflect some of the different trade-offs, mainly in terms of hull space considerations because I am not designing with a limited budget:

Weapons-Heavy: 601 MC (2S 2A 11W)
Intermediate: 598 MC (4S 4A 9W)
Defense-Heavy: 592 MC (8S 8A 5W)

Interestingly, the most expensive CA did not do so well in the simulations. But I would say the prices of the ships above are actually quite similar. The difference in cost between the most and least expensive CA is just 1.5%, not double, or an order-of-magnitude, so probably not significant.

So far, the most interesting cost-related outcome of the simulations is that two smaller ships with a near equal cost to a single larger vessel tend not to fare very well against said larger vessel. Thus it seems better to concentrate funds into a single bigger ship than spread out funds into a fleet of smaller vessels. However, as Cralis pointed out, the full-frontal charge simulator that I am using does not capture all the advantages one can have when using multiple ships against a single vessel, so the outcome is unclear. This is where I may do some play testing with an actual board game. I will have an opportunity to play Starfire in a few weeks (finally, no computer screen, just an old well-worn rule book, counters, a board and some dice!), and I intend to test some of the conclusions I have drawn from the computer simulations.

It would be interesting to test one's ship design mettle in terms of cost. Start this examination with a 1000 MC ship that has a more-or-less standard design. Then test lower-cost ships against this vessel, say carefully considered designs with a 950 MC limit for starters. See if any of the lower-cost ships can at least succeed in 51% of the trials. Also use a standard design at the 950 MC cost limit as a control. Then lower the cost limit to 900 MC and again try for the same 51% wins, and so on. This could pinpoint at what ratio cost differences become significant. As noted above, I do not think a few percent is that significant of a difference in terms of outcome, but I suspect that even 10-20% probably is.

At some point, I may look into testing cost more in-depth.

-best
Graywulffe
Lieutenant JG
Lieutenant JG
 
Posts: 23
Joined: Sat 28 May 2016 08:35
Location: Vancouver, BC

Next

Return to Original Starfire

Who is online

Users browsing this forum: No registered users and 2 guests