Experimenting to test forge boon bias

Do 100 forges with 1 gum applied. Then do 100 with 2 and then 3 etc. 100 iterations will not be enough for very exact numbers but will elucidate the rough numbers. I am surprised no one has done this on test up to this point. If no one else has done so I will do it this weekend and post my results. I will do 1-4 gums for special on hammer/axe/shovel

1 Like

Precisely, but if we have every forger do it 100 times, we should get a much larger sample size that way.

1 Like

To get the full picture we’d need data from two gums, preferably one with many boons, and one with less. Since we don’t know if the game rolls from a weighted table of all boons, or first just the category and then the boon from that category. Or do we already know and I just haven’t seen it?

I was so happy when @lucadeltodecso shared the crop growing algorithm, dev insight is always the best. But it’s possible to get a rough idea from a large enough data sample too. I should find time to do some testing myself too…

1 Like

I love this person. Also instantly cleared up settlements breaking when empires was released and gave some very straight answers about footfall and other issues as well.

The forge is a pretty complex system. With the RNG element having impact at two or more stages for each round it seems like they could be a little more open about how the system works without removing the random elements that so many people find frustrating, challenging, or whatever their personal feelings about the RNG and weighting are.

I left this running for about 30hours or so

image

However I noticed the longer it runs, the slower it gets. I am not sure if this is a problem with the program or the way Chrome is handling it.

A couple of months ago I ran a few tests like this on the testing server. I tried testing with a single gum on a boon type that has 2 boons, two gums on a boon type that has 2 boons, and a single gum on a boon type that has 1 boon. My results were consistent with the following:

Each Boon gets weighted by a weight 1 + n x, where n is the number of modifiers and x is a weighting factor. The probability of a boon being chosen is then equal to its own weight divided by the sum of all weights. For instance if you use a special gum, both aoe and magnetic collector receive the extra +x to their weight. My results were inconsistent with gums being added to types as opposed to boons. I also believe that gums only ever add weight and there is no way to subtract weight. E.g. trait reduction gum adds a weight, x, to existing boons. The quantity x is then a measure of the strength of the gum. I measured x to be 18 ± 6 in one trial and 12 ± 4 in another. Considering that a human wrote the code, I believed that x had a value of 10.

Now, I have heard people talk about forging becoming more difficult since the venerable update. I was skeptical for awhile. After all for other tools (besides hammer, axe, shovel) there is a real reduction in probability due to the existence of venerable boons, and I thought this could be what people were observing. But anecdotally, I started to see it. It seemed that x had reduced in value, but I did not record data well enough to make a solid conclusion. I have been using extra gums under the assumption that the strength of the gum was reduced.

For your data i find that x = 9.72 ± 5.4. This is less than my pre-venerable tests but considering the error bars, it is consistent with the pre-venerable test.

Would you be willing do another test or collect more data to get better precision? If you only use a single gum for a boon type which only has a single boon, it will reduce the error in the measurement substantially. (You can measure the strength of the gum better in a regime where there is nearly an equal contribution from the gum and the rest of the system). More measurements could improve things as well, but unfortunately it follows a square-root dependence. In order to double the precision you would have to multiply the number of trials by 4.

Also I hid the math for simplicity, let me know if anyone wants to know how this works. I used standard error analysis for stochastic processes.

3 Likes

Oh wow, yeah the visual representation, the squares, will just keep piling up so your browser will probably crash if you let it run for days. I didn’t optimise it at all, just wanted to see what the rng spread looks like.

You just saved me a lot of time, thank you! I was trying to simulate it with the boon categories instead and couldn’t find weights that work… But if it is by boon as you said it’ll be easier anyway.

Can you explain what do you mean by types?

@Tonezone So if I get what you just wrote…1 + House = You can’t get there from here.

Seriously I understood some of what you were saying but then I got lost. It has been years since I had to do real math. Would it be possible for you to do one example from start to finish (showing the math) so I could see it? If not that is ok I understand. Others will be smarter than me.

I think he meant types as in both aoe and magnetic collector are “special” types, but “aoe” and “magnetic collector” are both boons.

I may be wrong about that, but I’m pretty sure that’s what he meant.

1 Like

Sometimes that’s what I felt when I was trying to learn how to forge some of the trickier things from him lol. I’m in the same boat, that I understood most of what he said, but there were certain ones that I just didn’t get. It’s been far too long since I had to use that kind of math.

Seeing an example from start to finish with math would help me too.

1 Like

So does this mean each time a new boon is added to the forging equation, other boons probabilities shift? So now that there are going to be 2 new boons adding with the farming up date…RNG will most like get worse? Like they said ^^^^^ I followed then got lost…

1 Like

I think this is correct. The more different possibility of boons that get added, the harder and harder it is to get the ONE boon you’re targeting because it now has an increased chance of getting the new boon.

Let’s assumer that a thing only has 4 boon options period. They all would have an equal chance at 25%. If you add two more boons, this now drops down to only about a 16.67% chance of getting the one boon. Even if a gum “boosted” that chance by 5%, it would still be lower chance overall because of it.

The way to balance this is by making only certain items have the chance of certain boons, like if only totems or something could get the water breaking boon.

2 Likes

Probably. I’ve spent a long time examining the raw data files (to generate information on the wiki) and it’s pretty clear from looking at the JSON files that they use a discrete distribution for many of their models, for example drops from defeated creatures. I would not be at all surprised if they do the same for rolling boons, since the code is very easy to write. That doesn’t mean they can’t go in and rebalance the weights, but with everything the devs have to do to get a feature out the door, that might not be done every time.

1 Like

I mean boon categories.

Sorry about that last paragraph (the confusing one?) I was trying to convey how to measure the strength of gums better, and there is a formula, but I thought words would be easier. To measure the strength of the gum to a greater precision (1) use a single lightness gum and record the number of speed boons that appear and/or (2) conduct more trials.

Ok, let’s do some examples.

First off let’s look at a hammer with no gums applied. First I will write out the weights for each boon. (I can’t find an easy way to make a table so I will use abbreviations.)

AOE: 1
Mag: 1
Dam: 1
H-H: 1
SPD: 1
DUR: 1
ENG: 1
LUC: 1
GLO: 1

The sum of the weights is 9, so we divide all of the weights by 9 in order to get probabilities (remember you can multiply the probability by 100 to convert it to a percentage, 0.11->11%):

AOE: 0.11
Mag: 0.11
Dam: 0.11
H-H: 0.11
SPD: 0.11
DUR: 0.11
ENG: 0.11
LUC: 0.11
GLO: 0.11

Not surprising, with no gums each of the 9 boons has a probability of 1 in 9.
Now let’s repeat with a special gum. First the weights (I will use my deduced gum strength, x=10):

AOE: 11
Mag: 11
Dam: 1
H-H: 1
SPD: 1
DUR: 1
ENG: 1
LUC: 1
GLO: 1

The sum of the weights is 29, let’s divide everything by 29 to get the probabilities.

AOE: 0.38
Mag: 0.38
Dam: 0.03
H-H: 0.03
SPD: 0.03
DUR: 0.03
ENG: 0.03
LUC: 0.03
GLO: 0.03

So with a single gum the probability for mag collector and aoe are both 38%. Let’s compare this with what happens if you used a single lightness gum instead. First the weights:

AOE: 1
Mag: 1
Dam: 1
H-H: 1
SPD: 11
DUR: 1
ENG: 1
LUC: 1
GLO: 1

The sum of the weights is 19. Then the probabilities:

AOE: 0.05
Mag: 0.05
Dam: 0.05
H-H: 0.05
SPD: 0.58
DUR: 0.05
ENG: 0.05
LUC: 0.05
GLO: 0.05

So with a single gum SPD has a 58% chance of appearing. Notice how the weight gets added to the boon directly. When we used a special gum 20 points were added to the weight (10 to special and 10 to aoe). Whereas with a lightness gum 10 points were added to the weight (10 to SPD). With a special gum the chance of getting other boons was 3% but with a lightness gum the chance of getting other boons was 5%.

Ok, let’s try 2 special gum since that is OP’s original trial

AOE: 21
Mag: 21
Dam: 1
H-H: 1
SPD: 1
DUR: 1
ENG: 1
LUC: 1
GLO: 1

The sum of the weights is 49. Next the probabilities:

AOE: 0.43
Mag: 0.43
Dam: 0.02
H-H: 0.02
SPD: 0.02
DUR: 0.02
ENG: 0.02
LUC: 0.02
GLO: 0.02

So we have a probability of special boons of 86% and a probability of other boons of 14%. Remember OP got special boons 82 times and other boons 18 times.

Now, the probabilities I gave were for the case where there are no boons set yet. What happens if there are already some boons filled in? Let’s suppose we do the 2 special gums, and the first boon that appears is aoe. Now we apply compounds again, what happens? The probabilities are exactly the same!!! However, the 43% chance for aoe means the boon receives points to level up. That is a different than setting a boon.

So let’s try a different question. What will boon 2 be? Obviously it cannot be aoe since aoe is already on the board. If you hit aoe’s probability of 43% points go into it, but boon 2 is not decided, so you can essentially just ignore it. Here are the weights for boon 2:

Mag: 21
Dam: 1
H-H: 1
SPD: 1
DUR: 1
ENG: 1
LUC: 1
GLO: 1

The sum of the weights is 28. And the probabilities:

Mag: 0.75
Dam: 0.036
H-H: 0.036
SPD: 0.036
DUR: 0.036
ENG: 0.036
LUC: 0.036
GLO: 0.036

It’s almost certainly going to be magnetic collector.

OK, let’s suppose you get real lucky and get the 3 boons set which you wanted, AOE, Damage, and SPD. And you still have the 2 special boons up on the board. Which boons do the boon points go into? It uses the same math. This time we cannot get hard-hitter, mag collector, durability, energy, critically lucky, or glow. Only the 3 set boons can receive points. Still the gum affects which boon receives the points. First the weights:

AOE: 21
Dam: 1
SPD: 1

The sum of the weights is 23. And the probabilities:

AOE: 0.91
Dam: 0.04
SPD: 0.04

Ok, I think I did most of the main calculations. Let me know if you have any other questions. We are very close to the keys to the kingdom, can you find them? And also, let us know if you are willing to do some experiments. I know it is boring, I only did 100 of each trial myself, but I think all here would be interested in knowing more about the gums and that requires testing.

5 Likes

That was so much easier to understand!

Also, if you have a boon set… does it favor going into new boons equally or into the boons you already have?

I have had it where I have 3 special gums active, got aoe, and then the next one leveled up aoe and didn’t put mag collector on it.

Is there a second set of variables that check to see like this:

New boon: 1
Existing boon: 1

That the trait reduction and trait expansion gums affect?

1 Like

@Tonezone That made way more sense to me.

Thank you for the math class today. Now my brain hurts…vodka to the rescue! :sunglasses:

2 Likes

I believe there is not. I believe the way it works is that if you have a trait reduction gum on it adds an extra weight +x to the applicable boons. Likewise if you have a trait expansion gum I believe it adds an extra weight +x to the boons that have not yet been picked.

I have not done experiment’s like the OPs for trait expansion or trait reduction. I merely watched what I was getting when using them and it seemed to follow that calculation. The way to test if my calculation of the probabilities for trait reduction/expansion is incorrect is to perform an experiment and then calculate the value of x along with it’s error. If the value you get is not consistent with the other measurements, then it means there could be another way the probability gets calculated such as the 2-stage method you proposed.

I suspect this may not turn out the way you think it’s going to…

1 Like

Are you telling me that it won’t… add up to a good time?