Benchmarks

Summary: TO2 is not as fast as it could be.

But lets see the results first. The Benchmark is comparing a solver for Lambert’s problem written in to2 and C#. The two implementations can be found here:

Results for the current implementation

Method

testSet

Mean

Error

StdDev

LamberCSharp

new (…)) }, [351]

172.5 ns

1.54 ns

1.36 ns

LamberTO2

new (…)) }, [351]

308.1 ns

6.11 ns

6.00 ns

LamberCSharp

new (…)) }, [353]

177.7 ns

0.23 ns

0.21 ns

LamberTO2

new (…)) }, [353]

313.1 ns

0.90 ns

0.79 ns

LamberCSharp

new (…)) }, [350]

145.4 ns

0.68 ns

0.60 ns

LamberTO2

new (…)) }, [350]

294.8 ns

0.90 ns

0.80 ns

LamberCSharp

new (…)) }, [352]

172.9 ns

0.36 ns

0.32 ns

LamberTO2

new (…)) }, [352]

309.7 ns

5.25 ns

4.65 ns

LamberCSharp

new (…)) }, [351]

146.1 ns

0.27 ns

0.25 ns

LamberTO2

new (…)) }, [351]

292.6 ns

4.03 ns

3.77 ns

LamberCSharp

new (…)) }, [347]

152.9 ns

0.31 ns

0.26 ns

LamberTO2

new (…)) }, [347]

292.9 ns

4.59 ns

4.07 ns

LamberCSharp

new (…)) }, [347]

153.1 ns

0.40 ns

0.38 ns

LamberTO2

new (…)) }, [347]

297.8 ns

0.94 ns

0.88 ns

LamberCSharp

new (…)) }, [349]

175.2 ns

0.20 ns

0.17 ns

LamberTO2

new (…)) }, [349]

303.1 ns

0.83 ns

0.78 ns

LamberCSharp

new (…)) }, [351]

169.9 ns

0.48 ns

0.45 ns

LamberTO2

new (…)) }, [351]

306.2 ns

4.58 ns

4.28 ns

LamberCSharp

new (…)) }, [351]

152.1 ns

0.25 ns

0.23 ns

LamberTO2

new (…)) }, [351]

284.8 ns

2.09 ns

1.96 ns

So the TO2 implementation is about 1.5 - 2 times slower than the C# implementation, but …

… the TO2 also bakes in several checks to prevent a poorly written script to crash the game:

  • Loop timeout: Since TO2 scripts are running in the main game loop/thread an endless loop in a sync function could freeze up the game completely.

    • To prevent that every loop in a sync function contains an implicit timeout check, which is currently set at 100ms. So there might be a small hick-up, but now a freeze.

    • This does not affect loops in async functions (which is the default) or if the function is running in a background thread (see core::background)

  • Stack overflow prevention: Creating to a stack overflow in the main game loop/thread will crash the game

    • To prevent this every function call keeps track of the call-stack an fails early before a hard stack overflow can occur.

All these security measures eat up runtime. If they are removed (aka release the breaks) the same benchmark looks like this:

Method

testSet

Mean

Error

StdDev

Median

LamberCSharp

new (…)) }, [351]

171.0 ns

2.85 ns

2.66 ns

172.2 ns

LamberTO2

new (…)) }, [351]

166.9 ns

1.09 ns

0.97 ns

166.5 ns

LamberCSharp

new (…)) }, [353]

172.0 ns

3.34 ns

3.71 ns

175.1 ns

LamberTO2

new (…)) }, [353]

172.7 ns

0.25 ns

0.23 ns

172.7 ns

LamberCSharp

new (…)) }, [350]

146.3 ns

0.22 ns

0.20 ns

146.3 ns

LamberTO2

new (…)) }, [350]

156.6 ns

3.11 ns

2.91 ns

157.0 ns

LamberCSharp

new (…)) }, [352]

173.5 ns

0.24 ns

0.23 ns

173.5 ns

LamberTO2

new (…)) }, [352]

173.7 ns

2.39 ns

2.24 ns

174.9 ns

LamberCSharp

new (…)) }, [351]

151.0 ns

2.95 ns

3.15 ns

153.0 ns

LamberTO2

new (…)) }, [351]

159.7 ns

0.42 ns

0.39 ns

159.6 ns

LamberCSharp

new (…)) }, [347]

153.1 ns

0.22 ns

0.20 ns

153.1 ns

LamberTO2

new (…)) }, [347]

158.4 ns

1.84 ns

1.72 ns

159.1 ns

LamberCSharp

new (…)) }, [347]

153.0 ns

0.18 ns

0.17 ns

153.0 ns

LamberTO2

new (…)) }, [347]

154.9 ns

1.90 ns

1.77 ns

153.9 ns

LamberCSharp

new (…)) }, [349]

173.0 ns

0.39 ns

0.35 ns

173.0 ns

LamberTO2

new (…)) }, [349]

177.1 ns

0.17 ns

0.15 ns

177.1 ns

LamberCSharp

new (…)) }, [351]

168.1 ns

3.24 ns

3.33 ns

169.9 ns

LamberTO2

new (…)) }, [351]

170.9 ns

3.30 ns

3.08 ns

173.0 ns

LamberCSharp

new (…)) }, [351]

146.9 ns

0.33 ns

0.26 ns

146.9 ns

LamberTO2

new (…)) }, [351]

155.8 ns

3.15 ns

3.63 ns

153.5 ns

… i.e. both implementations are pretty much on par