Performance testing

[elementbound]

Background

There's plans to explore options to improve netfox's performance. But to do that, we first need a method to measure the current performance, and be able to compare new methods against it. Ideally, this measurement is automated and requires as little human intervention as possible.

Metrics

• Bandwidth
  • Sent bandwidth - min, max, avg, one second windows
  • Received bandwidth - min, max, avg, one second windows
  • See EnetConnection.pop_statistic
• CPU usage
  • Network tick loop duration
  • Rollback tick loop duration
  • Network ticks per loop
  • Rollback ticks per loop
  • Can be done with custom monitors

Implementation steps

• Perf test game
  • Game tries to host, otherwise connect
  • Address and port can be set from command line, defaults to localhost
  • Each client acts as a bot, wandering around randomly
• Implement NetfoxPerformance - feat: Performance monitors #195
  • Activate only in debug builds or if manually activated
  • Subscribes to netfox events and manages custom monitors
• Configure perf test build
  • Can use feature flag to differentiate
  • Runs perf test game for a given duration
  • Outputs perf data at end
  • Implement black box class to track and output perf test data
• ?? Configure perf testing GitHub action

Notes

• Why not pre-recorded inputs for Forest Brawl?
  • Pros: realistic traffic, as it's an actual game
  • Cons: doesn't scale well to arbitrary amount of players?