Testing the BlueRetro software revolves around 4 distinct categories: Manual, Community, Automation & Performance testing. As of today the vast majority of the test were from the manual testing category which is my wife, my friends and I playing games with BlueRetro. Each release I try to retest as much as possible.

Manual testing

I retest the following every release by playing around 5 minutes in each test. (Probably take me around ~3 hours when motivated ;) )

• FC/NES
  • 2 players mode
  • Four score 4P mode
  • FC 4P player mode
• SNES
  • 2 players mode
  • Multitap 5P mode
• Saturn
  • 2 players mode / Digital mode
  • 2 players mode / Analog mode
  • 1 multitap slot 1 / Digital mode
  • 1 multitap slot 2 / Analog mode
  • 2 multitaps / digital mode
• N64
  • 4 players mode with rumble enable
  • 2 players / 4 controllers dual joystick mapping (GE / PD)
• Dreamcast
  • 4 players mode with rumble enable
  • GamePad Alt (2nd joystick + D,C,Z) mode
• GameCube
  • 4 players mode with rumble enable
• JVS
  • 2 players mode (only what my Arcade board support)
• Bluetooth
  • Generic HID controller (8bitdo Dinput / xinput)
  • PS3 Sixaxis & DualShock 3
  • Wiimote + Nunchuck & Classic controller
  • WiiU Pro
  • PS4 DS4
  • Xbox One S
  • Xbox adaptive controller + Logitech 3D joystick
  • Switch Pro
  • Generic HID keyboard

Community testing

BlueRetro community is still quite small but very valuable feedback is already provided and resulted in bugs fixes!

Going forward this will be the main way of debugging BlueRetro. I will make a limited release of the BlueRetro DevKit PCBs for the open beta.

The beta will help in testing the various Bluetooth controllers revision that exist for already supported devices and also test new devices I do not own. While the wired interface work well with the games I tested, I fully expect some games to be troublesome and I can't myself test the 1000+ games that compose BlueRetro supported systems library.

Automation testing

Automation testing will be handle by using another BlueRetro with a special test firmware that will allow simulating the wired system and the Bluetooth controller. The special firmware will be minimal and leverage the existing BlueRetro code. The higher level control will be handled by a Python script using the pytest framework for running the test suite.

Connecting the two BlueRetros together will be done via a special test jig cable.

Since some of the ESP32 pins are inputs only, it's not possible to simply connect each pin of both DB25 to each other. Since the host BlueRetro SD card will not be used, those extra IOs are used to control the input only pin of the DUT. The UART output of the DUT will be connected to the host BlueRetro for helping validation by parsing the logs. The JVS UART driver will be leveraged for the PC interface via the USB UART of the host BlueRetro.

Performance testing

Performance of BlueRetro is measured by the added latency in the control input versus an original wired controller. Latency in a video games controller is the time between a button is pressed and an action appear on the TV screen.

The easiest way to test this would be to put BlueRetro in one of the parallel modes and measure the time between the buttons press and the output pins change. The problem doing this is that the parallel mode is not representative at all of the vast majority of supported system that requires a more complex wired driver. Measuring at that point for a serial base protocol would make triggering automatically on the output event quite complex. For a single test it would probably be fine to analyze the trace by hand and figure the timing. But for characterizing the latency properly multiple tests will need to be done in the order of thousand measurements.

A little bit more complicated but more easier to replicate a thousand time approaches is to probe one of the analog RGB lines of the console output. This make the measurement process identical across all platform (At least with those with RGB output). The only thing required is a flash card to run a simple test program on the console to toggle the screen between black & white on buttons press & release.

The trigger is done when the RGB line toggle between 0V (black) and 0.7V (white).