Introduction

In computer vision, Bundle Adjustment takes multiple images and refines the estimated camera pose between them.

I've been spending the past few days manually doing something similar with the projects Tetoroidiv, Tetrinsic, Tetent, Tetrescent and the newly renamed Tetinerary (which has been renamed from Itinervate as a direct consequence of this Bundle Adjustment).

I've decided to write everything in this log so that I can refer to it in subsequent logs in their respective projects.

Considering the phrase "Form vs Function", I'm designating that Tetent = Form and Tetrescent = Function, in that the aesthetics would be prioritised in Tetent and the functionality will be prioritised in Tetrescent.

Transflective Screen

The bundle adjustment started when I started searching for screen options when bundle adjusting for Tetent and Tetrescent. Tetent would be fine with a small square screen, but Tetrescent needs to be large enough to reasonably be used as a typing tool, such as Freewrite. 

I also had the idea of having a similar design for Tetrescent, where instead of a sigma, the side profile would be closer to '>_', resulting in the screen having a 30 degree angle up from the desk plane.

Not wanting to have to create UIs for two drastically different screens, I decided that Tetent would also use the same screen that Tetrescent used. 

Since Tetrescent is obviously designed to be used outside (where the sun is), A transmissive screen could not be used. I also strongly oppose sub 60Hz framerates, so an e-paper display wouldn't suffice either. It's possible to design a monochrome UI, but colour would be preferable. Noting some of the complaints people had with the Playdate and the LED mods implemented into the Alphasmart devices, I also needed something that could illuminate in low lighting conditions.

The first option I decided on was the Asumo 3.4" colour which is £63 on Digikey, which is higher than the £50 I budgeted for the screen but it had the dimensions I was looking for and front-lit. 

Then I found out that a 3.5", 640 x 480, transflective screen existed. It uses MIPI though, so not exactly ideal. What was ideal though, was the resolution. Tetinerary, formerly Itinervate, uses 640 x 400px optical modules, and this resolution is also very prevailent in the AR glasses industry. If I could implement this transfelctive screen into Tetent/Tetrescent, it would mean that I could then port things over to Tetinerary very easily with minimal circuit changes. It also meant that I could develop, test and validate the software (and battery life) on cheaper hardware before committing to creating Tetinerary.

The optical module uses RGB888, and I was able to find a similar RGB666 transflective screen from the same supplier, only it's out of stock:

Eventually, I was able to find a practically identical panel on AliExpress, and it's within my budget too:

Next, I looked to see if there were any YouTube videos of the panel and there is one that shows that it surprisingly has excellent viewing angles, which is important for Tetent as the screen will be aligned with the desk plane (i.e. facing straight up).

As the panel was 3mm thick, it made more sense aesthetically for the top part of the sigma to go over the Tetrinsics (instead of the Tetrinsics cutting into it). This then opened up the possibility of putting a 3Ah battery under the panel and small stereo speakers (the same ones that have been featured in concepts in the past) facing towards the user. I'm going to be using dual speakers per ear for Tetinerary, partially because there likely won't be any space for a single speaker over the ear and partially to obtain planar 360 degree audio. 

The battery move was important, as I hadn't gotten the power performance I wanted out of initial Tetoroidiv simulations and a good solution needs more space than what a 238mm belt can provide. Conveniently, there is a 268mm v-ribbed belt available in both orange PU and black rubber. I've never felt either, but the black rubber belt is more aesthetically pleasing so I'm using it on Tetent. Tetrescent requires 340mm long belts, which only comes in PU.

Microcontroller and OS

A VGA screen needs an input video signal, and this the hurdle I was at with Tetinerary. Typically, microcontrollers (ESP32S3, RP2040) can't run particularly high framerates, especially for something like 18-bit 640 x 480. There have been some 3-bit solutions written. I also had concern about actually making the UI look nice and have fluid animations. I worried that, due to the low hardware performance of MCUs, the libraries that exist would only allow for basic graphics.

On the other side of the spectrum, something that can run Linux or Android would open Tetent to a large suite of already-written apps, as well as MAUI support. One potential issue is power consumption, but "full android" smartwatches exist and get about 6hrs of runtime on a 600mAh battery. The bigger issue is dealling with such huge software projects. The options that exist are for general purpose hardware, and Tetent is quite the opposite. There is a chance that implementing all the changes needed to make Tetent work well would actually be more labour intensive than writing firmware and all the applications for a micrcontroller. Similarly, many of the apps that I want on Tetent don't even exist on these platforms anyway.

One thing that makes the MCU route more palatable is the existence of Tock, which is a Rust-written project that claims that I can write 3rd party apps (like a more traditional OS) and used to run smartwatches. I've been reading about embedded Rust recently and it sounds more beneficial than C++, so I'm planning to use it as my "backend" language and have C bindings so that I could use F# + MAUI for other targets. In this way, I could develop applications that can run on microcontrollers, microprocessors and standard processors with a shared core codebase.

I then rediscovered the STM32U5 series, with the STM32U5G9VJ sounding quite ideal. It's a low-power MCU that can output RGB888 or 2-lane MIPI DSI, and they've tested it on a 800 x 480px display. This chip also has enough ADC channels for 5 Tetrinsics, with loads of channels to spare! There are 2x 14-bit ADCs that have 20 multiplexed channels each and have a nifty-sounding dual mode feature, and all this can run at 2.5MSPS. This means that I could sample all 10 current sense inputs, and then all 5 force sense inputs, and do it at 250kHz. It makes sense to use the 512x oversampling feature to get a final read speed of 480Hz. There's conveniently even battery voltage monitoring, which could be read from the additional 12-bit ADC. 

There's also some fancy sounding "multi-function digital filter", where "motor control" is one of its target applications. This brings me onto my next point, which is that it sounds like MCUs typically struggle with generating the PWM frequencies for BLDC control. For example, for SimpleFOC, many microcontrollers support 3PWM output but much less support 6PWM. I looked into it, and it turns out that 3PWM drivers are more ideal, but cost more than 6PWM drivers. Well, I think something even more ideal would be for the MCU to just tell the controller over SPI what PWM it should be driving and it handles it. I believe that's what the DRV8311 does. With this, I hope it's possible to drive 5 motors from the single ARM-M33 core of the U5G9 chip.

Recently, a motor with an integrated absolute encoder was discovered and posted on the SmartKnob discord, and its specs are great. The TLE5012B E3005 has got a 16-bit ADC and can communicate with an SPI compatible protocol. It's also £1.15/ea on AliExpress.

With all this in mind, and in an effort to reduce the PCB space requirements of Tetoroidiv, reduce costs, simplify PCB manufacture and minimise the toolchain complexity of this entire project suite (e.g. flashing firmware), I've decided that Tetoroidiv won't include an MCU and the U5G9 will (attempt to) run everything.