Finally, after half a year or more of struggle, I've figured out a not-inelegant way to record and playback robot states.  It is now in the dev branch of the app.

In large part my confusion was caused by poor definition of terms.  There are many ideas all referred to as "model" - the STL model of a 3d mesh, the Model in an Model-View-Controller pattern, the "model" designed to fit a paradigm, the simulated model of the robot... these all got muddled together.  The knot was so bad that I couldn't see how to untangle it.

What I've realized is that there's a description of the PhysicalLimit of a robot and within that any number of states.  Put another way, states are bounded by the PL.  so a given state is one MVC Model and the PL is the MVC Controller.

Why does this matter?  Well... I have a Sixi2 class that contains a PhysicalLimit, a Sim, a Live, a Cursor, and a set of states that form the recording of the robot.  Cursor points to one of the states in the recording.  i can then adjust one state by selecting it (making it the cursor) and then tweaking it.  Dragging the IK pose of the cursor causes Sixi2 to use the PL to find the new FK pose of the cursor (using gradient descent).  Put another way, PL is an MVC controller inside a Flyweight design pattern, and each state is a Memento - the minimum needed to restore the robot to that pose in the world.

I can command the robot to move to the cursor state, at which point both Sim and Live attempt to do so.  if Live is connected to the physical machine (the serial connection is open) then the real machine will move.  Because Sim is using the same path planner and timing as Live, the Sim and Live will move as one.  This is a cheap way to detect unexpected interference - variation greater than some epsilon is sign that the real machine is colliding with something.  It also gives me a way to get accurate time prediction on every recording.

It's late and I ramble.  Maybe none of this will make sense to you, but it's the gibberish that got me to this point.  I'm just glad the damn thing seems to work.  Tomorrow I figure out how to add tools on the end of the robot as part of the state, and then I can finally record pouring myself a drink.