A friend sent me a link to a very minimal version of this same idea. An analog DC motor driver looking at the light level on the screen of a smartphone and using that to drive a robot. Around. Honestly, I'm hard pressed to see why that isn't better? 

https://www.voltpaperscissors.com/diy-smartphone-robot

The only advantage of using servos (which require a controller) is that the speed regulation is better.

I'm not crazy about the fact that he is using an app / photos / etc... instead of just using a web page. Also, his remote control depends on his server, but that could easily be changed. I think a local server is much better to start with especially as a gateway to the joy of simple e.g. node.js web servers or ESPs or whatever. Many security issues with an external server and apps. 

Of course, the analog route doesn't really support controlling other devices, but... shrug... each new device is just a square on the screen and a simple circuit. 

Also, speed regulation is really a non-issue if the program running in the phone can "learn" and watch it's accelerometers, and compass.