Train games are in a pretty sorry state in general. 

Found a great game that my son likes, it's called Metro Simulator 2019. It's in an active development phase and I'd like to connect my controls to it. Asked the devs about the API and was a bit surprised to get a rather harsh negative reponse, not now and never. A bit disgruntled by such shortsightedness on their part.

Previously I supported Subway Simulator 3D. It is a kind of game where you can't even count on an answer about what would be an absolute blocker bug in a sane world. Forget about asking for an enhancement. However I found a relatively simple solution for interfacing with it. They draw train state in a fixed location on the screen. So I made an adapter that takes a screenshot and scans the dials. It's not a polished product but just a quick hack and here's what it looks like:

The magenta is debug graphics that shows what the program is seeing, or rather paying attention to. We have a speed gauge, from which it's easy to tell the current speed and send this information to my physical speed gauge. 

More interesting are the brake lever position (here incorrectly labeled BP, Brake Pressure) and the throttle called "1000 AMP" (you guess about the correctness). The game only takes relative commands: throttle + / -, and brake + / -. No sticks. However by reading the current position of the lever displayed on this nice indicator I create a closed loop control that corrects the error by sending throttle up/down and brake up/down keypresses. When the program is uncertain about what it sees it stops sending updates until the gauges appear back gain. This works surprisingly well.

A bonus feature, I scan the state of the semaphore and blink my light accordingly. This game has no reverse control (yes, this advanced), so my reverser acts as an emergency brake.

Back to Metro Simulator 2019. Here I don't have the luxury of a static display that's always on screen. The trains have very elaborate and realistic control surfaces that display a ton of information about speed and the current state of controls. Unfortunately, this info is a part of 3d cockpit and can be anywhere on screen, or even completely off screen when you look away.

I guess I have three possible paths here:

  * reverse engineer the relevant game dlls and just rip the data such as speed and lever position from the train model

  * implement a more advanced machine vision scanner that would work in most cases

  * forget about physical speed display and simply emulate joystick inputs, which this game seems to support (unverified)

Here's what the relevant part of the instrument panel look like when in game. The most interesting parts are highlighted with perfect Pentium circles.

Realistically, the cockpit is almost always visible anyway so machine vision is not necessarily a terrible idea. I like it because it's not invasive and will probably survive many game updates. I think it shouldn't be very difficult to train a HOG classifier for a speed gauge, which has a distinctive look. Things like current state of the main lever are on a screen to the right. Maybe its edges can be found with relative ease once the position of speedometer is found.

Another option is to just emulate joystick input. This game has joystick support so although I won't get physical speed gauge, the input would still work. Kinda meh but it's the most doable solution considering time constraints.