I started this project to measure the input delay of the TV that I use for gaming. Specifically, I wanted to know what kind of delay I was experiencing when playing an FPGA-based NES emulator on the same FPGA and DVI output hardware. I'm pleased with the end results, and I think this project could serve as the basis for a more advanced input delay measurement tool.

For an overview and demo, check out the video on YouTube:



Components

The project uses a Digilent Nexys A7-100T FPGA development board for its FPGA platform. The HDL code is written in SystemVerilog, and all of the demos were synthesized in Vivado 2022.1. The DVI output is generated with the 24bpp-DDR version of the PMOD Digital Video Interface from 1BitSquared. It uses two adjacent PMOD expansion ports on the FPGA board. The light sensor circuit is built from an Everlight PD204-6C photodiode, a Microchip MC6291 op amp, and a Microchip MCP6541 comparator. It connects to the FPGA board through a ribbon cable and a hand-soldered, 3-pin PMOD expansion board.

The FPGA board and PMOD DVI chip were chosen because I already own them and have experience with them on much larger projects. The photodiode was chosen for its low cost, its visible spectrum response (be careful not to pick infrared only!) and its relatively fast response time. The Microchip ICs were chosen for their low cost, low voltage and power requirements, CMOS inputs (it's especially important for the op-amp to have virtually no input current) and relatively high speed. The ICs also had to be through-hole PDIPs for easy use on a breadboard / prototype board.

Design Goals

I jumped into working on this project without researching how other projects or commercial products perform these kinds of measurements. My goal was to measure the time elapsed from the moment a pixel's data is sent to the DVI chip to the moment the light sensor gets triggered. In order to meet an overall margin of error of +/- 100 microseconds, I designed the light sensor circuit with a response time bounded by about 25 microseconds, and I designed the pixel pattern generation and timing algorithm to have a margin of error of +/- 55 microseconds. My plan was to enable measuring the input delay at nine positions around the screen using a simple button and seven segment display interface on the FPGA board. My final design goal was to power the light sensor circuit using the FPGA's 3.3V supply voltage, i.e. without providing my own power supply.

Uncertainties and Challenges

My educational background is in digital design, and my career is in software engineering. I've only dabbled in analog circuitry as a minor hobby, so my confidence in the light sensor design is not very high. I think the strategy of amplifying the small photodiode signal and converting it to a digital signal with a comparator is a decent, textbook approach, but I don't have the experience to know whether it's totally sound. I'm also not entirely confident in the response time estimates. It would be much better to measure the circuit's actual response time with the one-shot feature of a digital scope. Unfortunately, I only have a 20MHz analog scope, and using it to observe the fast switching time of this circuit was difficult, if not comical.

I do, however, feel confident about the rest of the design. I had developed the 720p output circuitry for my NYDES (Not Your Dad's Entertainment System) project, and the remaining components of the design would be appropriate assignments for a first course in undergraduate digital design lab work.

Features Demonstrated

For readers interested in learning FPGA development or digital design, this project demonstrates several basic techniques:

Results

I tested the circuit primarily on a Sony KDL55HX750, an LED TV from 2012 with a maximum 240Hz refresh rate. The TV was set up in Game Mode with all processing features turned off. The results of using the measurement tool were somewhat surprising. I found that despite my efforts to keep a tight margin of error (0.1 ms), readings jumped all over the place by up to a few milliseconds.

The most likely cause of these inconsistent readings was the pixel response time of the display. LED displays have an LED backlight, but they use an LCD matrix of tiny apertures that allow the LED backlight to shine through. The response time of the LCD apertures can be quite slow (tens of milliseconds), especially when transitioning from completely closed to completely open. The light sensor circuit was designed to only trigger for the brightest backlight setting on this test TV (really, for slightly less), so this "pixel response time" was a big factor in measurements.

Measurements were most variable in corners and along the edges of the display, where the backlight tends to be a bit dimmer on this older TV. The variability in measurements was probably due to the light sensor not being triggered as "strongly." In these trouble spots, I had to hold the photodiode very still against the screen to get the reading to stabilize for a few cycles. In brighter areas, it was easier to get a reading. My suspicion is that the LCD pixel response time behaves like an RC circuit, opening a pixel's aperture along an exponential curve. If this is the case, opening to 100% would take quite a bit longer than say, 80%, and minor perturbances in the photodiode's position and orientation would produce different timings where the backlight is barely bright enough to trigger the sensor.

I concluded that this tool is actually measuring several different timings:

  1. The margin of error delays in the tool itself along with any possible (undocumented) delays in the DVI driver chip
  2. The video signal buffering and processing in the TV or monitor
  3. The pixel response time of the LCD matrix (at a variable level of "opening" depending on the local backlight brightness)
  4. The full-screen pixel refresh time

The timings of (1) - (3) are pretty self-evident from the description so far, but (4) is interesting to consider. Based on the timing of the video signal generation, we can calculate that 15.9 ms elapse between rendering the top and bottom rows of testing patterns. If the TV finishes buffering and processing the video signal and then updates the entire screen instantaneously, we would expect the difference between the top row and bottom row measurements to be exactly 15.9 ms. Any difference greater than 15.9 ms would be the time it takes the TV to refresh the values of all the pixels, scanning from top to bottom. We can also note that a difference less than 15.9 ms would indicate that the TV refreshes the pixels in a way that "keeps up" to the input signal with less than a full screen of buffering. Note that a pixel refresh operates independently of pixel response - the refresh simply tells the pixels what their next target "size" is, regardless of whether they've reached their previously targeted "size." This discrepancy is the source of "ghosting" in moving images.

As shown in the demo video, these timings come out to 47.0+/-0.1 ms (top row) - 26.7+/-0.1 ms (bottom row) - 15.9 ms (input signal difference) = 4.4+/-0.2 ms, so this TV appears to buffer the entire screen and then refresh the pixels in about 4ms. For this TV to actually run with its specified 240 Hz refresh rate, it requires a minimum full-screen pixel refresh time of 1 / 240Hz = 4.167 ms. Interestingly, this falls (very nearly) within the margin of error of our measurement, 4.2 ms.

If we accept the inference that this TV's pixel refresh time is 4.2ms, we can estimate bounds for pixel response time and for buffering / processing time. These actual times will lie somewhere in between, but we cannot determine them without more clever measurement techniques (see ideas below).

We know that the last row of pixels lights up at 26.7+/-0.1 ms after the last "row" of pixel data is sent to the TV. If the TV initiates the full-screen pixel refresh at the moment that the last pixel's data is received, it must have received it 26.7+/-0.1 ms - 4.2 ms = 22.5+/-0.1 ms earlier. Taking the high side of the error margin, 22.6 ms is the upper bound for the pixel response time. At the other extreme, if we assume the pixel response time is zero, then 22.6 ms is also the upper bound for the buffering / processing time.

I would speculate that the actual buffering / processing time is slightly longer than 15.9 ms to accommodate a pixel pipeline for scaling, adjusting color and contrast, etc., which would mean the pixel response time is slightly less than 22.6 ms to make up the difference. For gaming at 60 Hz, and especially for retro gaming, I believe we can definitively conclude that this TV has at least one frame of input delay, possibly two frames for very high-contrast content. If you define a frame delay by the first pixel or the center pixel, results will be worse. I believe that most experts I've read on this topic would call this TV "good enough for normal humans." That is, it's better than the perceptible limits of most people's resynchronization of reality engine in their thread of consciousness.

For contrast, I also tested my main computer's LG 32QN600-B monitor. I measured a 3-4 ms delay everywhere on the screen, suggesting that the monitor operates in the "keep up" mode without any significant buffering, much more like a CRT TV. In any case, my own nostalgia and muscle memory are held intact by playing on either the TV or the monitor for gaming on my FPGA NES implementation. I'm just more comfortable playing Nintendo on a couch. I guess I could move my computer :)

Potential Future Improvements

For now, I'm planning to shelve this project, but I do have several ideas for further improvements. Up to this point, the project is appropriately called a hack. Going deeper might approach the realm of product development, but here's what I would do:

  1. Improve the light sensor and output an analog signal
    1. Consider providing a separate, clean power supply with a higher voltage. This would allow us to decrease the photodiode's capacitance by increasing the reverse bias voltage, and it would provide more headroom for amplification.
    2. Amplify in two stages, at least one for amplifying and one for buffering.
    3. Measure the photodiode's response to a large variety of displays so the amplifier's gain can be tuned to fit the possible range. This may require a way to switch modes for different orders of magnitude in screen brightness (changing feedback resistor networks?), but it might let us eliminate the trim pots. The analog output should span the FPGA board's power rail voltage.
    4. Accurately measure the full circuit's response time(s) with a digital scope, so that measurement errors can be accounted (to within a better margin of error than 100 us).
  2. Use the FPGA's ADC for sampling the sensor's analog signal
    1. Using an ADC, we can calibrate each measurement to the local backlight brightness
    2. We can also measure various levels of brightness, opening a new realm of possibilities
  3. Support higher resolutions and framerates (or lower!)
    1. The 12bpp DVI PMOD from 1BitSquared can easily drive a 1080p signal at 60 Hz. In my experiments, I couldn't get 1080p working with the 24bpp DVI PMOD, as it requires faster signals than can reasonably be driven by PMODs - trace length and other parasitic capacitance issues become significant past 150 MHz (or only ~40 MHz, according to Digilent).
    2. A board like the Mimas A7 is relatively cheap and has its own HDMI output with a v1.3a transmitter, which is good enough for 1080p. Presumably, it would use the Xilinx toolchain similarly to Digilent's boards.
    3. Achieving 4K at 60 Hz for cheap might be possible with the Xilinx Kria KV260's DisplayPort 1.2a output, but it's a System-on-Module instead of just an FPGA board. It would require climbing another learning curve to use it effectively.
    4. If money is no constraint, perhaps a board based on an FPGA like the Intel Arria 10 could support higher resolutions, even up to 8K.
    5. Also, we could consider trying lower resolutions like 480p to make the system cheaper. The power of this design is that it can provide a precise way to measure the timing of the processing pipeline after a particular pixel or a full frame of video is received by the display.
  4. Build an enclosure for the sensor
    1. The sensor should be enclosed in a case that only exposes the photodiode.
    2. The case could use a ring of material like silicone to help hold the case flat and still against the screen.
  5. Improve the measurement algorithms
    1. With the analog signal and ADC from (1) and (2), we could calibrate the sensor's output at each measurement position to the local backlight brightness. This would allow us to measure relative brightness and test pixel response timing transitions precisely, since we may be able to detect the moment that pixels begin to change, i.e. when the pixel refresh signal has been received.
    2. We could measure statistics like the distribution of backlight brightness, the variance in pixel response time, etc.
    3. If the measurements in (5a) are accurate enough, we would be able to distinguish the measurement of buffering / processing time from pixel response time.
  6. Improve the interface
    1. Measurements could be aggregated to removable storage media (USB stick or SD card) or sent to a computer over UART, Ethernet, Bluetooth or Wi-Fi.
    2. The measurement position could be detected automatically by sending a binary search pattern to the sensor to "find" where it is located, perhaps after a button press.
    3. We could automatically measure and aggregate a large number of samples for statistical analysis, automatically tracking the position where they were taken on the screen.
    4. We could write software with a UI for offline analysis of the aggregated metrics.

Most of these sound like fun!