Close

Hackchat of the start of this..

A project log for P43D - Ping Pong Pixel Painter

Track the size and angle of a lighted ping pong ball and use this to draw in 3D virtual space.

daren-schwenkeDaren Schwenke 01/20/2021 at 19:301 Comment

Daren Schwenke11:14 AM
Dropping an idea here. Light up and track a ping pong ball on a stick with a stationary camera using OpenCV using size as your depth cue. Re-create as a 3D model, displaying it with rotation around the center of it. Allow changing the color of the ball, duplicate in the model.

3D light painting in realtime.

Daren Schwenke11:16 AM

Suppose you could use IMU and a cell phone and guess the approximate location of the camera as well to eliminate the stationary bit.

deʃhipu11:29 AM
you can also use looming to determine distance

deʃhipu11:29 AM
that's how insects avoid obstacles

anfractuosity11:35 AM
you mean using just a single camera then?

deʃhipu11:41 AM
sure, and low-res too

anfractuosity11:41 AM
i'm curious how well using a single std. camera would work compared to stereo

deʃhipu11:44 AM
one thing with looming is that it requires motion

anfractuosity11:44 AM
ahh

deʃhipu11:44 AM
works best when you move towards the object at a known speed

deʃhipu11:45 AM
you basically see how fast it grows -- objects that are close grow faster than object that are far

anfractuosity11:46 AM
could you simply use the diameter of the ping pong ball in pixels, from a single camera, and convert that to depth, which i thought is what @Daren Schwenke meant, or are there reasons that doesn't work well compared to 2 cameras

deʃhipu11:47 AM
it's easier than stereo, because instead of finding the same object on two different photos taken at slightly different angles, you just need to find the same object at two different sizes

deʃhipu11:47 AM
@anfractuosity that requires pretty large resolution, though, or very consistent lighting

anfractuosity11:48 AM
and knowing the size of the ping pong ball in reality too maybe?

deʃhipu11:48 AM
ping pong balls have standardized size

anfractuosity11:48 AM
oh true heh

anfractuosity11:49 AM
i think ToF cameras sound pretty nifty, not sure how pricey they are though

deʃhipu11:49 AM
Kinect

anfractuosity11:49 AM
the first one used a pattern i thought, is the 2nd one tof then

deʃhipu11:50 AM
if you can control the hardware in the wand, there are easier ways to do this

deʃhipu11:50 AM
like the wii rmote

anfractuosity11:50 AM
oh that uses ir i think?

anfractuosity11:50 AM
and a sensor bar thing right

deʃhipu11:50 AM
doesn't matter if the light is visible or not

anfractuosity11:51 AM
oh actually the 'bar' is the light emitter i think actually

deʃhipu11:51 AM
the important thing is it's modulated, so you can tell it from the background easily

anfractuosity11:52 AM
just found this, which sounds v. interesting, will have to read the paper

anfractuosity11:52 AM

https://www.youtube.com/watch?v=ZolWxY4f9wc

YOUTUBE TWO MINUTE PAPERS

Kristin joined  the room.12:04 PM

deʃhipu12:05 PM
The Algorithm

Nolan Hergert joined  the room.12:39 PM

MS-BOSS12:44 PM
The problem is that it will work well when you move near the camera. As you get further from the camera, the precision will drop fast.

Daren Schwenke12:46 PM
yes

deʃhipu12:53 PM
put the camera on the ball, and look for blinking light beacons in the anvironment

deʃhipu12:53 PM
environment

deʃhipu12:54 PM
you will also get orientation info from it

deʃhipu12:54 PM
and the farther they are, the more precision you actually get

Barry Baldwin joined  the room.12:57 PM

Daren Schwenke1:07 PM
The problem is with any camera held by the user, it has to be oriented towards some reference to work. That limits the size of what you can create versus a ball on a stick pretty quickly. Also with putting multiple sources or points, then you have to deal with orientation feedback or the distance between your points is no longer your relative distance to the object.

Daren Schwenke1:07 PM
A bigger ball would work and give you more accuracy at a greater distance.

Daren Schwenke1:08 PM
I just liked the small size too. :)

Daren Schwenke1:11 PM
button to select each color/turn on. On while held so you can stop your lines. Or select color via some interface then only light up when selected I guess.

Daren Schwenke1:16 PM
Hmmm.. how about a tetrahedron made with multiple balls on each vertex. Then the depth cue could be garnered from that. Probably the same level of accuracy increase as just using a larger ball, but then you could get orientation of the 'brush' as well.

Daren Schwenke1:18 PM
Or... if you are tracking multiple balls, when two are lit and the same color draw that as a plane in space.

Daren Schwenke1:21 PM
I did something similar for detecting distance to target with the Creeper project using the distance between your eyes. It was only accurate out to about 10ft at acceptable framerate. Tracking an object like a lit ball is a whole lot easier though and real-time at much better framerates.

Daren Schwenke1:21 PM
and resolutions.

Daren Schwenke1:23 PM
ooo.. in the virtual representation, you could have bins or areas in 3D space where you could position the target to select other features like line width, patterns, etc.

Daren Schwenke1:25 PM
I like the ball idea as well cause it puts the expensive bits remote from the user. You could literally use a flashlight, gels, and a ball to draw with no electronics needed on the end user side. Handing them out for an interactive thingy would not be cringe inducing.

Daren Schwenke1:34 PM
I just had a flashback to the MS 3D pipes screensaver.

Daren Schwenke1:35 PM

https://www.youtube.com/watch?v=MKqrLGFoK9E

YOUTUBE RONJAMESSCARESME

Discussions

marazm wrote 01/24/2021 at 19:46 point

hack the rpi 2040

  Are you sure? yes | no