-
March 2021 Update, >75Hz Frame Rate with Tracking WORKS!
03/31/2021 at 03:54 • 0 commentsThe project is back on track after losing a year to Covid19 & other distractions life throws at us.
Bottom line up front: Achieved 75Hz full processing including SBUS data read, frame grab, convert to numpy array, process image to track, generate PID control quad command, and finally SBUS write to the quad's flight controller.
Details:For now, I've abandoned the use of USB cameras due to cost and other reasons.
Switched to Pi 4 model b.
Currently switched to Pi Camera Module V2 noIR via CSI ribbon cable. Achieved high frame rates at 640x480 finally, 90fps in mode7. In order to perform manual image processing, I don't need the regular jpeg compressed frames, rather need frames as numpy arrays, which was also achieved with very low latency.
The SBUS update rate on the remote control system I'm using is 75Hz (Radiolink AT9S transmitter with R6DSM receiver on the quad).
Remember folks, the quad's flight controller (Omnibus F4 running Betaflight) would normally be commanded via SBUS from radio receiver. But in this project we run the SBUS through the Pi. The Pi can be in pass through mode or autopilot mode where the Pi can inject pitch, yaw, roll commands. I have not implemented 'throttle' yet, currently the Human controls throttle at all times.
Testing using my ceiling fan with a hair-scrunchy strung between blades using fishing line was conducted. To get the best loop rate, I shut off any (cv2.imshow) screen updates of the image. But I still need to see what the camera and processing was doing, so I store the images(numpy arrays) and X & Y tracker (box around target) output to local memory (just a python list). For test purposes, I can set the script to run for some number of frames ( I used 700 frames ), upon completion it writes .jpg of each image and plots determined X & Y track positions using matplotlib pyplot.
The PID flight control of pitch and roll is implemented and checked out by monitoring the Betaflight GUI for the values being sent to the flight controller.
Outdoor tests with very limited flight while tracking another quad (DJI Mavic mini) have also been conducted a few times.
The target drone was set up to fly around with a hard acceleration in one direction at first and then some minor movement in other direction as you can see in the plots of tracker position.
-
x220 Quad Loaded Weight Flight Tests
05/21/2019 at 03:54 • 0 commentsThe stock Arris x220 with battery is 524g. RasPi 3 B + c920 web cam + USBcable + chips-n-wires = 220g. This test has added 232g to 524g Quad for a total 756g. No CG balancing was conducted, just heaped some parts onto the Quad and used electrical tape for attachment. ( The c920 web cam housing with built-in stand and USB cable is very heavy, ripe for weightloss! )
See picture and videos below:
-
Basic Setup Tests, Learning, and Debugging
05/17/2019 at 03:06 • 0 commentsBasic SBUS, webcam, remote control target box, switched modes (start > manual targeting > auto tracking), OpenCV tracker, and casting to FPV screen working. Found a slight flicker/noise problem on SBUS read in Rasp Pi every now and then. I've debugged it down to the obtrusive OpenCV webcam frame grab .read(); so I plan to use threading.
My apologies, I need to increase my 'webcam' budget for taking these videos. (consider them 'retro' at NTSC 30Hz)
You may see the 'User Targeting' or 'Tracking' label on the screen. I'm using switch 'F' achannel(9) to control if user can move the box and resize (VrB Knob achannel(7)). When switching out of 'User Targeting' the code is setup to automatically start the tracker with that current box defining the ROI (region of interest).
-
Conducting the Development
05/14/2019 at 01:44 • 0 commentsSetup testbench to trade & develop:
- Image processing selection
- My custom algos need help...so let's hold off on those
- Let's use OpenCV built-in flavors
- Select a camera (limited to what's around the house)
- didn't get PiCamera V2 on ribbon cable to work (maybe more reasons than this to abandon PiCamera V2...framerates too low?)
- OpenCV captures webcam easily
- Rasp Pi receive & transmit SBUS serial from Quad's RF receiver and back to Flightcontroller
- need inverter since SBUS is inverted UART
- need 3.3v supply for inverter to keep SBUS thru inverter at 3.3v
- 5v not good for Flightcontroller board (Omnibus F4)
- need inverter since SBUS is inverted UART
- Rasp Pi to output analog video (composite) to Quad's VTX (video transmitter) so that the image can be seen on FPV goggle screen.
- Rasp Pi code to allow user to select region of interest square box on-screen.
- Pass this to OpenCV tracking algo
- Rasp Pi code to pursuit guide Quad based on control algo
- Based on error from boresight center
- Try to get at least 20Hz total loop time
- C920 cam happens to go 30Hz in bright environment
- Need to figure out SBUS repetition rate
- Afraid of camera frame grab vs. incoming SBUS serial collisions causing time issues
- Likely need 'threading'
- Image processing selection