Design Updates

Our team made significant process in reconfiguring the camera module into a web client in 4 key areas:

Image transmission. The camera’s Zentri AMW004 wifi module, as controlled by the Microcontroller Unit (MCU), is now capable of transmitting images to the host server on Google Cloud. This is done by sending HTTP POST requests with the appropriate headers for a image/jpeg Content-Type, and the Content-Length matching the size of the image file. At time of report, the Camera system is able to output about one frame of image output every second to the server in this manner.

Google Cloud server solution. The Google Cloud compute instance is the workhorse of the security camera system. Currently, a python Tornado web server accepts HTTP requests from any properly configured device on the internet that sends the appropriate headers. The contents of the request are stored as a jpeg device on the disk and a HTTP response indicates that the transaction was successful. A web interface allows a user to watch the image feed ‘live’ conditions, at a 200ms local refresh rate (the camera continues to be the bottleneck at one frame per second). Where appropriate, the response text will include data for x and y coordinate for the location of a detected face (see section on OpenCV).

PWM servo actuators.  The servos were successfully tested using the breakout boards. We were able to control the servo positions on the pan-tilt base using PWM and altering the value of the PWM duty cycle.

OpenCV facial recognition. The OpenCV library for python has been integrated into the web server. Currently, we are using a Haar Feature-based cascade classifier for face and eye detection. The classifier works reasonably well on the low pixel count of the images, but only when the face is directed squarely at the camera. It is not capable of distinguishing between faces of different individuals either.

The overall state of the project is healthy. So far, we are on target to meet the project’s minimum viable product goals. More time will be needed than expected to integrate the PWM servos into the current solution as this will require the wifi module to interface with the microcontroller seamlessly.

At the current moment, the stretch goals of incorporating and LCD screen and per-user facial recognition may not be feasible given the amount of development time and testing required to complete the features in the current iteration.

Planning and Organization

Figure: Gantt Chart for the project

Figure: Bill of Materials for the project

Results of Market Research

Our non-expert interviewees found a product that could recognize specific faces to be very useful in monitoring their private spaces, for example, alerting against front door package thieves or trespassers. On the topic of facial recognition, one interviewee noted that constant face detection alerts could get stale because of false alarms, and suggested using criminal face databases to reduce false alarms.

We will be interviewing Prof. Jack Tumblin for his expert input on the computer-vision enabled security camera product that we are making.

Next Steps

By the next presentation, we will have the servos autonomously operating as faces are detected by the camera system and the cloud server. This will require steps to be taken to integrate PWM functionality into the codebase as well as the ability to parse HTTP responses. Should this fail, we will operate the servos either by manually commands from a web interface or on an automated timed loop, panning across the entire room.

We will aim to improve the OpenCV recognition capabilities of the Google Cloud server to recognize faces more reliably, at more extreme angles, and potentially differentiate between recognized (trained) and unrecognized faces. Should the servo be too noisy, we will implement a filter to attempt to reduce noise in the servo movements.

Finally, as a stretch goal we will attempt to increase the frame rate of the camera to about 5 frames per second to increase the usefulness of the closed loop servo control.

References

https://www.cnet.com/pictures/security-cameras-with-facial-recognition-tech-inside/

https://docs.opencv.org/3.3.0/d7/d8b/tutorial_py_face_detection.html

https://www.adafruit.com/product/1967

https://www.adafruit.com/product/181

https://usa.banggood.com/3pcs-Mini-OV2640-Camera-Module-CMOS-Image-Sensor-Module-for-Arduino-p-1452968.html

https://www.digikey.com/product-detail/en/silicon-labs/AMW004/1586-1012-ND/5269916

https://www.digikey.com/product-detail/en/microchip-technology/ATSAM4S8BA-AU/ATSAM4S8BA-AU-ND/3593645