Close
0%
0%

Media Center automation

Using OpenCV and 6 x Raspberry Pi to detect if rooms are occupied and them turn on and transfuse media to the relevant OpenElec frontend.

Similar projects worth following
The purposes of this project is to track occupants around house, while transferring media to the nearest TV.

The project was completed as part of my studies at the University of Derby.

The aimed to explore and test an alternative control surface for the purpose of controlling home media content, giving an element of automation requiring minimal user input. Existing control surfaces such as remotes require direct interaction from the user to make changes and react to feedback whilst navigating the interface. This requires a number of prerequisites, such as good eyesight and the ability to read the text on screen, as well as being able to locate the remote control. The project aids in the transfusing of media between television sets and allows the user to start playback without a remote. This is done with minimal user interaction, adapting to the users movements and changing conditions to the environment. The system will uses small HD cameras and video processing to detect the

Introduction

Having discussed the aims of the project, the documentation for the project will serve to explore existing similar home automation solutions to ensure that there is a ‘gap in the market’ for this type of solution. The documentation will also offer support and provide a rationale for decisions made in the course of the project, as well as evaluating its overall effectiveness.

Project Rationale

Historically there has been little synchronicity between media centres and home automation, both devices were relatively standalone and unaware of the other’s function. For example, in home automation systems lighting fixtures would monitor specific devices for input, such as a light switches. This relationship could be described as ‘paired with’ whereby output devices such as a light fixtures and plug sockets would be associated with an input device, such as a light switch. A user may find this useful as lamps plugged into the socket would turned on in a room when they turned on the main light, all without the electric being re-wired within the house. Despite this advantage, the early home automation also came with a disadvantage as devices were not aware of each other’s state. This began to change when computers were programmed to run software or to use hubs to link the different devices together. An example of how this would work is that the hub or computer would have an overview of all of the devices in the home, and one way it would function is if the home owner set the house alarm after 8 pm, this could trigger all devices to turn off. However, this would be a disadvantage if the homeowner wanted to watch television upstairs. A better solution is needed. Therefore, the rationale is to provide a more synchronised media centre and home automation system that will improve the user experience. The system should also take into account ways it could be made more accessible for users who cannot read, whether through sight problems or illiteracy. The best way to do this will be evaluated in the course of the literature review and its effectiveness analysed.

Project Aims and Objectives

The purpose of this dissertation is to investigate and implement ways that a media centre could provide a better user experience and interact with the users in a home environment. For example, if a user was watching Top Gear on a television in the living room, and then turned the light off in the living room and went up to the bedroom, it would be beneficial if the system could detect the changes in state of the living room and bedroom (changes in occupancy) and proactively turn the TV off in the living room and transfer Top Gear to the television in bedroom without the need for any user interaction.

The second aim is to allow the uses to scan self-adhesive NFC Stickers that can be scanned by any smart phone to directly indicate an action (for example: start playing BBC 1) without the need for special software or present an QR code near the TV indicate media play back the relevant media centre. Although this does required direct user involvement it reduces the number of user requirements, for example that user must be capable seeing and reading small text. This can be further reduced by using easily identifiable colours and logos, for example a DVD case with the BBC logo could also contain an NFC tag. As it would be a large item would be easily found and clearly identifiable by someone with a small degree of sight loss.


  • Research Methodology

    Simon Everett01/09/2017 at 14:59 0 comments

    Introduction

    This project is split up to a number of different stages, first being data acquisition. The project uses the Raspberry Pi camera module due to the ability of three Raspberry Pis through the University of Derby for the purpose of this project. Any IP CCTV cameras would have sufficed, but through using with a standardized platform such as the Raspberry Pi improved the setup and monitoring aspects of this project.

    The test environment will consist of three televisions in three different rooms, the living room, kitchen and bedroom. The kitchen and living room are adjacent to each other and the bedroom is on the next floor. Face detection will be recorded as defined below, allowing for an infield analysis how effective the face detection is before the impletion stage as well as any anomalies that exist with the technology or cameras usage.

    Research Strategy

    Due to the optical detection methods to detect the rooms are occupied or vacant status there will be variations in the quality of the data that is collected due to changes in the light levels caused by time of day. For example, once the sun has set and user turn light on artificial lighting the main light source is generally dimmer, and in most cases mounted on the ceiling. This leads to shadows being cast over the users’ faces, affecting the accuracy of face detection. Conditions such as rain storms can also have the same effect on the accuracy of face detection as already outlined by (Belaroussi & Milgram (2012) whose research outlines the importance of lighting conditions as well as angles in faces when detecting faces.

    In order for the system to take this into account, an externally mounted CCTV camera without night vision has been used to calculate the lighting conditions in real time, allowing the system to drop into night mode if the external light levels become too dark. See image below:


    Data capture

    Every 10 seconds a photo of each room is taken using Raspberry Pi camera modules, the images ares then uploaded to the central server for processing. The image is then passed onto a Python script which uses OpenCV to process the image and identify the number of faces that exist in the image. The results are then passed onto to PHP, which then looks at the GRB values to estimate the brightness of the image. The results are stored in the database against a time stamp; some of the images are stored for debug purpose.

    From this a separate PHP service runs every 10 seconds, which is offset by 1 second to establish which rooms are occupied based on the data in the database, unless night mode is enabled where the room light levels as used to determining the occupancy status. Night mode can also be enabled by the time of day a using a PHP built in feature called ‘date_sunset’, which takes the longitude and latitude (PHP, 2015) to establish dawn and dusk times. Each media centre is then queried to see which are currently playing media and which are occupied. If any of the rooms are occupied but not playing media the television in that room is turned on and content is transferred. After a room has been unoccupied for the time out period of 5 minutes the media is stopped and the TV is turned off.

    Each room has been calibrated to take into account the strength of the artificial lighting, for example the install for this project considers the living room as a 80, the bedroom as a 90 and the kitchen as 100. For further information this data can be viewed in the PHP file, under the setting directory.

    Data Analysis

    The below graph is an example 2 hour window with 2 individuals watching a TV program under artificial light during the evening. At the end of the program the TV is left on for a while before the TV and room lights are turned off. The graph below shows wide fluctuations in light levels as well as well as fluctuations in the number of faces detected during ideal conditions where subjects are looking directly towards the camera.

    This 2 hr data set shows how clearly the...

    Read more »

  • Literature Review

    Simon Everett01/09/2017 at 11:03 1 comment

    Introduction

    In the late 1980’s Bill Gates founder of Microsoft designed and implanted an integrated media centre solution into his newly built home near Lake Washington, which used small pin badges to detect the presence of users and re-direct the media accordingly to the nearest appliance as the user moved around the home. Unfortunately there it is little literature to support how Gates’ achieved his project, however Gates (1995) published a video documenting the media centre, which is available on YouTube[ CITATION Nig07 \l 2057 ]. This project was coved in a number of publications at the time. U.S News (2012) reported that Bill Gates had completed construction of his house at cost of $100 million, and detailed that it had a number of features such as high-definition television monitors that displayed art digitally, computer controlled music, lighting and climate settings that are linked to individual users location within the mansion. Again, the article did not state how this was achieved.


    Since 1997, when the Gates home was completed, technology has significantly advanced in a number of linked technologies such as computing, televisions, radio-frequency tags, CCTV cameras as well as media centre technology. Alongside these developments many users have cited privacy concerns, relating to some detection methods, such as technologies processing images taken, tracking movement and even logging played content on the home automation platform. A further question is whether a user would want the fact that they’re out of the house recorded by others.

    Part of the literature review will focus on possible ways of reliably detecting room occupancy. This can be achieved in a number of ways, such as radio sensors which users must carry, face detection, movement or co2 sensors. This Literature review will offer a critique each approach, as well as looking at how the technologies are applied in different settings. The literature review will then go on to focus on media centre technology, possible hardware solutions and effective ways to synchronise all aspects of the project to allow the Media Centre to act as one.

    Ways of Detecting Room Occupancy:

    Radio

    Sensors:

    There are existing projects similar to the Bill Gates project (Gates, 1995) that use pins or devices to track room occupancy for the purpose of redirecting media content to the nearest appliance. Linux MCE (2015) is a whole home automation suite that uses also uses radio sensors, namely Bluetooth, as a method to track users around the home from mobile devices. There are limitations associated with Bluetooth, namely the inability to obtain signal strength between two devices around without first paring them together using standard hardware. On the other hand, there are some devices that are able to communicate to all Bluetooth devices without paring, such as Ubertooth (Chai, Deardorff and Wu, 2012) which allows the interception and transmission of Bluetooth packets from and to any mac address, as well as to detecting the single stretch of remote devices. Ossman (2015) has developed the product Ubertooth to allow users to directly access the Bluetooth protocol Hardware such as the Ubertooth One makes tracking Bluetooth without paring possible without excessively expensive hardware Hay and Harle (2009) discuss the process involved, in their paper, and conclude that connection-based tracking which permits tracking of a previously identified handset within a field of fixed base stations is a viable alternative to inquiry-based Bluetooth tracking. Although the paper is from 2009 the research is still relevant today.

    Although Bluetooth tracking could be seen as a convenient approach for tracking individuals, this would not be suitable for this project due to usability issues. Users would be required carry a phone or an object to be detected as occupying a room. Although this approach suitable for the project discussed in this paper, there are commercially available products.

    Tracking in...

    Read more »

View all 2 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates