Home Control System (HCS)
The purpose of the Home Control System (HCS) is to monitor and control the main features of our home, to improve its efficiency and to reduce its environmental impact. An important aspect is also the user experience. The HCS is designed to overlay the existing infrastructure of our home, to be totally intuitive and not at all intrusive. This means that there is no obvious physical presence of the HCS in our home, other than some additional switches for automated curtains. Everything else, including the light switches work as you would expect.
The HCS serves three main functions:
- Security and safety monitoring.
- Efficiency of space heating/cooling, water heating and lighting.
- Monitoring external environment and weather.
The HCS is under continual development and is being tested out in our current home. It is built using open source software (mainly Java) and runs on a mini-ITX PC, which uses very little power (typically <11W). For the first six years, this machine ran the Windows 2000 operating system and performed flawlessly.
In 2010, we upgraded this machine to the Windows 7 operating system, mainly because Windows 2000 was no longer supported by Microsoft. This upgrade required us to install more memory (up to 1Gb) for Windows 7 to run with acceptable performance. The main reasons for choosing Windows 7 were the availability of drivers and our familiarity with this OS.
In November 2013, we fully migrated our HCS to Debian Linux. The HCS PC interfaces to various bits of hardware, which are described in the following sections.
The main reasons behind our approach are:
- Off the shelf, low cost, low power and reliable hardware.
- Not tied to any one operating system.
- It provides a totally flexible and extensible platform, which is not tied to any one hardware or software vendor.
- Can handle multiple automation technologies simultaneously (a hybrid technology solution).
- We have no limit to the number of sensors and devices we can connect to our system. This is important as we have many types of sensors installed in every room and on the outside of our house.
- Can be easily programmed to support complex functionality, as well as hardware groups and interactions.
- Individual automation technologies can be swapped out for new ones.
- Multiple USB or serial port interfaces are readily available and can be used to interface to any home automation technology.
The HCS itself is actually a collection of multi-threaded Java processes, using inter-process communication at the socket level. This allows us to wrap any technology an abstract away the complexities of the hardware and drivers, exposing a simple, real-time, event driven services at the core. What this means is that we can use any home automation technology and more importantly the best one for each specific task. It also means that we can combine multiple technologies and get them to work seamlessly. Right now, our HCS is using X10, 1-Wire, USB IO, Ethernet IO and we are planning several others. As technology changes we can swap old ones out and new ones in.
The HCS has been developed on the back of many years experience doing real-time systems design, working with real-time operating systems and mission critical systems, whilst working in defence and communications companies.
Our Smart Home Philosophy
In making our home 'smart', there are some guiding principles that we have adopted:
- The underlying technology is not important. What the technology does for us is.
- Just like an iceberg, most of a home automation system should be invisible. That which is exposed must be stylish and attractive.
- Smart home automation should just work as required and as expected, with minimal human interaction. The ideal is 'zero touch' home automation.
- Any interfaces exposed must be simple and intuitive. The results of any human interaction must be obvious and predictable.
- There must always be a manual over-ride or a backup and this does not have to be visible. Assume that no electronics or software is 100% reliable.
- The intelligence and control elements must be autonomous and self-sufficient with no dependency on external services (e.g. the mains power supply, cloud services, remote servers, etc.), though they can be used when available.
- Efficiency (in all its forms) is an inherent characteristic of 'smart' home automation.
- It must collect sufficient data to learn and adapt, whilst providing the required insight to enable its occupants to also learn and adapt their behaviour.
Home automation is not the same thing as smart home automation. Using technology to enable a device to be controlled by a remote switch or Smartphone app is automation. Providing the learning and intelligence to ensure that the switch or app doesn't need to be used is smart home automation.
Smart Home Model
In order to implement our plans, we have used Java to create a smart home model.
'Whole House' (or Whole Home) Context
A core principal behind the design of our smart home is something we call 'whole house' or 'whole home' context. This is based upon the premise that our smart home controller (HCS) has knowledge of anything and everything that happens within it, around it and visibility of any external factors or data that may influence its behaviour. This means that it has ALL of the information required to make the BEST decision regarding any aspect requiring control or automation. This is something that you simply don't get by using a collection of 'off the shelf' devices and services and then try to link them all together. It also means that all of this highly valuable information stays within our smart home, allowing us to keep control of our privacy.
The lack of whole home context is one reason why companies like Google, Apple and Samsung will struggle to provide a smart home with anything like the power of our current self-build solution.
Our Home Control System (HCS) is a 'hybrid technology' implementation, that uses a mix of wired and wireless home automation technologies. This means we are free to choose and use the most appropriate technology for each feature being implemented, within a single and coherent system. This is enabled by our use of technology abstraction.
This means we can seamlessly integrate multiple technologies into a single coherent system. Our HCS is currently using hard wired sensors, USB I/O, Ethernet I/O, Bluetooth, Z-Wave, 1-Wire, etc. We can add new technologies and we have remove older ones, such as X10 in the past.
We have adopted a design methodology based around 'technology abstraction'. This means that all user interfaces, software and hardware references the devices and their state in a simple, human readable form. All inter-process messaging is also done this way. Put simply, we are speaking the same language as our house.
This also means that there is no technology specific addressing of devices and older technologies can be replaced with new ones, without having to change the configuration or implementation. As an example the PIR sensor in our kitchen is referenced as 'Kitchen PIR' and has a state of 'On' or 'Off'. It is currently a wired sensor linked to a USB I/O board but we could replace it with a Z-Wave sensor and not have to change any code or configuration files.
Main Components & Features
This is now described in more detail here.
Our HCS has a networked calendar, in order to store events and context. The calendar is also another way to interact directly with the HCS. The calendar stores structured events that are machine and human readable. The calendar is typically exposed and interacted with via the Smart Home Assistant.
Devices are components that can be controlled both manually and/or using automation. This includes things like Z-Wave appliance and lighting modules, bathroom fans, etc.
As previously mentioned, our HCS is written in Java and models our home, its context and the rooms and main objects within them. The house itself has five modes of operation that are manually set and stored in the database:
- In - Someone is at home.
- Auto - The status is determined by a programmed schedule, which learns and is augmented by detected presence.
- Out - Nobody is at home, e.g. at work, school, etc.
- Away - Nobody is at home and won't be for some time, e.g. we are on holiday.
- Test - A test mode where the status is set independently.
In addition to these manually defined states (which take priority), there is also a Java class which actually models the more dynamic (automated) aspects and scheduled values. This then determines the status actually used within our home automation system and thus allows our house to be more intelligent and also learn from the presence of people detected inside it.
The status is the actual 'working value' that the HCS is using based on the mode set. It has one of three values and these are essentially a subset of the 'Mode' selected. When the 'Mode' is set to 'Auto' the 'Status' value is set based on the HCS understanding and learning:
- In - Someone is at home.
- Out - Nobody is at home, e.g. at work, school, etc.
- Away - Nobody is at home and won't be for some time, e.g. on holiday.
There is also a Java class which actually models the more dynamic (automated) aspects and scheduled values. This then determines the status actually used within our home automation system and thus allows our house to be more intelligent and also learn from the presence of people detected inside it.
Lighting control via the existing wall switches works as expected. Lighting in each room can also be based upon detected and/or programmed occupancy. Room lights can be configured to come on automatically at night, when occupancy is detected and the brightness can also be controlled. Lights can also be switched off automatically if there is no programmed or detected occupancy. These use profiles to achieve this. There are many other types of lighting in our home though.
Scenes are covered on a separate page.
Curtains & Blinds
As part of the security system, automated curtains will be closed when dusk is detected. To avoid large load spikes, each pair of curtains will be closed sequentially.
Notifications are something we can turn on and off and enables alerts to be sent for events of importance.
Security & Alarms
Our HCS also monitors security of the house using PIR and other sensors in each zone.
Security Images & Video
The HCS will capture, store and share still images and video clips from cameras.
Sensors are components in our home that generate events on change of detected value. This includes:
- Contact - Sensors on windows and doors have a value of Open or Closed.
- Humidity sensors
- PIR - Passive Infra-Red (PIR) movement sensors have a value of On or Off.
- Temperature sensors
All of the sensors in our home are modelled in XML, as are the zones and our whole Home Control System (HCS). The XML supports the setting of alarms for each sensor based on things like excessive temperatures detected or even doors being left open for too long.
Those sensors that are battery powered will also generate alarms when their battery level drops below a preset value.
Services Usage Monitoring
The HCS will monitor and record usage of electricity, water, etc. It also monitors availability of power, Internet network connections, etc.
Smart Home Assistant
The Smart Home Assistant provides the ability to query and command the HCS using voice and text interfaces.
Space Heating & Cooling
The combination of the house and room context leads to each room having two target temperatures based upon occupancy or lack of occupancy. Control is only possible on a per room basis, if each individual room has a temperature sensor. These sensors are also used to detect fires and an upper limit is set to signal alarm points.
Traditionally, water heating and space heating systems are linked, often by the same boiler. Whilst this is a cheap solution it is often very inefficient as there is little connecting between the two in terms use throughout the seasons of a year and even the time of day. Our HCS models water heating as a separate and isolated capability and also supports the concept of multiple water heaters and hot water tanks. It assumes that there may be many heat sources available such as solar water heaters, ground source heat pumps and electric heaters.
The user interfaces are covered on a separate page.
We have defined a number of zones recognised by our home control system. Each zone also has a 'parent' zone. When a zone checks its occupancy and presence state, it also takes into account the 'child' zones that sit within it. Many zones have a parent zone of 'Upstairs' or 'Downstairs' and these two zones have a parent of 'House' zone. The zones currently defined are:
- Dining Room
- Ensuite bathrooms
- Entrance Hall
- Outside - Things that have an effect across all zones, e.g. Air Pressure
- HCS - Devices and sensors that are part of the HCS and are not room specific.
Each zone (room) also a number of attributes:
- A temperature sensor (some have more than one).
- Occupied (true or false) - depends on whether someone is detected in the room or not.
- Persistence - Detected occupancy also has persistence, configurable for each zone. This is basically the delay between last detected activity and the zone thinking it is now unoccupied.
- Usage - A flag which stores whether the room is in use or not. This enables detected occupancy and profiles to be ignored.
The HCS architected to be extensible to a wide range of existing technologies and new ones as they emerge on the market. This approach also enables us to test out new technologies, to assess their benefits and performance.
Each technology is integrated into the overall HCS design by using a real-time Java process to 'wrap' it and then expose it using socket layer communications.
The HCS is a number of processes that perform real-time communications using inter-process messaging. These structured messages or events are driven by external sensors and sensors networks, in-built clocks and timers and hardware controllers:
This is the core process with the main intelligence and logic. It is event driven and waits for messages to arrive from the other processes. It also creates new Java threads for certain functions.
All of the processes log debug information and key events.
As well as dynamic event driven messages, a clock thread sends regular 'heartbeat' messages/events to the HCS process.
This process polls the Dallas 1-Wire sensors on the network and reports any changes back the HCS process, in structured messages.
This process used to managed the PC to X10 interface and accept actionable messages from the HCS process. There was no reporting back to the HCS process because the X10 protocol does not really support the monitoring and status polling of X10 modules. This has been removed from our system now and replaced with Z-Wave technology.
This process manages the USB I/O board, which appears to the HCS mini-ITX PC as another serial port. This board has both input and output capability, so the HCS can send commands to this process as well as receive events from it.
This process manages the Ethernet I/O board, which uses a bespoke UDP protocol over the Ethernet network.
As described above, the HCS is a number of processes that perform real-time communications using inter-process messaging. These structured messages or events are driven by external sensors and sensors networks, in-built clocks and timers and hardware controllers. Initially, the messages were sent as serialized objects. To facilitate a number of web apps (run on iPhone and iPod Touch) via Apache web server and PHP and to allow these to communicate directly with the lower level processes, the messages were simplified to simple text strings. The event types currently used are:
- Alarm - device alarm, e.g. Dallas 1-Wire DS1920 sensor.
- Battery - battery level/capacity.
- Clock - a heart beat message
- Command - user command to be processed.
- Contact - Contact sensor
- Environment - external environment message, e.g. sunset, sunrise, etc.
- Humidity - humidity sensor
- Binary - binary input sensor
- Presence - presence update
- Query - query request for a value
- Report - process is logging a report, e.g. devices current seen on home network
- Scene - request to run a scene
- Temperature - temperature sensor
- Test - test event
- Voltage - voltage measurement
The message structure is basically a string of the form:
In a typical day, the HCS will handle over 100,000 real-time events.
A database is used maintain persistence of states, enable event logging and to enable status sharing and distribution.