The Software Only Smart Home

The idea of a "software only smart home" came about whilst testing our current Home Control System (HCS). To make it easier to test new ideas and features, we run another instance of our smart home software on a Raspberry Pi in 'debug mode'. This means it doesn't try to interact with any of the 200+ objects in our current smart home, but all the other functionality works as expected. We quickly realised that even without the hardware interaction our smart home is very powerful and does many useful things, gathering enough information and context from our home network and Internet services to enable it to make intelligent decisions and provide useful services.

You need a piece of hardware on which to run the smart home software, but this could be a very compact, cheap and low-cost processor. Such a processor probably already exists in your home, in the form of a TV Set-Top box (STB), Network Attached Storage (NAS) device, Apple TV, etc. We have used a Raspberry Pi because they are cheap, extensible and easy to work with. In normal operation our smart home software (written in Java) uses just a few percent of the CPU's capability, yet it can handle millions of events per day and process each one in a few milliseconds.

We are NOT proposing that run a software only smart home and never add any hardware in the form of sensors and devices! What we are saying though is that in the modern home, there are many networked devices (games consoles, Smartphones, tablets, etc.) that can be used to passively provide information and context to our smart home software. There are also many cloud services, Internet data feeds and local (3rd party) IoT devices that expose useful data and insight. In most homes, this is enough to enable advanced functionality and a lot of clever home automation. In this form, the 'software only smart home' is a valuable service in its own right.

Some of the key reasons why our 'software only' approach works are:

  • Our smart home software models the whole home in terms zones/rooms and the relationships between them. This is achieved using an incredible simple JSON configuration file that can be produced for any home in a few minutes, quicker using a graphical tool or web form. It models people, objects (sensors and devices), object types and their characteristics. It also models things like occupancy, presence, proximity and location.
  • Our smart home software uses technology abstraction, meaning it will pretty much work with any sensor or device. This is done in way that is both machine and human readable and this means that we all speak the same language. This is key to enabling the artificial intelligence (AI) and the associated natural language interaction.
  • Our technology abstraction also supports a hybrid technology approach, meaning we can use a wide range of wired and wireless technologies. This makes our smart home future proof and very cheap to deploy in new house builds. It can also be retro-fitted to existing homes easily, using the best technology for each specific installation challenge.
  • Our smart home software works on the basis that it has 'whole home context' and "sees" everything that happens in our home, allowing it to make more intelligent decisions.
  • We have adopted a lego brick approach to adding new hardware, meaning we can quickly and cheaply extend our smart home. This also makes it very flexible to meeting changing user needs.

So before we add any smart home sensors and devices to our home, our smart home software is providing the framework to enable many clever features and services. When we add a single sensor, it will add to the overall 'whole home context' and will adopt all of our smart home's features and capabilities, instantly making that one 'dumb' sensor much more valuable and reusable across all services.

Example

Let us consider a simple temperature sensor by way of an example. Our smart home software models this type of sensor and its characteristics. This includes an upper and lower temperature limits, so an alarm or alert can be configured if these limits are breached. The temperature sensor now has the ability to be used for fire detection. Similarly, the rate at which the temperature is rising is also modelled and can also be used for very early detection of fires, often before smoke is present.

The sensor adopts all of the common capabilities of our smart home software and this means it can generate SMS messages and make voice announcements. It can be used to control any device and queried via all of the supported interfaces, including our artificial intelligence (AI).

Our smart home software can also use any temperature sensor as part of a 'thermostat' to control your heating, at a zone level or on a whole home (single zone) basis. The software thermostat inherently models more features than any of the hardware connected thermostats available, so you don't need to buy expensive devices like Nest or Hive. Each adopts all the capabilities inherent in our smart home software, including voice announcements, remote control, sending SMS alerts, ability to trigger the alarm, etc. If you need a second, third or even fourth 'smart' thermostat (for multi-zone heating control), you simply configure another software instance at zero cost.

All these software features also inherit all of the 'whole home context' including our smart home's ability to automatically know if you are in, out or away on holiday. Our smart home knows if it is occupied, who is at home, where people are and can predict when they are likely to be home. It also supports schedules in software to allow dynamic, flexible and learning heating and hot water control. It is easy to personalise the whole experience, setting temperatures based on who is in the house or each room.

In fact any device added to our smart home can have an associated software controllers which can use the temperature sensor as part of an adaptive, scheduled or context aware control process. By adding it to the smart home, it instantly becomes available to every other sensor and device in our smart home.

Another inherent capability of our smart home software is the query and control every aspect of it via numerous interfaces. Our software also models the trusted people and their permissions, to limit what is exposed and possible via these various interfaces. This includes natural language interaction by text or voice and our software enabled artificial intelligence (AI), which enables temperature sensors to be queried by voice. This also includes all of their associated attributes, e.g. last updated, last changed, daily highest temperature, etc.

What our smart home is doing is taking a cheap, dumb sensors and making them incredibly smart using software alone.

Each sensor and device inherits the wider set of capabilities enabled by our smart home software and instantly becomes much more useful and valuable.

What Can A 'Software Only Smart Home' Do For You?

Customer Experience

The 'software only smart home' approach was developed to enable the best user experience for a diverse set of users. It does this by making no assumptions about the interfaces used to expose services and features and by supporting many different and varied types of interfaces. There is no one interface that meets all user needs and the best interface for each specific task will vary depending on the user's circumstances.

A simple example might be that voice control of a lamp is easiest in a quiet room but if the room is busy and noisy, then another method of interaction would probably be better. This is why our approach enables many types of interfaces to achieve the same goal. This includes voice control, Smartphone and tablet apps, physical switches, touch screen controls, natural language interaction via text and remote controls.

Where there are existing controls (e.g. light switches), they function as expected. Any new methods of control must be intuitive and accessible to all. Our approach has a focus on autonomous operation though, with a 'zero touch' user experience desirable where this makes sense. Our approach is also based upon having all the information and context to make intelligent decisions.

Because the service is not tied to any particular piece of hardware, networking technology or protocols, the whole design and architecture can be customer experience led and not driven by technology and its associated limitations.

Instant Reward

The 'software only smart home' delivers instant rewards in return for minimal investment (in terms of time, money and smart home hardware). It can be deployed very quickly with minimal setup and configuration, delivering useful services from day one. The inherent extensibility of the approach also means that new services and features can be delivered into the market very quickly.

Insight

The 'software only smart home' allows the home owner to clearly see how they are using their home and how well it is working for the occupants. It provides real insight as to how best extend and improve it. This is done through in-built performance monitoring and reporting.

Privacy

Our software only smart home approach gives the home owner complete control and ownership over ALL of the smart home data generated. A lot of the data gathered can be very personal and used to provide very personal insights. From the very start of our experimentation in our smart home, we decided that we wanted to retain ownership and control of this data and that we didn't want it to leave our home. Some might argue that we miss out on the shared learning and insights but, we haven't seen any justification to support this view or anything that justifies this loss of privacy.

Our approach does allow remote access and support if required, but with the home owner in control.

Smart Home Models

In software alone, our smart home models all of the key smart home elements:

Zones (Rooms)

Our smart home models the house layout in terms of floors, room/zones and the relationship between them. It also models things like our Home Control System (HCS), outside, the garage, the shed, back garden and driveway. This is easily done in software and is configurable to any home with some very simple (JSON) configuration files. Typically a simple graphical tool or web form would be used to make the configuration process very simple and quick to do.

Our Java software has a class for zones and these model the characteristics of the room or zone, inheriting all of the attributes and capabilities. This includes things like occupancy and presence (which is covered in more detail below). The model includes relationships between rooms, such as doors and arches and so our smart home knows things like whether zones are open or closed. Even if you have no door sensors or PIR sensors, you still have a useful model for storing context.

Occupancy, Presence, Proximity & Location

These terms are defined here. A software only approach uses existing personal devices and shared devices in the home. The software spots them joining and leaving the home network and infers occupancy (shared device or unknown owner) and presence (personal device with known owner). The modelling of the device comes into play here as wireless, battery powered devices don't simply join a network when in range and stay connected. The software model incorporates a clever algorithm for each device type, to enable optimal understanding of its presence or absence on the home network.

Proximity to the home is useful context for a wide range of services. When combined with occupancy and presence it can be used to automatically work out if home owner is in, out, or away on holiday. No clunky user experience, where you have to set this on one (or more) apps!

The smart home status (in, out or away) is a key enabler to adaptive behaviour that can be applied to all smart home services and products. It enables devices to power save intelligently, come out of standby modes, enables convenience lighting, personalisation and many other useful features. It is also possible to count the number of people in the home.

People / Users

Our smart home models known people and unknown people. For known and trusted people it models their location, personal devices, etc. It also knows about communications methods and preferences, to enable trusted and personalised communications channels (SMS, Twitter, Google Hangouts, etc.)

Another key part of the smart home user model is enabling trust relationships and permissions.

Devices

Our smart home software models devices. The model includes types of devices, their network interfaces and MAC address (LAN, Wi-Fi, Bluetooth, etc.), characteristics and their owner.

Simply by tracking devices appearing and disappearing on the home network, it is possible to infer activity, occupancy and presence of people. In addition, it is possible to model proximity and location in software alone as well.

Alerts & Notifications

Some definitions:

Alarm
A local audible or visual warning requiring urgent action. Typically this is implemented in hardware with a sounder, strobe or warning light but it could be done in software using push notifications to mobile apps. It usually results in an alert also being sent.
Alert
A remote warning requiring action. Typically these are delivered in near real-time via IP-based comms, SMS, etc.
Notification
Useful information, with no immediate action required. Examples include: Mail delivered or Letterbox stuck open. The Parcel Store was opened. The Garage Door was left open.

Cloud services (software) can be used to enable SMS alerts and notifications. This is a re-usable common capability across all smart home services.

Personalisation

The previously mention software model for users in the smart home, enables rich personalisation across all smart home services. This can include generic and service specific preferences. In our smart home we generically model explicit personal interests to enable content recommendation and a personalised TV guide.

Communications

Our Smart Home is a communications end-point in its own right and this is all enabled in software. This includes XMPP, Twitter, email, SMS and local voice interaction. We are currently working on IP voice interaction. This is all implemented in software, as is our natural language interaction artificial intelligence (AI). This allows us to query and control all aspects of our smart home by voice or text.

Essentially, our software gives our smart home an identity, so it becomes a 'contact' and something you can have a conversation with, just like any other person in your contacts. This works both locally and remotely.

We see the smart home as an intelligent assistant and one that could perform call screening and intelligent call routing, from within your home. Only the smart home has the location information and context to do this well.

Voice Announcements

Whole home voice announcements can be delivered in software and can use text-to-speech (TTS) synthesis and/or pre-recorded voice segments. As a centralised, core capability of the smart home it has the benefits if providing a consistent and recognisable voice throughout the home, which can be used for all services. This type of interface is less intimidating and much more friendly than wall mounted displays and many other types of devices used to convey basic information. It removes the need for the complex wiring required to install display panels as well.

In our smart home, voice announcements can be intelligently delivered to individual rooms, zones or to the whole home. Voice announcements can even be personalised based on presence.

Voice announcements require some hardware with amplifier and speakers to deliver them but in many cases such hardware already exists in the home and can be re-used.

Local Data Feeds

There are many free, localised Internet data sources that provide useful smart home information to improve context and power alerts, notifications and voice announcements. These can be easily accessed using software to get information on things like local weather conditions and weather forecasts, to drive wind, ice and rain alerts. Pollution levels and pollen counts are also useful for asthma sufferers. High UV levels can be used to initiate reminders to wear sunscreen.

Predicted or forecast temperatures can be used to change behaviour of many services within the smart home, including things like the central heating.

Local traffic information and news briefings are other examples of useful information that can be used pre-emptively.

All of this data is reusable across all services and simply requires software to collect it, store and add it the wider pool of context used by the smart home.

Sunrise & Sunset

Sunrise and sunset data is freely available from various online services for any UK location. It can used to automate smart home services and features and to dynamically change behaviour and activate lights. When combined with hardware that can be smart home controlled, it enables simple, zero-touch and convenience lighting.

Internet Connectivity & Security

It is very easy for a networked processor to use simple software to monitor your Internet connection and its availability and performance. Our smart home uses software to track devices connecting and leaving our home network and will detect unrecognised or unauthorised devices, raising sending an alert if this happens.

Our smart home also uses this technique to monitor and track device usage, such as our children's game consoles and it will send and alert if excessive usage is detected.

Cloud Storage

Cloud storage provides an extremely simple way to access smart home data and generated content from any location and in a secure manner. It is particularly powerful when you are away on holiday in a foreign country with expensive Internet connectivity. We use it to distribute smart home logs, store captured voice messages, still images and video from IP security cameras and to share content.

Logs

Our smart home software logs all significant events and activity to disk. This can be extremely useful to customers in those rare circumstances where things go wrong and provides useful information to diagnose problems and also help debug the software itself. It provides an audit trail of activity and this can easily be presented to users in a clear and simple form. Because we use technology abstraction the logs are both machine and human readable.

There is a lot of power in looking at smart home logs, to see what happened or went wrong. The sequence of events is particularly useful. This is not something that needs to be exposed to all users or very often though. Because the smart Home 'sees' and captures all activity, it provides a very complete picture. These logs can be shared these with trusted services and people and our smart home software facilitates this.

Software Controllers

Software controllers provide the rules and intelligence for objects (devices and sensors) modelled in our smart home. They are a very simple way to enable powerful and intelligent control and are very easy to use and configure. As a central control point for each device, it eliminates the conflicts that occur when many sensors are providing control triggers as inputs.

This is all done in software, with a Java class instance being created for each object to be controlled. The controllers also support the concept of adaptive timers and schedules.

Calendars, Timers & Schedules

Our smart home uses a common calendar to schedule behaviour, device states, etc. It can handle things like holidays and be used to control things like the central heating, scheduled services. This includes things like making the home look occupied when no one is home.

Simply Add Simple Hardware

If you start adding connected devices and sensors to the software only smart home, many more things are possible. But all of the software capabilities previously described provide an framework within which to add, configure and use them, immediately adding value to any investment made in hardware. This means that you don;t need to dive in to complex smart home technologies and mesh networked devices from the outset.

In this section we will look at the hardware options and networking technologies that can be used to easily extend the capabilities of our 'software only' approach.

Coming soon ...

Share ...
We are on ...
Facebook Twitter
YouTube Flickr Follow us on Pinterest