With Smarter Objects, a virtual object is associated with every physical object to support an easy means of modifying the interface and the behavior of that physical object as well as its interactions with other "smarter objects".


Smarter Objects focuses on the physical operation of everyday objects. The virtual interface defines the relation and setup. The smart phone or tablet used for the virtual interaction is a powerful tool for editing the physical object’s functionality.


As a user points a smart phone or tablet at a physical object, an augmented reality (AR) application called Reality Editor recognizes the object and offers an intuitive graphical interface to program the object's behavior and interactions with other objects. Once reprogrammed, the Smarter Object can then be operated with its simple, tangible interface. As such, Smarter Objects combine the adaptability of digital objects with the simple tangible interface of a physical object.


Smarter Objects are also simple to create. Since all the virtual interfaces are based on web standards and open source communities, common design work flows can be used to generate the interactions. In addition to the web standards, the Smarter Objects platform offers a creator tool that visualizes the interactions in real-time and publishes the virtual interface into the Smarter Object with one mouse click.

Smarter Objects is a new paradigm for interaction with everyday smart objects. The paradigm combines optimized physical interaction with well-designed objects with the benefits of a flexible and detailed virtual user experience.













MIT Media Lab

Many thanks to an amazing team that help to form Smarter Objects:

James Hobin, Benjamin Reynolds, Anissa Talantikite, Shunichi Kasahara, Kevin Wong


Imagine, for example, that you have 20 smart lights, 5 radios, a refrigerator, 10 doors, a car and maybe even sensors on your body. Today all these objects have to be controlled from different apps on your smart phone


It becomes very complicated to keep in mind all the apps and names in pull down menus when all you want to do is change the color of your kitchen light.


Interaction with physical objects is most satisfying when the interaction that is performed directly maps to the action that is following*. It is important that the virtual and tangible user interfaces connected with an object are directly mapped in order to remove unnecessary levels of abstraction. Since our world is three-dimensional, such a mapping must be performed in this three-dimensional space.


Mental load is minimized when a user has a direct view of the object of interest and can manipulate it on the spot, or when the user can make changes onto an impage of the object in question  in its surroundings. This is what we refer to as a direct mapping of operation.

With a Smarter Object, one can simply interact with a virtual interface directly mapped onto the object that needs to be controlled so that no mental relation between object and virtual interface must be memorized.


A user can easily link two Smarter Objects to form a new functionality.

Simply draw a virtual line from one interface element to another element to generate a new control relation.

Smarter Objects share the best properties of two worlds, the easy interaction of physical objects and the flexibility of virtual objects.


All interfaces found on physical objects can be broken down into value sliders and on/off actions. For example, a radio has a volume knob, a tuning knob and the volume of its speaker. Each of these three components can be represented as a value slider. Combined in a meaningful way, they form the functionality of a radio. Combined with a standard light, the knobs of a radio could become the controller for the light hue.


While Smarter Objects are broken down into their simplest components, they can be combined with one another in any meaningful way a user wants to experience their functionality.


The Smarter Objects approach can be adapted to any object in our physical world from a washing machine to a car’s air conditioning system; from the home entertainment system to the assembly line that produces the computer in front of you.


Links can also be formed using virtual components that can scale, switch, delay or invert the linked signals.

For example, a Lego Mindstorms robot with two motors and two distance sensors placed on the left and right side of the robot can be programmed in an intuitive way. The distance sensors are connected to a virtual inverter element. The signal then goes into the motor on the opposite side. When the robot drives close to a wall, the motor of the opposite side stops, so that the robot starts to move in a curve away from the wall.


Smarter Objects are very simple to create using a Smarter Objects open source library for Arduino. One needs only a few lines of Arduino code to create a fully working Smarter Object. Once it is created, it is compatible with any other Smarter Object.


Smarter Objects come with an open source reference design using ARM powered microprocessors. Smarter Objects use WIFI and Web-Standards like HTTP and JSON to communicate with one another.


It is important to note that communication between the objects is peer to peer and decentralized. A connection to the Internet is only established if demanded by the user. For instance, the user might wish to connect the activation of the car AC with the use of the coffee machine, and therefore choose to use an Internet connection.


The Reality Editor is the editing and browsing software for Smarter Objects. It uses iOS built in 3D acceleration capabilities for HTML 5 content. The Reality Editor provides the virtual interaction with the Smarter Objects using Qualcomm's Vuforia platform to recognize the objects and web standards such as HTTP and JSON to communicate with the Smarter Objects around the user.


The Reality Editor acts as a mediator between the objects, whereby the user uses the editor to tell the object how they relate to each other. Once the new links have been established, the device with Reality Editor no longer has to be be present for the new connection to take place.


The Smarter Objects Creator is a tool to support the web design and industrial design work flow used for creating Smarter Objects. It combines and visualizes the HTML 5 designed interfaces, the Arduino code, and allows the user to place the interface and the programming tags onto the Smarter Object. All of these interactions occur in real time augmented onto the object so that an intuitive work flow is guaranteed. With a simple button click, the interfaces get send to the Smarter Object and can be used and linked to any other Smarter Object.

  © 2014 Valentin Heun, MIT Media Lab