Flexible Display and Input Device at Once: Flexpad is Made of Regular Paper

Human organs shimmer in red on a sheet of paper displaying a longitudinal view of the human abdomen. The spinal column and pelvic bones form contrasting yellow islands. As the sheet of paper is bent downwards at the ends, the bones appear to come into the foreground while the soft tissue recedes. What appears to be science fiction at first glance, is the result of the “Flexpad” research project developed under the leadership of Jürgen Steimle in the Media Lab at the Massachusetts Institute of Technology in the US and the Max Planck Institute for Informatics in Saarbrücken, in cooperation with Kiel University.

“We routinely deform objects intuitively in many different ways. We bend back pages in books, deflate balls, fold paper, and sculpt clay”, explains Jürgen Steimle. “And by projecting user interface elements onto tangible, deformable objects we can control computers and other technical devices better and more easily.”

A depth camera records the motions of hands and paper

In order for his ideas to function in the digital world, only a small amount of technology is required meanwhile - however, all the more programming and brain work. “For one thing, we use a projector that reproduces an image or a video on a sheet of paper”, says Steimle, describing the approach. “For another, we work with a Kinect sensor from Microsoft. The depth camera permits people to use their movements to control computer games. In our case, it records the hands and paper to determine their position in space.” To capture the movements of their hands and the paper, the projector and camera are mounted in the ceiling above the user.

Flexpad thereby works with two algorithms as follows: The first initially subtracts out the interference caused by the fingers and hands of the user. If the user moves the paper - whether left or right, or bends it into an arc or wavy form - the camera senses this. A specially developed computer model subsequently describes these movements in fractions of a second, so that the projector can reproduce it on the sheet in near real time.

With Flexpad, diagnosis consultations become simpler

“The paper takes on two simultaneous functions in our system”, explains Steimle. “It is input device and display at the same time.” The user can interact with the device in a similar way to using a mouse for controlling a computer. Other materials besides paper are also suitable, for example sheets of plastic or other deformable materials.

Active flexible displays, as they are referred to, go one step further. According to the study “OE-A Roadmap for Organic and Printed Electronics” from the Organic and Printed Electronics Association, these will be available for end users in just ten years. “Our concepts, which we are researching using Flexpad, can be transferred to this new type of display”, explains Steimle. However, applications based on the cost-effective technology are already conceivable. “In medical diagnostics, the physician can quickly and easily discuss for instance the results of computer tomography with the patient. In addition, the system can operate as a kind of interactive children’s book in which certain figures, like a goldfish, can move”, says Steimle. -- © Max-Planck-Gesellschaft