Galileo in hell: a scientist's imaginary trip

Studio Azzurro and the Nurnberg Open Haus Ballet will make use of technologies developed within the CALLAS project to enhance their latest performance, "Galileo in hell", which will take place at the Arcimboldi Theatre of Milan on the 10th, 11th and 12th of July.

The CALLAS technology will be used to allow a digital scenography installation to react to the public's emotions, which will be assessed by analysing voices and sounds coming from the arena.

Studio Azzurro is directly involved in the CALLAS project, funded by the European Commission and led by Engineering Ingegneria Informatica.

Speechless Conversations

A new device translates your thoughts into speech so that you can have a cell-phone conversation without uttering a word.

Ambient Corporation, a company based in Champaign, IL, that develops communications technologies for people with speaking disabilities, is calling its latest system "voiceless communication" with good reason. The company has engineered a neckband that translates a wearer's thoughts into speech so that, without saying a word, he or she can have a cell-phone conversation or query search engines in public.


World's First, Live Voiceless Phone Call Made at TIDC 2008
Speaker: Michael Callahan, CEO and Co-Founder, Ambient Corporation Ambient Corporation demonstrates silent phone communication using TI's ultra-low power MSP430 microcontroller technology.

Don't fret: the device, called Audeo, can't read minds, so it won't capture your secret thoughts. It picks up the neurological signals from the brain that are being sent to the vocal cords--a person must specifically think about voicing words--and then wirelessly transmits them to a computer, which translates them into synthesized speech. At the moment, the device has a limited vocabulary: 150 words and phrases.

The video below shows Michael Callahan, a cofounder of Ambient and a developer of the device, demonstrating the technology at the Texas Instruments Developers Conference, which was held in Dallas from March 3 through 5. In his speech, he says that by the end of the year, the device will be ready for use by people with Lou Gehrig's disease, a progressive neurodegenerative disease that can cause sufferers to become completely paralyzed. He also says that in the future, if a person is walking down the street thinking about where a bus station is located, the device will automatically wirelessly query a search engine to find one.

[Text pasted from MIT's Technology Review]

Human-Computer Interaction Redefines Science

University of Maryland's Ben Shneiderman, one of the world's leading researchers and innovators in human-computer interaction, says it's time for the laboratory research that has defined science for the last 400 years to make room for a revolutionary new method of scientific discovery.

He calls it Science 2.0., and it combines the hypothesis based inquiry of laboratory science with the methods of social science research to understand and improve the use of new human networks made possible by today's digital connectivity. Through Science 2.0, the societal potential of such networks can be realized for applications ranging from homeland security to medical care to the environment.

Shneiderman points to the effect that the World Wide Web and cell phones have had on building human collaborations and influencing society. "eBay, Amazon, Netflix have already reshaped consumer markets. Web-based political participation and citizen journalism are beginning to change civil society. Online patient-centered medical information has improved health care. MySpace and Facebook encourage casual social networks, but they may soon play more serious roles in emergency disaster response, for instance.

"It's time for researchers in science to take network collaboration like this to the next phase and reap the potential intellectual and societal payoffs. We need to understand the principles that are at work in these systems," said Shneiderman.


A "Google TechTalk" about HCI by Allison Druin, Director of the Human Computer Interaction Lab @ University of Maryland

Francis Bacon vs. Science 2.0

Four hundred years ago, Francis Bacon promoted the research strategy that has ruled scientific quests ever since, what Shneiderman calls Science 1.0. As Shneiderman describes it, Science 1.0 is "reductionist thinking closely linked to controlled experiments," a method that, while successful in explaining natural phenomena "sometimes diverges from solving practical problems and only occasionally advancing broader goals."

"Science 2.0 is about studying design of rapidly changing socio-technical systems. These studies are not replicable in a lab," said Shneiderman. "You have to study social interactions in the real world. Traditional social scientists have tried to understand these systems by data collection, but more effective Science 2.0 research involves design interventions to rapidly improve e-commerce, online communities, healthcare delivery, and disaster response.

"Science 1.0 remains vital, but this ambitious vision of Science 2.0 will require a shift in priorities to combine computer science with social science sensitivity. It will affect research funding, educational practices and evaluation of research outcomes," Shneiderman says.

911.gov

Shneiderman and a number of colleagues at the University of Maryland are already on the frontier of applying Science 2.0 methods to the computer-based human networks that Shneiderman calls socio-technical systems. Here are a few new intriguing lines of research.

• Disaster and emergency response -- Shneiderman, Jennifer Preece and several other colleagues are developing 911.gov Community Response Grid, an emergency response system that would rely on the Internet and mobile communication devices to allow citizens to receive and submit information about significant homeland security community problems.

• Why do we trust MySpace? - Jennifer Golbeck is using Science 2.0 methods to understand how people come to trust technical communication networks, something that can't be studied in a laboratory, Shneiderman says. Her results can be applied to many applications of social networking including medical care, voting and homeland security.

• Why We Respond - Philip Wu looks at motivation for participating in community response through information and communication technologies, and studying average citizens' information needs and behavior when they prepare for, respond to, and recover from large-scale emergencies and disasters.
  

Text pasted from http://www.sciencedaily.com/releases/2008/03/080306170924.htm

Connecting Your Brain to the Game

Using an EEG cap, a startup hopes to change the way people interact with video games.

Emotiv Systems, an electronic-game company from San Francisco, wants people to play with the power of the mind. Starting tomorrow, video-game makers will be able to buy Emotiv's electro-encephalograph (EEG) caps and software developer's tool kits so that they can build games that use the electrical signals from a player's brain to control the on-screen action.

Emotiv's system has three different applications. One is designed to sense facial expressions such as winks, grimaces, and smiles and transfer them, in real time, to an avatar. This could be useful in virtual-world games, such as Second Life, in which it takes a fair amount of training to learn how to express emotions and actions through a keyboard. Another application detects two emotional states, such as excitement and calm. Emotiv's chief product officer, Randy Breen, says that these unconscious cues could be used to modify a game's soundtrack or to affect the way that virtual characters interact with a player. The third set of software can detect a handful of conscious intentions that can be used to push, pull, rotate, and lift objects in a virtual world.


A demonstration with Randy Breen, CPO @ Emotiv Systems

The notion of using brain activity to interact with computers isn't new. A number of schools--such as the University of Minnesota; University of California, San Diego; and Purdue--have research labs devoted to decoding thoughts from the brain and manipulating cursors on a screen, which is especially useful for disabled people. In addition, companies have cropped up in the past couple of years claiming to offer an effective brain-computer interface for video games or for biofeedback purposes. For instance, S.M.A.R.T. BrainGames, a company based in San Marcos, CA, sells games and EEG caps designed to treat people with attention deficit/hyperactivity disorder.

To use Emotiv's system, a person puts on the EEG cap and adjusts it to her head, making sure that most of the sensors touch the scalp. The system automatically picks up blinks and emotional states. However, in order to move virtual objects, such as a box on a computer screen, a person must go through a series of training sessions in which she concentrates for about 10 seconds on mentally moving the box. Tan Le, one of Emotiv's cofounders, says that there is a large amount of machine learning built into the software, so the more a person concentrates on a specific task, the more precisely the system follows the mental instructions.

Since Emotiv's technology is currently patent pending, the company will not disclose the details of its system. However, Le claims that the company's heads of research--optics expert Allan Snyder and former Bell Labs chip engineer Neil Weste--have made a number of scientific discoveries that are worthy of academic research papers. But so far, none have been published, and no game manufacturer has publicly committed to using the technology.

[text pasted from MIT's Technology Review]

iBar: from research to chill out

iBar is a system for the interactive design of bar-counters.


The iBar

Integrated video-projectors can project any content on the milky bar-surface. The intelligent tracking system of iBar detects all objects touching the surface. This input is used to let the projected content interact dynamically with the movements on the counter. Objects can be illuminated at their position or virtual objects can be "touched" with the fingers.

Every glass, cup, cellphone, car key, businesscard or even fingers will be recognized.

Transactions are made faster, and people might have one more reason to go drinking...

The iBar is manufactred by the London based Mindstorm Ltd.

Mixed reality and interactive storytelling: MXR wIzQubes

The MXR Corporation, based in Singapore, is marketing wIzQubes: a toy which exploits mixed reality technology to allow kids to have 3D interactive storytelling experience. It looks like a RPG game in physical reality.


A wIzQubes demo found on youtube

wIzQubes enables users to enjoy virtual stories from a special 3-D angle. The Magic Cubes leverage off Mixed or Augmented Reality technology to bring stories to life. Users access 3D virtual reality and interact with story characters to dive into the world of mixed reality.

By allowing children to actively participate in the storytelling and directly engaging with their most favorite story characters, children can learn and play at the same time. wIzQubes are designed to capture children imaginations, are easy to handle and fun to play with.

It does not take long to get started. Users take two plastic cubes with images on them. A web camera then capture s the images on the cube and a software calls up the animated story already stored in a software CD to let the users view and interact with the story.

The unique advantage of the product is that users not just view and visualize the animated story in an all rounded 3-D format, but also interact and play an active part in the story, through the interactive software component.

[Text pasted from MXR's website]

iO: natural interaction made in Italy

The iO Agency, based in Milan, is about to release the tabulaTouch technology, developed by Stefano Baraldi, an Italian researcher in the fields of human-machine interaction, new media, knowledge management, software design and concept mapping.

The tabulaTouch technology will be branded as SensitiveTable, and will offer a line of templates and custom developments on it.


The SensitiveTable

The iO Agency also markets other natural interaction technologies such as the SensitiveWall and the SensitiveFloor.

The SensitiveWall was booked by fiat for the presentation of the new 500. At the FIAT 500 presentation, users could create their own 500 on a classic touch screen interface and then all creations were sent and displayed on a sensitive wall were people could see everybody's customisations. Some of the cars were sold on presentation day through this mechanism.


SensitiveWall - towers


SensitiveWall - bubbles


SensitiveWall - landscape

Digital Theatre: interactive installation 'Euclide' in San Paolo, Brasil

In the context of the San Paolo workshop in Brasil, Studio Azzurro has introduced for the first time technologies developed by CALLAS. The new scenario was focused on the dialogue between video, physical environment and the spectator with particular attention being paid, in this instance, to the recognition of emotions in the audience and to everything that is emotionally involving in the context of the scenic space.

The interactive installation allows a virtual character named Euclide to talk with the audience. The action takes place in two or more separate locations: in one a concealed animator wears a cyber glove connected to a graphic computer; in another a monitor, connected by a cable to the computer, shows the virtual character to the audience. The animator’s hand movements activate the virtual character, controlling his mimics; digital effects alter animator’s voice.


BIT - An interactive digital character made by studio azzurro

People can talk to the character while a camera shows them to the concealed animator; he can therefore react and respond to them. The audience gets truly caught and surprised at seeing a virtual character reacting to them; the contrast between the cold appearance of computer imagery and the lively, fun virtual character entices even the most reluctant spectator.

Studio azzurro noticed that the reaction of the audience to technology is extremely different when virtual characters are involved. Even if the audience is aware of the existence of the concealed animator (the audience knows where he is hidden), people react to the virtual character as if it was an independent entity and not a puppet in the hands of the hidden animator.

The character can be realised with different styles from realistic to cartoon or assembling objects (similar to Arcimboldo's paintings) and different effects and accessories can be added to personify it. Euclide can have the role of a guide and an educational assistant.

[text pasted from CALLAS' website]

Tangible Acustic Interfaces: the TAI CHI project

With ordinary interface devices (e.g. keyboard, mouse, touch screen and ultrasonic pen), the interaction of humans with computers is restricted to a particular device at a certain location within a small movement area. A challenge in human computer interaction research is to create tangible interfaces that will make the interaction possible via augmented physical surfaces, graspable objects and ambient media (e.g. wall, tabletop and air), as well as making the interaction natural without the need for a hand held device.

In the EU funded project “Tangible Acoustic Interfaces for Computer Human Interaction” (Tai-Chi), acoustics-based remote sensing technology is utilised since vibrations are the natural outcome of an interaction and propagate well in most solid materials. This means that the information pertaining to an interaction can be conveyed to a remote location (sensor) using the structure of the object itself as a transmission channel and therefore suppressing the need for an overlay or any other intrusive device over the area one wishes to make sensitive. The advantage of this new sensing paradigm over other methods of interaction such as computer vision or speech recognition suggests a strong potential for the computer industry. New applications can include wall-size touch panels, three-dimensional interfaces and robust interactive screens for harsh environments. The acoustics-based sensing used in Tai-Chi is the most promising way for human computer interaction, bringing another of the five human senses into the realm of computers: the sense of touch.



The ultimate goal of the Tai-Chi project is to develop acoustics-based remote sensing technology which can be adapted to virtually any physical object to create tangible interfaces, allowing the user to communicate freely with a computer, an interactive system or the cyber-world by means of an arbitrary object from the environment. The methods for contact point localisation developed in Tai-Chi utilise the location-signature embedded in the acoustic wave patterns caused by contact, as well as triangulation and acoustic holography.

The Tai-Chi project has involved the fundamental study of acoustic physical properties, advanced theoretical development of localisation algorithms, hardware structure design and demonstrations at public events. New results have been achieved for localising impacts (finger tapping, nail clicking and knocking) and tracking continuous movement (scratching) on large surfaces. Various localisation approaches have been thoroughly investigated, mainly, Time Delay of Arrival, Location Pattern Matching and Acoustic Holography with high precision attained using enhanced filtering techniques. In-air localisation has also been studied and tested using Time-Delay and Doppler-Shift to track a moving acoustic source in the air. Experiments were successfully conducted with different objects such as wood boards, window glass, plastic blocks and metallic sheets. In the final stages of the project, work is being carried out to produce a Tai-Chi Developer’s Tool Kit (TDK) comprising sensors, signal conditioning circuitry, a DSP unit, a PC interface together with a library of algorithms. Several papers on this work have been presented at conferences and published in journals.

Among the application demonstrators developed by Tai-Chi partners are: a Virtual Piano (a piano keyboard projected on white board that produces the relevant note when touched), a Memory Game (cards projected on plastic sheet), a giant tablet (that enables curve drawing by continuous movement of finger on a large wood board), a music chair and an interactive map interfaced to Google Earth.

[text pasted from TAI CHI's website]

The Reactable: from research to mainstream

The reactable is a collaborative electronic music instrument with a tabletop tangible multi-touch interface. Several simultaneous performers share complete control over the instrument by moving and rotating physical objects on a luminous round table surface. By moving and relating these objects, representing components of a classic modular synthesizer, users can create complex and dynamic sonic topologies, with generators, filters and modulators, in a kind of tangible modular synthesizer or graspable flow-controlled programming language.

With her first tour concert at the Coachella Festival in California, the Icelandic singer Björk introduced the reactable for the first time to a mainstream audience.


Bjork performs Declare Independence at Coachella, 2007, Reactable featured

[text pasted from reactable's website]

CALLAS - Conveying Affectiveness in Leading-Edge Living Adaptive Systems

The CALLAS Integrated Project aims to design and to develop a multimodal architecture which will include emotional aspects in order to support applications in the new media business scenario with an “ambient intelligence” paradigm.

The vision underlining the CALLAS Project is that new media can enhance user participation in content, media, and social interaction. In many ways, if old media is something static in the user experience, new media is a dynamic process that stimulates interaction and communication with users and technology. This dynamic approach entails a strong attention to all interface and interaction aspects. CALLAS would like to analyse, understand, and advance this participation including all the emotional aspects relevant to human communication processes.

Contemporarily, the CALLAS Consortium believes that one of the most interesting factors in human emotional interaction is the space where it takes place. For this reason, the scenarios chosen for CALLAS are all relevant different typologies of space: theatres, home, squares, festivals, etc. The CALLAS goal is to bridge the gap between the emerging capacity of conveying emotional aspects within multi-modal interaction and the growing expectations of people for a more natural and pervasive interaction with digital media applications in intelligent adaptive spaces.

[text pasted from CALLAS' website]