Contact address logo Eisko, 24 rue de l'Est 75020 Paris
Contact phone logo +33 (0)1 85 08 52 51
Contact mail logo
> contact us!
Eisko logo


tech friday #2 : Eisko capture protocol

November 28, 2014

Hi folks!

Last week we started to explain our capture system and approach. Today, let's focus on the protocol : what does the scanned person have to do while being scanned?

When it comes to capturing expressions, a lot of things have to be planned in advance: you cannot afford to recall the person later because you missed a pose. Nor you can capture thousands of expressions just to be on the safe side: in production, things usually need to be done very quickly.

Having worked on human capture for years, we have explored all of the existing capture protocols, and have taken the best aspects of each one to create our own. After years of iteration and improvment, we think we have quite a well-optimized capture protocol. It basically has three main parts:

- Muscle deformations of face (eye ball deformations, opening the mouth, jaw movements, etc.)

- Emotionnal expressions. These are grouped in two categories: primary emotions (happiness, anger, fear, etc.) and secondary emotions (interest, irritation, disregard, sarcastic, etc.)

- Visemes deformations. Visemes are a subset of phonemes. Basically, they represent the shapes that a face makes while speaking. Recording them is a bit more complicated than reciting the letters of the alphabet, as the same as the same phoneme induce different deformations in different contexts (depending on surrounding words and emotional state).
Visemes fall into three main categories:  widened consonants, rounded consonants and monophtongs.

By asking the subject to recite a standard phrase that contains all the visemes in the English or French language, we can quickly capture all the speech-related facial deformations. These phrase are derived from our study of various scientific papers on the subject.

This protocol allows us to capture and characterize all the possible deformations linked to the 104 physiological degree-of-freedom of your face, or my face, or anyone else's.

Once this is done, we are left with Gigabits of data. What we do with that data is a question we will answer in our future Tech fridays. Stay tuned!