«Revived» fairy tale about Red Riding Hood with 3D masks of heroes

SberPortal smart display application

SberDevices logo
СберПортал
SberPortal

Client

Sber Devices is a subsidiary of Sber. Produces smart devices and services

Features of the project

Render a fairy tale and make 3D masks of heroes. The fairy tale is pre-installed in the SberPortal smart display

Task


Red Riding Hood

Main information

01/
The project was developed from scratch: ready-made solutions did not suit us
The Miracle Books application is an interactive fairy tale: while reading, you can «play» one of the characters and change backgrounds.
Decision

Scheme of work

02/
Parent
Child
tells a fairy tale
The camera detects a face in the frame
A hero's mask appears on it
The adult switches text and changes backgrounds as the plot progresses

Examples of masks

03/

Mom mask

Wolf mask

How the app works

04/

Fairy tale characters

05/
Красная шапочка, Мама, Бабушка, Серый волк, Охотники
Red Riding Hood
Mother
Grandmother
Gray wolf
Hunters

Transfer of emotions

06/
The characters' faces are not static: they express surprise, joy, sadness.
Each character has his own set of poses, which reflects his character.

Rendering scenes

07/
Сцена с охотниками
Сцена с красной шапочкой, серым волком
Сцена с красная шапочкой, серым волком
Сцена с красной шапочкой, Мамой, Бабушкой

Development process

08/
1 month
1 month
1 month
3 months
2 months
study of materials, creation of design
development and testing of 3D masks
preparation of test demonstrations
launching the demo version of the application
QA testing, release

Difficulties of the project

09/
Problem
Due to the peculiarities of SberPortal, we could not use a ready-made solution for face recognition and masking.
Decision
Instead of using off-the-shelf solutions like ARCore and Banuca, we used pre-trained BlazeFace and FaceGeometry networks.

Project team

10/
Manager
Anton Shipov
Developer
German Tarasov
Technical Director
Sergey Verentsov
A team of 3D visualization specialists

Emotions from the project

11/
It was interesting to work on a serious design with 3D visualization and rendering of all the characters. I am proud that millions of people will see our project
The team was delighted: this is a completely new product on the market, this is a complex project that has no analogues in the CIS
German Tarasov
Anton Shipov
Project manager
Developer

What's new we learned

12/
How to build Android apps
The service was tested on#mnsp;a prototype of a «smart» display
How to work with SberPortal
The service was adapted for different Android devices
Contact us