A neural network that enhances videos with animation and vibrant colors
Analysis and processing of video streams
About
01/
Client
Advertising agency APM is a partner of AB InBev Efes, a producer of beer and non - alcoholic beverages in Russia and Ukraine.
Background
ESSA ran an ad in which a person and their background were painted in different colors.
A task
Create a service where VKontakte and Telegram users will be able to create similar videos.
What we came up with
02/
A neural network that creates different video effects:
«Neon boom»
with the appearance of contrasting countrours the user's body, eyes, mouth, and clothes
«Superwave»
with a wave filling the contour of a person
«Bright party»
with solid colors fill in the different body parts and clothes
What the neural network is capable of
03/
Distinguish a person from furniture and background
Segment objects and paint them in different colors
Designate eyes, hair, drawing on clothes with different colors
01
02
03
How it works
04/
User reads the QR code on the ESSA beer bottle with his phone
The ESSA "VKontakte" service page opens on the phone
/start
*uploads video*
*selects video effect*
Hey! This is Essa's chatbot and I can animate any dance video of yours! Send me your incendiary video from 14 to 40 seconds long!
Video received! Choose a processing style for your video:
Neon boom
Superwave
Bright party
It will take me some time to process. I'll send everything soon.
Everything worked out! Your video is available at https: // ...
And here's how the neural network «saw» our team:
Tools
06/
To create a bot, we used our own services:
Simple and convenient platform for developing chatbots, voice assistants and contact center automation systems.
zDialog
AI service for chatbots analytics. Allows you to conduct a deeper analysis of human-robot dialogues and get a more accurate assessment of the effectiveness of the bot.
OneDash
We create a hosting platform for the execution of computer vision models on any device. Models are executed on our servers and are available via REST API
Vision Hub
Development process
06/
Step 01
Trained the neural network to «see» and segment body parts and clothes on video
Step 02
Step 03
Step 04
We tested the service, made necessary debugging
Step 05
Step 06
Wrote a user guide on how to best shoot videos
Launched the service
Trained the neural network to «see» and segment furniture on video
Made three different effects for the video
Difficulties of the project
07/
01
Problem
The custom videos were of low quality, with poor lighting.
Decision
To improve the picture, we used four computer vision models. They predicted the position of an object in the frame, making the image clearer.
02
Problem
The neural network could color different frames in different ways; noises appeared in the video.
Decision
We have improved the work of the algorithm that was responsible for the clarity of the picture.
03
Problem
It was a time-consuming process: a 15- second video would take 15 minutes to process. This created a risk that the user would not wait or that the load on the serve would become too great
Decision
We indicated to the user exactly how much time was needed to queue a video to inform them of possible wait times. Added additional spare capacity which made provided stable operation
Project results
08/
A ton of new knowledge is our largest and most complex project related to image segmentation on video
01
02
The client was satisfied and decided to launch a similar service for users in Ukraine and Belarus
Project team
09/
2 project managers
3 backend developers
Technical Director
2 data analytics
What we learned
10/
The implementation of a complex combination of machine vision models
Process video recordings of different quality, including compressed algorithms
Contact us
PRODUCTS
SECTIONS
STUDIOS
Russia, Innopolis, st. Universitetskaya, 7
2023 © EORA. All right reserved