Chatbot to help women find gifts for men
Image analysis
Main information
01/
Customer
Create a picker of gifts through «Vkontakte»
Task
Bot function
Girl uploads a photo of man to the chatbot
Project within the Avon advertising campaign by February 23
Audience
Women 25-45 years old
About the project
The bot helps the lady choose a gift depending on the type of man the gift is for
Project character
02/
As part of the project, showman Timur Rodriguez tried on the role of a professional translator from male language to Russian.
«Привет! Я Тимур Родригез. Я профессионально перевожу с «мужского» на русский. И помогу тебе выбрать лучший подарок для твоего мужчины»
After the girl uploaded a photo of the man, the bot made comic comments on behalf of Timur:
The chatbot distributed original gift ideas based on the data received about a particular man
artificial intelligence with Timur Rodriguez
Project team
03/
Techlead
Vladislav Vinogradov
Chatbot developer
Roman Afanasiev
Project manager
Emil Maharramov
Data Scientist
Ramil Gizzatulin
Konstantin Kubrak
Computer Vision Engineer
Development duration
04/
Integration
Markup
Validation
10.2%
13.6%
20.3%
Preprocessing
10.2%
Testing
8.1%
Deploy
5.1%
Training
32.5%
Total:
3.5
months of work
Seven neural network models are used for image analysis
Interesting fact
Work scheme
05/
Girl uploads a photo of a man to the chatbot
The neural network analyzes the photo according to the parameters:
01
Type of clothing
02
Shape type
Hair color
03
Hair type on the face and head
04
The neural network creates a segmentation mask:
fair-haired
mustache
stubble
thin
business
Depending on the parameters obtained, the man in the photo is assigned one of four types:
Brutal
Homebody
Metrosexual
Fidget
Each type has its own recommended gift - Avon cosmetic product
Tools
06/
To create a bot, we used our own services:
A simple and convenient platform for developing chatbots, voice assistants and contact center automation systems.
zDialog
AI service for chatbots analytics. Allows you to conduct a deeper analysis of human-robot dialogues and get a more accurate assessment of the effectiveness of the bot.
OneDash
We created a hosting platform for the execution of computer vision models on any device. Models are executed on our servers and are available via REST API
Vision Hub
Some statistics
07/
photographs
175 435
audio
78 434
During the campaign
2 343
messages
Technical implementation
08/
revealing gender of a person
01
Since the project was designed only for men, it was necessary to teach the neural network to determine gender.
To do this, we took a large dataset consisting of photographs of people with indication of gender.
To increase the accuracy, we used the dlib neural network, which detected the face on the photo.
clothing analysis
02
To analyze clothes, we also used a segmentation model, that is, the selection of clothes, and then assigned it one of five classes:
In addition, it was necessary to teach the neural network to find nude photos, i.e. lack of clothes in the photo.
casual
sportswear
military uniform
business clothes
extreme equipment
figure analysis
03
In order to determine the type of person in the photo more accurately, it was necessary to train the neural network to recognize the figure.
We decided to take two main parameters - to learn to understand the complexion of a person and his approximate weight.
So we were able to separate four classes:
thin
full
normal
muscular
analysis of vegetation zones on the face and hair color
04
First, it was necessary to determine the type of vegetation by zones or its absence, and then determine the color of the hair / beard.
To do this, we also needed a face detector, and in the next step we used a pre-trained Tiramisu model and a set of rules to imporve the accuracy of the detection.
After that, it was necessary to determine the hair color:
background analysis
05
The last task was to classify the background - it was necessary to try to determine the shooting location or characteristic features such as the sea, a car, and so on.
Due to the low quality of real data, the result was not very accurate, so it was practically not taken into account in the final model.
An even more detailed presentation of the project can be found here
Project difficulties
09/
«Oddly enough, there were almost no difficulties. The hardest part was to classify the background. We spent with it a lot of time, but as a result we just decided to remove it - it turned out that the other parameters were enough».
Konstantin Kubrak
Computer Vision Engineer
Project results
10/
It was interesting and exciting, because many things were new:
The first chatbot to use machine vision
A pipeline of seven neural networks within one project
First attempts to analyze sound
Contact us
PRODUCTS
SECTIONS
STUDIOS
Russia, Innopolis, st. Universitetskaya, 7
2023 © EORA. All right reserved