Closed

Face tracking & lip activity detection real time video processing (AI powered)

Requirement:

A computer vision software that processes multiple video feeds to find somebody talking in the video to then select him and magnify him.

Problem:

A "hybrid meeting" takes place. This means, 10 people are sitting in a conference room on one table and you are taking part over Zoom.

For you, this is a bad experience. You feel (and are) excluded: All other meeting members are sitting in the same room, chatting, looking into each others eyes and having all the advantages that real-life meetings have. You on the other end are only seeing all 10 people all together. The one camera in the meeting room only gives you one angle. You mostly see them from the side. You only see them very small since they all have to fit into one video. Some of the meeting attendees you don't see at all, because someone else is sitting in front of them and the camera does not always have a clear view. Sometimes people turn and talk to someone sitting across the table, so you can only see their back of the head.

For the 10 people, you are ruining their meeting, because every time they tak to you, they have to make sure that you can see them and are turning to the screen and camera, possibly neglecting the main audience, all others in the room.

Solution:

The meeting room is equipped with multiple cameras. Each camera produces a video that contains one or multiple people.

All videos should be processed in realtime and the person that is currently speaking should be selected, cropped out and resized to fill the frame to be then sent on as input for video conferencing software (Zoom, Teams, Skype, whatever).

When speaker is clearly detected, the output video should be the video with the highest number of people in it, cropped with all faces still visible and resized to fill the frame.

When only one person speaks at the same time (which usually happens if the meeting is going well), this should result in a much better meeting experience for everyone.

Input:

Multiple video feeds of people in a conference room holding a meeting.

Output:

The one person that is currently speaking selected and cropped out of the rest and resized to fill the frame

Methods:

- Face tracking

- lip activity detection

- scoring system to select the most likely speaker from hist most head-on viewing angle

- post processing to crop and resize the speaker

- re-evaluating in real time to select different speaker

- optional: Also analyze sound input to detect change of speaker

Software suggestions:

- Face detection / tracking: [login to view URL]

- Lip Movement Detector: [login to view URL]

- Lip Tracking: [login to view URL]

Hardware Suggestions:

- Coral USB Accelerator Edge TPU as a coprocessor to accelerate inferencing for machine learning models:

[login to view URL]

Skills: Machine Vision / Video Analytics, Machine Learning (ML), Face Recognition, Video Processing, Computer Vision

See more: face recognition software, amazon rekognition, face recognition algorithm, amazon rekognition pricing, how facial recognition works, amazon rekognition face recognition, face recognition, facial recognition system, java code send real time video audio, real time video streaming java, desktop activity viewer real time, needs project addresses real time video streaming, asp net real time video streaming, augmented reality real time video, object detection algorithms real time video code, real time video upload, real time video conversion, android real time video streaming, android real time video, iphone development real time video chat

About the Employer:
( 22 reviews ) Berlin, Germany

Project ID: #27886793

27 freelancers are bidding on average $2259 for this job

Himerr0kovoi

Hello, thanks for your posting and i read you description carefully. I have 7 years experience of computer vision and machine learning. So i have full skills and experience of this field, and have developed a lot of pr More

$2700 USD in 30 days
(15 Reviews)
6.1
softbox06

Interested in your project. The bid is negotiable and we can talk about the price. I am computer vision/machine learning and data analyst engineer. I did many industrial projects. see My work: [login to view URL] More

$2650 USD in 7 days
(18 Reviews)
4.7
ahmedelhelow

Hi bro, I am Ahmed. I have a Master's degree in Computer Science from Europe. I can immediately and perfectly do your face tracking and lips activity detection task. I am a Machine Learning expert. Please contact me to More

$1500 USD in 6 days
(5 Reviews)
4.1
rostomide

*** EXPERIENCES IN COMPUTER VISION *** Thanks for your posting! I am a image processing expert using machine learning, such as tensorflow, caffe, darknet and etc. I have developed a lot object detection and recognition More

$1500 USD in 7 days
(2 Reviews)
4.2
helmot

Hello, I have a masters degree in AI and have worked on AI/ML projects for 8+ years. If you are interested I can share my desktop and show you some demos/samples. I have worked with most AI/ML tools like NLTK and More

$2500 USD in 30 days
(1 Review)
3.9
Demenntor

Dear Employer, I have read your project description and I am glad to let you know that I can build a computer vision software that processes multiple video feeds to find somebody talking in the video. I'm an expert in More

$2300 USD in 3 days
(6 Reviews)
3.4
akulovvladimir91

******** MACHINE LEARNING EXPERT AND VISION PROFESSIONAL ************ Thank you for your posting! I am an Machine Learning expert with full experiences such as tensorflow(including TfLite), caffe, DARKNET, OPENALPR, e More

$1500 USD in 10 days
(3 Reviews)
3.7
asifmahfuz1405

hello sir, i am highly interested in your project. i have gone through your requirements and i believe i can be a valuable asset for your project. i am an expert in deep learning and computer vision in python. previou More

$1500 USD in 3 days
(5 Reviews)
3.6
nikolaytoplev

Hi, there! This job is an ideal match for my skills and experience in artificial intelligence, especially for deep learning projects with Python, Matlab and so on. Since I majored in mathematics and am specialized in a More

$3000 USD in 7 days
(2 Reviews)
3.4
PKonstiantyn

HI, I am experienced developer in the field of AI, ML, deep learning and Time series analysis (Algo Trading) and have been working as data scientist from last two years. I can handle task by using Python( Tensorflow, k More

$2250 USD in 7 days
(3 Reviews)
2.7
WIFTCAP

Hi, nice to e-meet you. We have a team of experienced full stack developers with a focus on Python/Django. We have built a number of web and mobile based applications including (not limited to): 1. A sophisticated trad More

$2100 USD in 18 days
(2 Reviews)
1.4
mukshynandrey

Hello! As an expert in deep learning, I've many experiences in computer vision and pattern recognition. Specially, face recognition and verification, emotion detection, face landmarks detection, detection of mouth ope More

$2000 USD in 7 days
(2 Reviews)
1.6
protaga

Hi I read your project very carefully and I think it is a very interesting project and I love to work on this project. we have 3+ years of experience in development. We have been working with clients across India for More

$2250 USD in 7 days
(1 Review)
0.7
BPaulhbc

Hello Sir, I have gone through your job posting and become very much interested to work with you. I'm an expert in this field. I have done several projects like this before. I will provide my best effort to complete yo More

$1500 USD in 2 days
(0 Reviews)
0.0
tecogno

Hi, We at Tecogno are a team of passionate Data Science professionals having more than five years of combined experience in multiple areas including Backend, Frontend, Machine learning (ML) and Artificial Intelligence More

$3000 USD in 21 days
(0 Reviews)
0.0
sstarcorp

Hi, We are a team of data science and ML/AI experts who excel across multiple areas with more than twenty five years of combined experience. We hold expertise in Python, Backend Architecture (micro-services, Kubernete More

$2500 USD in 20 days
(0 Reviews)
0.0
sajjadtaghvaeifr

Hi, I hope you are doing fine. I have done many image processing and video processing projects in matlab, python, JAVA, etc. My PhD thesis was also visual analysis of human motion. I have also published several journal More

$1500 USD in 7 days
(1 Review)
0.0
andrewpirlya

Hi, I have 10+ years of experience in Image processing using OpenCV and ML. My main programming languages are C#, C++ and Java script. I think this project is very suitable for me and I am sure to give good results. B More

$2500 USD in 15 days
(0 Reviews)
0.0
shanecarlyon

Hi, mate. I am really glad to have an opportunity to help you with my skills & experience in Computer Vision. I developed lots of similar projects to this - face tracking & lip detection. So I have rich experience in More

$2250 USD in 5 days
(0 Reviews)
0.0
BlackQR7632

hello, My self jaimin patel. Recently our team done a project releted your description. we track and analyse the person details and track it. According to your description we will surely deliver best over than you e More

$1778 USD in 35 days
(0 Reviews)
0.0