The Project is to create a "Proof of Concept" mini-application (later extendable, of course) - yet, for now WITHOUT any design, menus, user handling, authentication, etc - in which a pre-recorded, High-FPS Mp4 video should be adjusting its on-screen playback speed in real-time, gradually, proportionally to the detected & calculated approximate actual musical BPM ("Beats Per Minute") from the Phone's Microphone. (The Video itself will not have sound, or, at least not for now). Video Playback speed should follow the rhythm / tempo change in real-time.
In plain words - The mini-Application should download (prior to starting the visualization) a High-framerate MP4 Video file from the Client's server, and the use the Microphone to continuously take samples (ie. in every 5-10-20 milliSeconds), to be stored for evaluation in a Sliding-time-Window array, to look for "Volume Peaks" inside which are significantly (ie. min 1.5x) louder than the average Volume of the latest few seconds, count them (by excluding double-detection of the same) Peaks, and then approximate the musical BPM, based on the last few seconds ie. the latest 5 seconds) by counting the Peaks & linearly projecting them up to 60 seconds (BPM).
BPM evaluation & video playback speed variation should be continuous & instantaneous, ie on each new sample taken from the MIC (ie. every 5-10-20 miliSeconds), BPM estimation should be re-calculated, & video playback speed altered, if needed. Since BPM is calculated from a sliding window, averaging BPM over ie. 5 seconds, this should almost never result in sudden "jumps" in BPM and / or correlated Video playback speed.
The Client-provided videos will be short (30-40 seconds), but of relatively high framerate (ie. 60-240 true FPS), so that slowing down the Video should & could will result in very SMOOTH playback, WITHOUT GLITCHES OR JUMPS (ie. a 120 FPS video would still have 15 Frames/ second even if slowed down to 1/8, or 0.125 speed.
The Target video playback speed rate - relative to the calculated BPM - will also be provided by the Client's server, for example (a typical value) would be 1.0x playback speed for 120 BPM, changing linearly (0.5x speed for 60 BPM, 2x speed for 240 BPM), and so on.
The Project is for both Android & iOS, but Android should be prioritized in Milestones as well because it is 10x times more common in the Client's country.
The project will be split up to 4-6 milestones, depending on the accepted bid, to be discussed after granting the job. More technical details will also be available in the Tech Spec document for the selected Freelancer.
For serious bidders, even more detailad technical specification will be available.
Bidders should be deeply familiar with AudioVisual / Multimedia capabilities of both Android and iOS devices, the project might require low-level, Native function calls on both Audio Pre-processing, Audio Sampling, as well as Real-time Video playback speed variations (ie. Speeding up / slowing down a high-FPS video on-the-fly).
26 freelancers are bidding on average €2700 for this job
Hi. I can take care of the BPM and I have experience with analysing audio and producing visuals using FFT and other means. I can also work with video synthesis and resembling them based on requirements.