Senior Design Projects
ECS193 A/B Winter & Spring 2022
Super Slow-Motion Video
| ********** | |
| Brian Thomas | |
| Telestream |
Project's details
| Super Slow-Motion Video | |
| Slow-motion video is used frequently in broadcast video and, particularly, in sports production to provide the viewer with additional perspective on the content. Slow-motion video is best achieved with a high frame rate camera, with the resultant footage played back at a lower frame rate. If a high frame rate camera is not available, then software techniques must be applied to achieve the same results. A simple way to achieve this effect is to play back the recorded content at a frame rate slower than the recorded speed. This method produces “jerky” video. An enhanced method involves interpolating the video at intermediate points between frames to restore smooth playback. Optical flow is defined as the apparent motion of surfaces in a series of images. By using optical flow information to determine the position of surfaces at times logically between images, a super slow-motion video can be created. In addition to rendering video for display, NVIDIA graphics cards can be used to accelerate compute tasks. On certain models, there are dedicated hardware resources to perform operations related to video compression, machine learning etc. One such acceleration task is the calculation of optical flow vectors. This is made available through an SDK (https://developer.nvidia.com/opticalflow-sdk) | |
| The goal of this project is to create and integrate a C++ library that uses NVIDIAs optical flow SDK to produce super slow-motion video from standard frame rate video files. The team will work on a novel algorithm that takes into account non-linear motion and occlusions in the video, rather than trying to use any slow-motion video samples that might be present in the SDK. A stretch goal will be to apply this technique to other video processing algorithms which also requires the interpolation of video images (e.g. deinterlacing, frame rate conversion). The project is setup to mirror a typical R&D project in a software development company. The team receives this project description and is expected to meet with their technical mentors on a weekly basis. The mentors will help adjust the project priorities given the discoveries made, with the goal of being able to present a proof of concept to an executive team at the end of the project. The presentation would include enough information for the executives to make a go / no-go decision on productizing the development. It is anticipated that the team would divide the work, for example a four-person team might split the task … • Algorithm Development (selection and interpolation of optical flow vectors) • Integration of optical flow SDK into a video processing library • Test video player application; or alternatively development of a super slow-motion video conversion application • Quantification of results; how well does this method work relative to other slow-motion tools. | |
| • Create an application demonstrating the conversion of video from normal to super slow-motion. • Present to company executives on the application / viability of the developed technology. | |
| It is expected that the project team would have coding skills appropriate to a final year computer science undergraduate degree and that, between the group, they would have some exposure to C++. Knowledge of video processing would be gained during the course of the project. The mentors would be on hand to assist with questions related to video processing / C++ / NVIDIA SDKs etc. In addition, the mentors would provide access to hardware with the appropriate NVIDIA graphics cards installed, or alternatively loan graphics card(s) to the team. | |
| ********** | |
| 30-60 min weekly or more | |
| Open source project | |
| Attachment | N/A |
| Yes | |
| Team members | N/A |
| Rex Liu | |
| N/A |