Video synopsis

Video synopsis combines a visual summary of stored video together with an indexing mechanism.

When a summary is required, all objects from the target period are collected and shifted in time to create a much shorter synopsis video showing maximum activity. A synopsis video clip is generated, in real time, in which objects and activities that originally occurred in different times are displayed simultaneously.Y. Pritch, A. Rav-Acha, and S. Peleg, [http://www.vision.huji.ac.il/video-synopsis/pami08-synopsis.pdf Nonchronological Video Synopsis and Indexing], IEEE Trans. PAMI, Vol 30, No 11, Nov. 2008, pp. 1971-1984.

Image:WhitePaper fig 2.jpg

The process begins by detecting and tracking objects of interest. Each object is represented as a tube in space-time of all video frames. Objects are detected and stored in a database in approximately real time.

Following a request to summarize a time period, all objects from the desired time are extracted from the database, and indexed to create a much shorter summary video containing maximum activity.

Real time rendering is used to generate the summary video after object re-timing. This allows end-user control over object/event density.

Video synopsis technology was invented by Prof. Shmuel PelegA. Rav-Acha, Y. Pritch, and S. Peleg, [http://www.vision.huji.ac.il/video-synopsis/cvpr06-synopsis.pdf Making a Long Video Short: Dynamic Video Synopsis], CVPR'06, June 2006, pp. 435-441. of the Hebrew University of Jerusalem, and is being developed under commercial license by BriefCam, Ltd.S. Peleg, Y. Caspi, [http://www.briefcam.com/PDFs/White_Paper_new.pdf BriefCam White Paper]

BriefCam received a license to use the technology from Yissum which is the owner of the patents registered for the technology. In May 2018, BriefCam was acquired by Japanese digital imaging giant, Canon Inc., for an estimated $90 million.{{Cite news|url=https://en.globes.co.il/en/article-canon-buys-israeli-video-solutions-co-briefcai-1001235224|title=Canon buys Israeli video solutions co BriefCam for $90m|last=Yablonko|first=Yasmin|date=9 May 2018|website=Globes|language=Hebrew|access-date=2018-12-05}} Investors in the company include Motorola Solutions Venture Capital, Aviv Venture Capital, and OurCrowd.{{Cite web|url=https://www.cbinsights.com/company/briefcam-funding|title=BriefCam Funding & Investors - CB Insights|last=CB Insights|website=www.cbinsights.com|language=en-US|access-date=2018-12-05}}

File:Video Synopsis before+after (horizontal).jpg

Recent advances in the field of video synopsis have resulted in methods that focus in collecting key-points(or frames) from the long uncut video and presenting them as a chain of key events that summarize the video. This is only one of the many methods employed in modern literature to perform this task.Muhammad Ajmal, Muhammad Husnain Ashraf, Muhammad Shakir, Yasir Abbas, Faiz Ali Shah, Video Summarization: Techniques and Classification [https://link.springer.com/chapter/10.1007%2F978-3-642-33564-8_1] Recently, these event-driven methods have focused on correlating objects in frames, but in a more semantically related way that has been called a story-driven method of summarizing video. These methods have been shown to work well for egocentricZheng Lu, Kristen Grauman [https://www.cs.utexas.edu/~grauman/papers/lu-grauman-cvpr2013.pdf] Story-driven summarization for egocentric video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition - 2013.

settings where the video is basically a point-of-view perspective of a single person or a group of people.

Video synopsis techniques have a number of standardized properties in common, which can be quantified as follows: (a) the video synopsis should contain the maximum activity with the least redundancy; (b) the chronological order and spatial consistency of objects in space and time must be preserved; (c) in the resultant synopsis video, there must be minimal collision; and (d) the synopsis video must be smooth and able to permit viewing without losing the region of interest. Ingle, P.Y. and Kim, Y.G., 2023. Video Synopsis Algorithms and Framework: A Survey and Comparative Evaluation. Systems, 11(2), p.108.

1. Keyframe-Based Synopsis
2. Object-Based Synopsis
3. Action-Based Synopsis
4. Collision Graph-Based Synopsis
5. Abnormal Content-Based Synopsis

• ActionShot
• Video content analysis

{{Reflist}}

• [http://www.cs.huji.ac.il/~peleg/patents/US8311277.pdf US Patent 8,311,277] - Method and System for Video Indexing and Video Synopsis. Also [http://www.cs.huji.ac.il/~peleg/patents/AU2007345938.pdf Australian Patent AU2007345938.] Filed in 9 additional countries
• [http://www.cs.huji.ac.il/~peleg/patents/US8102406.pdf US Patent 8,102,406] - Method and System for Producing a Video Synopsis. Also [http://www.cs.huji.ac.il/~peleg/patents/EP1955205.pdf European Patent EP1955205] and [http://www.cs.huji.ac.il/~peleg/patents/JP4972095.pdf Japanese Patent JP4972095]. Filed in 7 additional countries
• [http://www.cs.huji.ac.il/~peleg/patents/US7852370.pdf US Patent 7,852,370] - Method and System for Spatio-Temporal Video Warping. Also [http://www.cs.huji.ac.il/~peleg/patents/IL182006.pdf Israeli Patent IL182006]. Filed in Europe: PCT/IL2005/001150
• The owner of the patents are Yissum which gave a License to BriefCam to use the technology.

• [http://www.vision.huji.ac.il/video-synopsis/ Video Synopsis and Indexing project] Further information, published papers and video clips from the School of Computer Science, The Hebrew University of Jerusalem
• [http://www.briefcam.com/ BriefCam company website] Includes demonstration video on using video synopsis technology.
• [http://www.yissum.co.il Yissum's Website]
• [http://sites.google.com/site/videosynopsis2008il/home Video Synopsis paper in Hebrew ] - 2008
• [https://www.youtube.com/watch?v=ovXt_3DL2No] Demo video for paper: Story-Driven Summarization for Egocentric Video - Zheng Lu and Kristen Grauman - CVPR 2013

Category:Image processing

Category:Israeli inventions

Category:Video surveillance