Patent US20040194033A1 successfully refreshed from Google Patents.
Patent US6647535B1 successfully refreshed from Google Patents.
US6647535B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Methods and systems for real-time storyboarding with a web page and graphical user interface for automatic video parsing and browsing
Assignee: Xerox Corp
•
Filing: 1999-03-18
•
Granted: 2003-11-11
•
Expires: 2019-03-18
Abstract
Abstract Systems and methods to enable real-time and near real-time storyboarding on the World Wide Web in addition to a graphical user interface for video parsing and browsing the of the storyboard. Specifically, storyboarding can be accomplished on the World Wide Web by parsing an input video into representative or key frames. These frames then can be posted to a web document, or the like, for subsequent viewing by a user. This allows a video to be distilled down to the essential frames thus eliminating storage and bandwidth problems as well as eliminating the need for a user to view the entirety of the video. Furthermore, the graphical user interface allows a user to visually interact with an input video signal to determine the key or representative frames, or to retrieve video segments associated with already determined key frames. Furthermore, the interface allows manipulation of these frames including, but not limited to, playing of the entire segment represented by that key or significant frame as well as actual determining of the cuts between significant segments.
Inventors
- 👤 Gozde Bozdagi
- 👤 Hong Heather Yu
- 👤 Michael R. Campanelli
- 👤 Robert Bryll
- 👤 Steven J. Harrington
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 21:11
CPC Classifications
H; H04; H04N; H04N21/00; H04N21/80; H04N21/83; H04N21/845; H04N21/8453; G; G06; G06F; G06F16/00; G06F16/70; G06F16/73; G06F16/738; G06F16/739; G06F16/74; G06F16/745; G06F16/78; G06F16/783; G06F16/7834; G06F16/7844; G06F16/7847; H04N21/20; H04N21/23; H04N21/234; H04N21/23418; H04N21/40; H04N21/45; H04N21/462; H04N21/4622; H04N21/85; H04N21/854; H04N21/8543; H04N7/00; H04N7/08; H04N7/087; H04N7/088; H04N7/0884; H04N7/0885; H04N7/0887; H04N7/16; H04N7/162; H04N7/165
Claims Analysis
Total Claims: 24
Claim 1
Independent Claim
108 words
A real-time storyboarding system, comprising: a frame capture device; a command detector that detects if the input multimedia image data signal contains a command data that indicates key representative frames, the command data is embedded in a closed-caption portion of an input multimedia image data signal in addition to a closed-caption data to be displayed; a command decoder that decodes the detected command data; and an image significance determiner that determines in response to the command detector if there is at least one representative frame in the input multimedia image data signal, and that, if at least one representative frame is found, outputs the at least representative frame.
Claim 2
Dependent Claim
108 words
A real-time storyboarding system, comprising: a frame capture device; a command detector that detects if the input multimedia image data signal contains a command data that indicates key representative frames, the command data is embedded in a closed-caption portion of an input multimedia image data signal in addition to a closed-caption data to be displayed; a command decoder that decodes the detected command data; and an image significance determiner that determines in response to the command detector if there is at least one representative frame in the input multimedia image data signal, and that, if at least one representative frame is found, outputs the at least representative frame.
Claim 3
Dependent Claim
16 words
The system of claim 1 , wherein the command data identifies at least one representative frame.
Claim 4
Dependent Claim
28 words
The system of claim 1 , further comprising a frame difference determiner that determines a change between at least two frames of the input multimedia image data signal.
Claim 5
Dependent Claim
32 words
The system of claim 3 , wherein a change between the at least two frames is a change in intensity between at least two frames of the input multimedia image data signal.
Claim 6
Dependent Claim
121 words
The system of claim 4 , wherein the change in intensity E(t i ) for a current frame occurring at a time t=t i , relative to a next frame occurring at time t=t i+1 , is: E ( t i )=Σ (x,y) |I ( x,y,t i )− I ( x,y,t i+1 )|, where: x and y are spatial locations within a frame; I(x,y,t i ) is an intensity of a pixel at the spatial location (x,y) of the frame occurring at time t i ; I(x,y,t i+1 ) is an intensity of a pixel at the spatial location (x,y) of the frame occurring at time t i+1 ; and the summation is over all the pixels within the current frame.
Claim 7
Dependent Claim
39 words
The system of claim 5 , wherein, if the change in intensity between the at least two frames is greater than a predefined threshold, one of the current frame and the next frame is identified as a representative frame.
Claim 8
Dependent Claim
27 words
The system of claim 1 , wherein the at least one representative frame is output to a device capable of displaying the at least one representative frame.
Claim 9
Dependent Claim
18 words
The system of claim 1 , wherein the at least one representative frame is output to a printer.
Claim 10
Dependent Claim
19 words
The system of claim 1 , wherein the at least one representative frame is output to a web document.
Claim 11
Dependent Claim
21 words
The system of claim 9 , wherein the web document is updated once a threshold number of representative frames are determined.
Claim 12
Dependent Claim
29 words
The system of claim 1 , wherein at least a portion of the input multimedia image data signal is analyzed before the at least one representative frame is output.
Claim 13
Dependent Claim
25 words
The system of claim 11 , wherein the entire input multimedia image data signal is analyzed before the at least one representative frame is output.
Claim 14
Independent Claim
116 words
A method for real-time storyboarding, comprising: selecting at least a portion of an input multimedia image data signal; determining whether a command data is present such that the command data is embedded in a closed-caption portion of the input multimedia image data signal in addition to a closed-caption data to be displayed; the command data indicates key representative frames, wherein if command data is present, decoding the command data to determine at least one representative frame; determining in response to the command data is present the at least one representative frame of the selected portion of the input multimedia image data signal; adding the at least one representative frame to a document; and outputting the document.
Claim 15
Independent Claim
116 words
A method for real-time storyboarding, comprising: selecting at least a portion of an input multimedia image data signal; determining whether a command data is present such that the command data is embedded in a closed-caption portion of the input multimedia image data signal in addition to a closed-caption data to be displayed; the command data indicates key representative frames, wherein if command data is present, decoding the command data to determine at least one representative frame; determining in response to the command data is present the at least one representative frame of the selected portion of the input multimedia image data signal; adding the at least one representative frame to a document; and outputting the document.
Claim 16
Dependent Claim
49 words
The method of claim 13 , wherein: determining whether command data is present comprises determining if speaker data identifying at least one representative frame is present; and if speaker data identifying at least one representative frame is present, adding the at least one identified representative frame to the document.
Claim 17
Dependent Claim
55 words
The method of claim 13 , wherein: determining whether command data is present comprises determining if at least one special characters identifying at least one representative frame is present; and if at least one special character identifying at least one representative frame is present, adding the at least one identified representative frame to the document.
Claim 18
Dependent Claim
57 words
The method of claim 13 , wherein: determining whether command data is present comprises determining if a number of at least one special character identifies at least one representative frame; and if the number of the at least one special character identifies at least one representative frame, adding the at least one representative frame to the document.
Claim 19
Dependent Claim
81 words
The method of claim 13 , wherein: determining whether command data is present comprises determining if a time lapse in the selected portion of the input multimedia image data signal since a last representative frame is greater than a threshold; if the time lapse is greater than the threshold, identifying at least one additional frame as a representative frame of the selected portion of the input multimedia image data signal; and adding the at least one identified frame to the document.
Claim 20
Dependent Claim
47 words
The method of claim 13 , further comprising: determining a frame difference between a pair of frames; and if the frame difference exceeds a threshold, identifying one of the pair of frames as a representative frame of the selected portion of the input multimedia image data signal.
Claim 21
Dependent Claim
13 words
The method of claim 18 , wherein the threshold is an intensity threshold.
Claim 22
Dependent Claim
119 words
The method of claim 19 , wherein the intensity E(t i ) for a current frame occurring at a time t=t i , relative to a next frame occurring at time t=t i+1 , is: E ( t i )=Σ (x,y) |I ( x,y,t i )− I ( x,y,t i+1 )|, where: x and y are spatial locations within a frame; I(x,y,t i ) is an intensity of a pixel at the spatial location (x,y) of the frame occurring at time t i ; I(x,y,t i+1 ) is an intensity of a pixel at the spatial location (x,y) of the frame occurring at time t i+1 ; and the summation is over all the pixels within the current frame.
Claim 23
Dependent Claim
38 words
The method of claim 20 , wherein if a change in intensity between the current frame and the next frame is greater than a predefined threshold, at least one of the frames is identified as a representative frame.
Claim 24
Dependent Claim
47 words
The method of claim 13 , further comprising: determining whether a threshold number of representative frames for the document have been identified; and if the threshold is reached, selecting a new document; and adding the at least one representative frame exceeding the threshold to the new document.
References Cited (Backward Citations)
US5481296A
Cited
US5485554A
Cited
US6271892B1
Cited
US5703655A
Cited
US5959687A
Cited
US5852435A
Cited
US5959622A
Cited
US6098082A
Cited
US6031526A
Cited
US6097442A
Cited
US6006241A
Cited
US6463444B1
Cited
US6219837B1
Cited
US6297797B1
Cited
US6366699B1
Cited
US6363380B1
Cited
US6154771A
Cited
US6462754B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2006-11-10 | 2007-05-12 | 2007-11-11 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2010-11-10 | 2011-05-12 | 2011-11-11 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2014-11-10 | 2015-05-12 | 2015-11-11 | $7700.00 | Window Closed / Lapsed |