US6785329B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Automatic video object extraction
Assignee: Webassuredcom
•
Filing: 1999-12-21
•
Granted: 2004-08-31
•
Expires: 2019-12-21
Abstract
Abstract Automatic video object extraction that defines substantially precise objects is disclosed. In one embodiment, color segmentation and motion segmentation are performed on a source video. The color segmentation segments the video by substantially uniform color regions thereof. The motion segmentation segments the video by moving regions thereof. The color regions and the moving regions are then combined to define the video objects. In varying embodiments, pre-processing and post-processing is performed to further clean the source video and the video objects defined, respectively.
Inventors
- 👤 Jinhui Pan
- 👤 Shipeng Li
- 👤 Ya-Qin Zhang
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-10 02:16
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/10; G06T7/11; G06T7/174; G06T7/187; G06T7/20; G06T7/215; G06T7/90; G06V; G06V10/00; G06V10/20; G06V10/26; G06V10/267; G06T2207/00; G06T2207/10; G06T2207/10016; G06T2207/30; G06T2207/30221; G06T2207/30224
Claims Analysis
Total Claims: 35
Claim 1
Independent Claim
115 words
A computer implemented method for extracting objects from a source video comprising: performing color segmentation on a frame of the source video to segment the video to a first precision by substantially uniform color regions thereof; performing motion segmentation on a plurality of frames of the source video to segment the video to a second precision by moving regions thereof; and, combining at least the moving regions resulting from the motion segmentation and the substantially uniform color regions resulting from the color segmentation to define the objects of the source video to a third precision; wherein the third precision is greater than the first precision, and the first precision is greater than the second precision.
Claim 2
Dependent Claim
115 words
A computer implemented method for extracting objects from a source video comprising: performing color segmentation on a frame of the source video to segment the video to a first precision by substantially uniform color regions thereof; performing motion segmentation on a plurality of frames of the source video to segment the video to a second precision by moving regions thereof; and, combining at least the moving regions resulting from the motion segmentation and the substantially uniform color regions resulting from the color segmentation to define the objects of the source video to a third precision; wherein the third precision is greater than the first precision, and the first precision is greater than the second precision.
Claim 3
Dependent Claim
24 words
The method of claim 1 , further comprising pre processing the frame of the source video to remove noise prior to performing color segmentation.
Claim 4
Dependent Claim
27 words
The method of claim 2 , wherein pre processing the frame comprises applying a median filter on each of a plurality of color channels of the frame.
Claim 5
Dependent Claim
34 words
The method of claim 1 , further comprising, prior to combining at least the moving regions and the substantially uniform color regions, generating a frame difference mask from two frames of the source video.
Claim 6
Dependent Claim
49 words
The method of claim 1 , wherein the performing motion segmentation comprises performing motion segmentation on the at least three frames of the plurality of frames of the source video to produce at least two motion segmentation masks that are used to segment the video by moving regions thereof.
Claim 7
Dependent Claim
22 words
The method of claim 1 , further comprising post processing the objects of the source video to remove noise from the objects.
Claim 8
Dependent Claim
22 words
The method of claim 6 , wherein post processing the objects comprises removing small holes from the objects to generate integrated objects.
Claim 9
Dependent Claim
26 words
The method of claim 6 , wherein post processing the objects comprises removing small regions from background areas of the source video to generate cleaner objects.
Claim 10
Dependent Claim
22 words
The method of claim 1 , wherein performing color segmentation defines substantially precise boundaries of the objects extracted from the source video.
Claim 11
Dependent Claim
29 words
The method of claim 1 , wherein performing color segmentation comprises growing the substantially uniform color regions by growing each of a plurality of seed pixels of the frame.
Claim 12
Dependent Claim
27 words
The method of claim 10 , wherein performing color segmentation comprises merging smaller of the substantially uniform color regions with larger of the substantially uniform color regions.
Claim 13
Dependent Claim
21 words
The method of claim 1 , wherein performing motion segmentation defines approximate boundaries of the objects extracted from the source video.
Claim 14
Dependent Claim
126 words
At least one machine readable medium having instructions stored thereon for execution by a processor to perform a method comprising: performing color segmentation on at least one frame of a plurality of frames of video to segment the video to a first precision by substantially uniform color regions; performing motion segmentation on at least two frames of the plurality of frames of the video to segment the video to a second precision by moving regions; and, combining the substantially uniform color regions resulting from the color segmentation and the moving regions resulting from the motion segmentation to define objects of the video to a third precision; wherein the third precision is greater than the first precision, and the first precision is greater than the second precision.
Claim 15
Dependent Claim
126 words
At least one machine readable medium having instructions stored thereon for execution by a processor to perform a method comprising: performing color segmentation on at least one frame of a plurality of frames of video to segment the video to a first precision by substantially uniform color regions; performing motion segmentation on at least two frames of the plurality of frames of the video to segment the video to a second precision by moving regions; and, combining the substantially uniform color regions resulting from the color segmentation and the moving regions resulting from the motion segmentation to define objects of the video to a third precision; wherein the third precision is greater than the first precision, and the first precision is greater than the second precision.
Claim 16
Dependent Claim
36 words
The at least one machine readable medium of claim 13 , wherein the performing color segmentation comprises at least one of edge detection, a region technique, a Maximum A posteriori Probability (MAP) technique, and pixel growth.
Claim 17
Dependent Claim
32 words
The at least one machine readable medium of claim 13 , wherein the performing color segmentation comprises obtaining a seed pixel and using the seed pixel to grow a neighborhood of pixels.
Claim 18
Dependent Claim
31 words
The at least one machine readable medium of claim 13 , wherein the performing color segmentation comprises generating substantially homogenous color regions responsive to maximum and minimum values for the regions.
Claim 19
Dependent Claim
37 words
The at least one machine readable medium of claim 13 , wherein the performing color segmentation comprises generating substantially homogenous color regions responsive to maximum and minimum values for the regions and at least one predetermined threshold.
Claim 20
Dependent Claim
34 words
The at least one machine readable medium of claim 13 , wherein the performing color segmentation comprises merging relatively smaller color regions by integrating the relatively smaller color regions into relatively larger color regions.
Claim 21
Dependent Claim
49 words
The at least one machine readable medium of claims 13 , wherein the performing motion segmentation comprises at least one of an optical flow technique, a block-based matching technique, a Maximum A posteriori Probability (MAP) technique, a simultaneous motion estimation and segmentation technique, and a region-based motion vector approach.
Claim 22
Dependent Claim
43 words
The at least one machine readable medium of claim 13 , wherein the performing motion segmentation comprises determining motion vectors for a plurality of regions with respect to a first frame and a subsequent frame of the plurality of frames of the video.
Claim 23
Dependent Claim
40 words
The at least one machine readable medium of claim 20 , wherein the motion vectors are determined with regard to the substantially uniform color regions resulting from the color segmentation or to a precursor of the substantially uniform color regions.
Claim 24
Dependent Claim
47 words
The at least one machine readable medium of claim 13 , wherein the performing motion segmentation comprises (i) determining motion vectors for a plurality of regions using a matching error minimization technique and (ii) determining a motion mask using the motion vectors and at least one threshold.
Claim 25
Dependent Claim
37 words
The at least one machine readable medium of claim 13 , wherein the performing color segmentation comprises determining at least one color segmentation mask and the performing motion segmentation comprises determining at least one motion segmentation mask.
Claim 26
Dependent Claim
38 words
The at least one machine readable medium of claim 13 , wherein the combining the substantially uniform color regions and the moving regions comprises performing a mapping operation using the substantially uniform color regions and the moving regions.
Claim 27
Dependent Claim
55 words
The at least one machine readable medium of claim 24 , wherein the performing the mapping operation comprises mapping an entirety of a particular substantially uniform color region to a particular moving region if a percentage of the particular substantially uniform color region that is assignable to the particular moving region exceeds a predetermined threshold.
Claim 28
Dependent Claim
49 words
The at least one machine readable medium of claim 13 , wherein the method further comprises performing a frame difference operation to produce a frame difference mask that reflects an amount of difference between a first frame and a subsequent frame of the plurality of frames of the video.
Claim 29
Dependent Claim
45 words
The at least one machine readable medium of claim 26 , wherein the combining the substantially uniform color regions and the moving regions comprises combining the substantially uniform color regions, the moving regions, and the frame difference mask to define the objects of the video.
Claim 30
Dependent Claim
59 words
The at least one machine readable medium of claim 13 , wherein the combining the substantially uniform color regions and the moving regions comprises combining (i) the substantially uniform color regions, (ii) the moving regions, and (iii) one or more of a frame difference mask, a depth mask, and a texture mask to define the objects of the video.
Claim 31
Dependent Claim
32 words
The at least one machine readable medium of claim 13 , wherein the method further comprises: pre-processing the video prior to the performing color segmentation; and post-processing the objects of the video.
Claim 32
Dependent Claim
146 words
At least one machine readable medium having instructions stored thereon for execution by a processor to perform a method comprising: generating a color segmentation mask that defines contours of objects of a video to a first precision based on colors of the video; generating a motion segmentation mask that defines the contours of the objects of the video to a second precision based on motions of the video; generating a frame difference mask that reflects differences in the video between a first frame and a second frame of the video; and combining the color segmentation mask, the motion segmentation mask, and the frame difference mask to define the objects of the video; wherein the combining defines the objects of the video to a third precision; and wherein the third precision is greater than the first precision, and the first precision is greater than the second precision.
Claim 33
Dependent Claim
146 words
At least one machine readable medium having instructions stored thereon for execution by a processor to perform a method comprising: generating a color segmentation mask that defines contours of objects of a video to a first precision based on colors of the video; generating a motion segmentation mask that defines the contours of the objects of the video to a second precision based on motions of the video; generating a frame difference mask that reflects differences in the video between a first frame and a second frame of the video; and combining the color segmentation mask, the motion segmentation mask, and the frame difference mask to define the objects of the video; wherein the combining defines the objects of the video to a third precision; and wherein the third precision is greater than the first precision, and the first precision is greater than the second precision.
Claim 34
Dependent Claim
117 words
At least one machine readable medium comprising: means for performing color segmentation on at least one frame of a plurality of frames of video to segment the video by substantially uniform color regions to a middle precision; means for performing motion segmentation on at least two frames of the plurality of frames of the video to segment the video by moving regions to a precision that is lower than the middle precision; and, means for combining the substantially uniform color regions produced by the means for performing color segmentation and the moving regions produced by the means for performing motion segmentation to define objects of the video to another precision that is greater than the middle precision.
Claim 35
Dependent Claim
117 words
At least one machine readable medium comprising: means for performing color segmentation on at least one frame of a plurality of frames of video to segment the video by substantially uniform color regions to a middle precision; means for performing motion segmentation on at least two frames of the plurality of frames of the video to segment the video by moving regions to a precision that is lower than the middle precision; and, means for combining the substantially uniform color regions produced by the means for performing color segmentation and the moving regions produced by the means for performing motion segmentation to define objects of the video to another precision that is greater than the middle precision.
References Cited (Backward Citations)
US5479218A
Cited
US6005625A
Cited
US5608458A
Cited
US5995668A
Cited
US5936671A
Cited
US6075875A
Cited
US6337917B1
Cited
WO1998033323A1
Cited
US6035060A
Cited
US6141434A
Cited
US6348918B1
Cited
US6266443B1
Cited
US6400846B1
Cited
US6625333B1
Cited
US6421090B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2007-08-31 | 2008-03-01 | 2008-08-31 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2011-08-31 | 2012-03-01 | 2012-08-31 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2015-08-31 | 2016-03-01 | 2016-08-31 | $7700.00 | Window Closed / Lapsed |