Patent US6859554B2 successfully refreshed from Google Patents.
US6859554B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method for segmenting multi-resolution video objects
Assignee: Mitsubishi Electric Research Laboratories Inc
•
Filing: 2001-04-04
•
Granted: 2005-02-22
•
Expires: 2021-04-04
Abstract
Abstract A method for segmenting video objects in a video sequence that is composed of frames including pixels first assigns a feature vector to each pixel of the video. Next, selected pixels are identified as marker pixels. Pixels adjacent to each marker pixel are assembled into a corresponding a volume of pixels if the distance between the feature vector of the marker pixel and the feature vector of the adjacent pixels is less than a first predetermined threshold. After all pixels have been assembled into volumes, a first score and descriptors are assigned to each volume. At this point, each volume represents a segmented video object. The volumes are then sorted a high-to-low order according to the first scores, and further processed in the high-to-low order. Second scores, dependent on the descriptors of pairs of volumes are determined. The volumes are iteratively combined if the second score passes a second threshold to generate a video object in a resolution video object tree that completes when the combined volume or video object is the entire video.
Inventors
- 👤 Fatih M. Porikli
- 👤 Yao Wang
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:36
CPC Classifications
G; G06; G06T; G06T9/00; G06T9/001; G06T7/00; G06T7/10; G06T7/11; G06T7/187; G06T7/20; G06T7/215; G06T9/20; G06V; G06V10/00; G06V10/20; G06V10/26; G06V10/267; G06T2207/00; G06T2207/10; G06T2207/10016; G06T2207/20; G06T2207/20016; G06T2207/20112; G06T2207/20156
Claims Analysis
Total Claims: 14
Claim 1
Independent Claim
172 words
A method for segmenting a video including a plurality of pixels into a plurality of video objects, comprising: assigning a feature vector to each pixel of the video; identifying selected pixels of the video as marker pixels; assembling each marker pixel and pixels adjacent to the marker pixel into a corresponding volume if the distance between the feature vector of the marker pixel and the feature vector of the adjacent pixels is less than a first predetermined threshold; assigning a first score and descriptors to each volume; sorting the volumes in a high-to-low order according to the first scores; and processing the volumes in the high-to-low order, the processing for each volume comprising: comparing the descriptor of the volume to the descriptor of an adjacent volume to determine a second score; combining the volume with the adjacent volume if the second score passes a second threshold to generate a video object in a multi-resolution video object tree; and repeating the comparing and combining steps until a single volume representing the video remains.
Claim 2
Independent Claim
172 words
A method for segmenting a video including a plurality of pixels into a plurality of video objects, comprising: assigning a feature vector to each pixel of the video; identifying selected pixels of the video as marker pixels; assembling each marker pixel and pixels adjacent to the marker pixel into a corresponding volume if the distance between the feature vector of the marker pixel and the feature vector of the adjacent pixels is less than a first predetermined threshold; assigning a first score and descriptors to each volume; sorting the volumes in a high-to-low order according to the first scores; and processing the volumes in the high-to-low order, the processing for each volume comprising: comparing the descriptor of the volume to the descriptor of an adjacent volume to determine a second score; combining the volume with the adjacent volume if the second score passes a second threshold to generate a video object in a multi-resolution video object tree; and repeating the comparing and combining steps until a single volume representing the video remains.
Claim 3
Dependent Claim
34 words
The method of claim 1 wherein each pixel has spatial (x,y) and time (t) coordinates to indicate a location of the pixel and the volumes in a spatial-temporal collocated overlapping scene of the video.
Claim 4
Dependent Claim
52 words
The method of claim 2 wherein the video includes a plurality of frames and further comprising: projecting a portion of caclivideo object in a particular frame to intersect the projection of the video object in an adjacent frame to provide continuous silhouettes of the video object according to the time t coordinates.
Claim 5
Dependent Claim
24 words
The method of claim 3 further comprising: applying a spatial-domain 2D median filter to the frames to remove intensity singularities, without disturbing edge formation.
Claim 6
Dependent Claim
31 words
The method of claim 1 further comprising: partitioning the video into a plurality of identically sized volumes; and selecting the pixel at the center of each volume as the marker pixels.
Claim 7
Dependent Claim
50 words
The method of claim 1 further comprising: determining a gradient magnitude ∇V=∂V/∂x+∂V/∂y+∂V/∂t for each pixel in the video; selecting the pixel with a minimwn gradient magnitude as the marker pixel; removing pixel in a predetermined neighborhood around the marker; and repeating the selecting and removing steps until no pixd remain.
Claim 8
Dependent Claim
19 words
The method of claim of claim 1 wherein the feature vector is based on a color of the pixel.
Claim 9
Dependent Claim
17 words
The method of claim 1 further comprising: merging volumes less than minimum size with an adjacent volumes.
Claim 10
Dependent Claim
20 words
The method of claim 8 wherein the minimum size is less than 0.00 1 of the volume representing the video.
Claim 11
Dependent Claim
49 words
The method of claim 9 further comprising: sorting the volumes in an increasing order to size; processing the volumes in the increasing order, the processing for each volume comprising: including each pixel of the volume in a closest volume until all volumes less than the minimum size are processed.
Claim 12
Dependent Claim
24 words
The method of claim 1 wherein the descriptors include self descriptors of the volume, and nmtual descriptors of the volume and the adjacent volume.
Claim 13
Independent Claim
124 words
A method for segmenting a video sequence of frames, each frame including a plurality of pixels, comprising: partitioning all of the pixels of all frames of the video into aplurality of volumes according to features of each pixel, the pixels of each volume having frame-based spatial coordinates and sequence-based temporal coordinates; assigning descriptors to each volume; representing each volume as a video object at a lowest level in a multi-resolution video object tree; and iteratively combining volumes according to the descriptors, and representing each combined volurm as a video object at intermediate levels of the multi-resolution video object tree, until all of the combined volumes form the entire video represented as a video object at a highest level of the multi-resolution video object tree.
Claim 14
Independent Claim
124 words
A method for segmenting a video sequence of frames, each frame including a plurality of pixels, comprising: partitioning all of the pixels of all frames of the video into aplurality of volumes according to features of each pixel, the pixels of each volume having frame-based spatial coordinates and sequence-based temporal coordinates; assigning descriptors to each volume; representing each volume as a video object at a lowest level in a multi-resolution video object tree; and iteratively combining volumes according to the descriptors, and representing each combined volurm as a video object at intermediate levels of the multi-resolution video object tree, until all of the combined volumes form the entire video represented as a video object at a highest level of the multi-resolution video object tree.
References Cited (Backward Citations)
US5502510A
Cited
US5604822A
Cited
US5592231A
Cited
US5978031A
Cited
US5793895A
Cited
US5852683A
Cited
US6055330A
Cited
US5832115A
Cited
US6452637B1
Cited
US20010038718A1
Cited
US6496228B1
Cited
US6529202B2
Cited
US6112112A
Cited
US6603484B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2008-02-22 | 2008-08-23 | 2009-02-22 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2012-02-22 | 2012-08-23 | 2013-02-22 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2016-02-22 | 2016-08-23 | 2017-02-22 | $7700.00 | Window Closed / Lapsed |