US6819797B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for classifying and querying temporal and spatial information in video
Assignee: Sinoeast Concept Ltd
•
Filing: 2000-01-28
•
Granted: 2004-11-16
•
Expires: 2020-01-28
Abstract
Abstract A method and apparatus for classifying or annotating image and video content by assigning each class of event, action, region or object a unique symbol and then building symbol strings to represent sequences in space or time. The symbol strings can be decoded using a library of annotated PT descriptors to automatically label the image and video content. Furthermore, the PT descriptors can be used for searching by sketch or searching by example where the search query input is converted to symbol strings, which are efficiently compared based on the presence and relative counts of PT descriptors mapped into a PT matrices. The Precedence Template (PT) descriptor can be used for classifying and querying video based on the spatial and temporal orderings of regions, objects, actions or events. Applied to video, the PT descriptors provide a way to compare the temporal order of events, actions, or objects such as those represented in a scene transition graph, key-frame list, or event string. Applied to images, the PT descriptors provide a way to compare the spatial arrangement of image regions or objects. By capturing the spatial and temporal relationships statistically, the PT descriptors provide a robust way to measure similarity in the presence of insertions, deletions, substitutions, replications and relocations of events, actions, regions or objects.
Inventors
- 👤 John R. Smith
- 👤 Chung-Sheng Li
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 13:41
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/70; G06F16/78; G06F16/783; G06F16/7847; G06F16/786; G06F16/71; G06F16/73; G06F16/732; G06F16/7328; G06F16/7335; G06F16/785; G06V; G06V10/00; G06V10/40; G06V10/42; G06V10/422; G06V10/424; G06V20/00; G06V20/40; G06V20/46; Y; Y10; Y10S; Y10S707/00; Y10S707/99931; Y10S707/99933; Y10S707/99941; Y10S707/99944; Y10S707/99945
Claims Analysis
Total Claims: 29
Claim 1
Independent Claim
52 words
A method for generating descriptors for media content comprising the steps of: assigning a plurality of content symbols to each of a plurality of content components; ordering the plurality of content symbols based on their relationships to each other in the content; and evaluating the ordering of said plurality of content symbols.
Claim 2
Independent Claim
52 words
A method for generating descriptors for media content comprising the steps of: assigning a plurality of content symbols to each of a plurality of content components; ordering the plurality of content symbols based on their relationships to each other in the content; and evaluating the ordering of said plurality of content symbols.
Claim 3
Dependent Claim
47 words
The method of claim 1 wherein said evaluating the ordering comprises the steps of: determining the frequency of each of said plurality of content symbols having a specific relationship to another of said plurality of content symbols; and assigning content symbol relationship values based on said frequency.
Claim 4
Dependent Claim
15 words
The method of claim 2 further comprising mapping said content relationship values into a matrix.
Claim 5
Dependent Claim
26 words
The method of claim 3 further comprising comparing content relationship values for a plurality of media content and classifying said media content based on said comparing.
Claim 6
Dependent Claim
80 words
The method of claim 4 wherein said classifying comprises the steps of: identifying a plurality of classes for classifying media content; determining the frequency of each of said plurality of content symbols having a specific relationship to another of said plurality of content symbols within each of said plurality of classes; assigning class symbol relationship values based on said frequency; and forming a pooled class matrix for each of the plurality of classes based on said class symbol relationship values.
Claim 7
Dependent Claim
17 words
The method of claim 4 further comprising storing content relationship values for a plurality of media content.
Claim 8
Dependent Claim
18 words
The method of claim 6 further comprising searching said stored content relationship values for similarity to query input.
Claim 9
Dependent Claim
100 words
The method of claim 7 wherein said searching comprises the steps of: receiving search input; assigning a plurality of query symbols to each of a plurality of query components; and ordering the plurality of query symbols based on their relationship in the query content; evaluating the ordering of said plurality of query symbols by determining the frequency of each of said plurality of query symbols having a specific relationship to another of said plurality of query symbols and assigning query symbol relationship values based on said frequency; and comparing the query symbol relationship values to the content symbol relationship values.
Claim 10
Dependent Claim
13 words
The method of claim 8 wherein receiving search input comprises receiving a sketch.
Claim 11
Dependent Claim
25 words
The method of claim 8 wherein receiving search input comprises providing a query grid comprising query options and receiving user input onto said query grid.
Claim 12
Dependent Claim
27 words
The method of claim 8 wherein said comparing comprises comparing at least one matrix of content symbol relationship values with a matrix of query symbol relationship values.
Claim 13
Dependent Claim
14 words
The method of claim 1 wherein the ordering comprises spatial ordering of image content.
Claim 14
Dependent Claim
21 words
The method of claim 1 wherein the ordering comprises at least one of spatial ordering and temporal ordering of video content.
Claim 15
Independent Claim
63 words
A system for providing descriptors for media content comprising: a symbol table component for assigning a plurality of content symbols to each of a plurality of content components; a symbol string component for ordering the plurality of content symbols based on their relationships to each other in the content; and a processing component for evaluating the ordering of said plurality of content symbols.
Claim 16
Independent Claim
63 words
A system for providing descriptors for media content comprising: a symbol table component for assigning a plurality of content symbols to each of a plurality of content components; a symbol string component for ordering the plurality of content symbols based on their relationships to each other in the content; and a processing component for evaluating the ordering of said plurality of content symbols.
Claim 17
Dependent Claim
55 words
The system of claim 14 wherein the processing component for evaluating the ordering comprises: an analysis component for determining the frequency of each of said plurality of content symbols having a specific relationship to another of said plurality of content symbols; and a value component for assigning content symbol relationship values based on said frequency.
Claim 18
Dependent Claim
19 words
The system of claim 15 further comprising at least one storage component for storing the content symbol relationship values.
Claim 19
Dependent Claim
14 words
The system of claim 16 further comprising search input interface for receiving query content.
Claim 20
Dependent Claim
127 words
The system of claim 17 further comprising a search engine comprising: a symbol table component for assigning a plurality of query symbols to each of a plurality of query components; a symbol string component for ordering the plurality of query symbols based on their relationships to each other in the query content; a processing component for evaluating the ordering of said plurality of query symbols, including an analysis component for determining the frequency of each of said plurality of query symbols having a specific relationship to another of said plurality of query symbols; and a value component for assigning query symbol relationship values based on said frequency; and a comparator component for comparing query symbol relationship values to content symbol relationship values stored in said storage component.
Claim 21
Dependent Claim
28 words
The system of claim 18 wherein said comparator component compares at least one matrix of stored content symbol relationship values with a matrix of query symbol relationship values.
Claim 22
Dependent Claim
66 words
The system of claim 18 further comprising a classification component for evaluating the frequency of each of said plurality of content symbols having a specific relationship with another of said plurality of content symbols within a defined class and for assigning class symbol relationship values based on said frequency and wherein said comparator additionally compares said class symbol relationship values with said query symbol relationship values.
Claim 23
Dependent Claim
19 words
The system of claim 17 wherein said search input interface comprises a graphical input device for receiving a sketch.
Claim 24
Dependent Claim
31 words
The system of claim 17 wherein said search input interface comprises display processing means for providing a query grid comprising query options and for receiving user input onto said query grid.
Claim 25
Dependent Claim
49 words
The system of claim 14 further comprising a classification component for evaluating the frequency of each of said plurality of content symbols having a specific relationship with another of said plurality of content symbols within a defined class and for assigning class symbol relationship values based on said frequency.
Claim 26
Dependent Claim
97 words
A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for generating descriptors for media content, said method comprising the steps of: assigning a plurality of content symbols to each of a plurality of content components; ordering the plurality of content symbols based on their relationships to each other in the content; and evaluating the ordering of said plurality of content symbols by determining the frequency of each of said plurality of content symbols having a specific relationship to another of said plurality of content symbols.
Claim 27
Dependent Claim
97 words
A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for generating descriptors for media content, said method comprising the steps of: assigning a plurality of content symbols to each of a plurality of content components; ordering the plurality of content symbols based on their relationships to each other in the content; and evaluating the ordering of said plurality of content symbols by determining the frequency of each of said plurality of content symbols having a specific relationship to another of said plurality of content symbols.
Claim 28
Dependent Claim
61 words
An improved descriptor stored on a program storage device readable by machine for describing and handling media content which can be characterized by a plurality of content symbols, the improvement comprising a content symbol relationship value assigned based on the frequency of each of said plurality of content symbols having a specific relationship to another of said plurality of content symbols.
Claim 29
Dependent Claim
61 words
An improved descriptor stored on a program storage device readable by machine for describing and handling media content which can be characterized by a plurality of content symbols, the improvement comprising a content symbol relationship value assigned based on the frequency of each of said plurality of content symbols having a specific relationship to another of said plurality of content symbols.
References Cited (Backward Citations)
US4799270A
Cited
US5357250A
Cited
US5684999A
Cited
US5923380A
Cited
US6400853B1
Cited
US6253201B1
Cited
US6564263B1
Cited
US6446060B1
Cited
US6490370B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2007-11-16 | 2008-05-17 | 2008-11-16 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2011-11-16 | 2012-05-17 | 2012-11-16 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2015-11-16 | 2016-05-17 | 2016-11-16 | $7700.00 | Window Closed / Lapsed |