US20030195883A1
Expired / Lapsed
US
⏱️ 0.0 years remaining
System and method for measuring image similarity based on semantic meaning
Assignee: Wrp Ip Management LLC
•
Filing: 2002-04-15
•
Granted: —
•
Expires: 2022-04-15
Abstract
Abstract A method and system for determining image similarity is based on the semantic meaning of images. The method includes deriving a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories; for each semantic category, forming a set of the perceptual features comprising required features and frequently occurring features; comparing an image to said semantic categories; and classifying said image as belonging to one of said semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in said image. A database contains image information, where the image information includes at least one of already classified images, network locations of already classified images and documents containing already classified images. The database is searched for images matching an input query, that can comprise an image, text, or both. The images stored in or referenced by the database may be obtained at least in part through the Internet, such as by the activity of an automated web crawler. In the presently preferred embodiment the images are medical images, and the medical database may be searched for at least one image that satisfies thresholds established by a search query. Also disclosed is a method and system for searching the Internet. The method includes (a) providing at least one server accessible by a plurality of clients; (b) downloading a program to a client, where the program includes an Internet searching program operable for locating at least one type of data; and (c) executing the downloaded program on the client, and returning to the server URLs corresponding to located data.
Inventors
- 👤 Aleksandra Mojsilovic
- 👤 Bernice Rogowitz
- 👤 Jose Gomes
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:25
CPC Classifications
G; G16; G16H; G16H50/00; G16H50/20; G06; G06F; G06F16/00; G06F16/50; G06F16/58; G06F16/583; G06F16/5838; G06F16/5854; G06F16/5862; G06F18/00; G06F18/20; G06F18/22; G06V; G06V10/00; G06V10/40; G06V20/00; G06V20/10; G06V30/00; G06V30/10; G06V30/26; G06V30/262; G06V30/274; G16H30/00; G16H30/20; G16H30/40; G16Z; G16Z99/00; Y; Y10; Y10S; Y10S707/00; Y10S707/99931; Y10S707/99933; Y10S707/99936; Y10S707/99941; Y10S707/99944; Y10S707/99945
Claims Analysis
Total Claims: 64
Claim 1
Independent Claim
121 words
1 . A computer implemented method for characterizing images and determining image similarity based on semantic meaning of images, comprising: deriving a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories; for each semantic category, forming a set of the perceptual features comprising required features and frequently occurring features; comparing an image to said semantic categories; and classifying said image as belonging to one of said semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in said image.
Claim 2
Dependent Claim
121 words
1 . A computer implemented method for characterizing images and determining image similarity based on semantic meaning of images, comprising: deriving a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories; for each semantic category, forming a set of the perceptual features comprising required features and frequently occurring features; comparing an image to said semantic categories; and classifying said image as belonging to one of said semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in said image.
Claim 3
Dependent Claim
51 words
2 . A method as in claim 1 , where a database stores image information, where the image information comprises at least one of already classified images, network locations of already classified images and documents containing already classified images, and where said database is searched for images matching an input query.
Claim 4
Dependent Claim
38 words
3 . A method as in claim 2 , where said input query comprises an input image, further comprising extracting features from said input image and applying a metric to determine the semantic category for said input image.
Claim 5
Dependent Claim
76 words
4 . A method as in claim 2 , where said input query comprises an input image, further comprising processing said input image and comparing said input image to each semantic category using a metric that computes a similarity measure between the features used to describe the semantic category and corresponding features extracted from said input image; and assigning said input image to the semantic category that corresponds to a highest value of the similarity measure.
Claim 6
Dependent Claim
22 words
5 . A method as in claim 2 , where said image information is obtained at least in part through the Internet.
Claim 7
Dependent Claim
15 words
6 . A method as in claim 1 , where said images are medical images.
Claim 8
Dependent Claim
15 words
7 . A method as in claim 5 , where said images are medical images.
Claim 9
Dependent Claim
33 words
8 . A method as in claim 2 , further comprising searching said database for at least one image or image-related document that satisfies at least one threshold established by said search query.
Claim 10
Dependent Claim
34 words
9 . A method as in claim 2 , where the database is located at a remote location and is searched by issuing a query to a database server through a data communications network.
Claim 11
Dependent Claim
22 words
10 . A method as in claim 9 , and further comprising retrieving an image or image-related document from the remote database.
Claim 12
Dependent Claim
53 words
11 . A method as in claim 2 , where the database is located at a remote location and is searched by issuing a query to a database server, through a data communications network, in conjunction with a text-based search algorithm executed by said server to retrieve a multi-media object from the database.
Claim 13
Dependent Claim
39 words
12 . A method as in claim 2 , where the database comprises image information obtained from a plurality of autonomous searchers that execute in client computers and that return URLs corresponding to located image data and image-related documents.
Claim 14
Dependent Claim
23 words
13 . A method as in claim 1 , further comprising segmenting an image by employing texture segmentation, color segmentation and foreground/background segmentation.
Claim 15
Dependent Claim
17 words
14 . A method as in claim 13 , where texture segmentation includes forming a texture map.
Claim 16
Dependent Claim
17 words
15 . A method as in claim 14 , where color segmentation includes forming a region map.
Claim 17
Dependent Claim
39 words
16 . A method as in claim 15 , where foreground/background segmentation includes using the texture map and the region map to determine if there is at least one dominant object in the image, and forming an object map.
Claim 18
Dependent Claim
64 words
17 . A method as in claim 16 , where the image, texture map, region map and object map are further processed to compute, for each region in the region map and for each object in the object map, a set of local features comprising information descriptive of size, boundary, neighbors, boundary curvature, texture, mean color, color name, and shape properties for that region/object.
Claim 19
Dependent Claim
25 words
18 . A method as in claim 13 , where local features are analyzed to compute regional features for indicating a presence of semantic cues.
Claim 20
Dependent Claim
31 words
19 . A method as in claim 18 , where said semantic cues comprise at least one of skin, sky, grass, snow, nature, water, man-made objects, man-made blobs, flowers and texture.
Claim 21
Dependent Claim
26 words
20 . A method as in claim 13 , where local features are combined to compute global features that describe a global appearance of the image.
Claim 22
Dependent Claim
54 words
21 . A method as in claim 20 , where said global features comprise at least one of number of details, histogram of color names, description of color composition, number of regions, number of blobs, number of objects, number of straight lines, number of regular curves, energy, spatial activity, symmetry and measures of contrast.
Claim 23
Dependent Claim
176 words
22 . A data processing system comprising a data processor, a graphical user interface and a memory that stores a database containing image information, where the image information comprises at least one of already classified images, network locations of already classified images and documents containing already classified images, said data processor operating in accordance with a stored program for classifying images by deriving a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories; said data processor, for each semantic category, forming a set of the perceptual features comprising required features and frequently occurring features; comparing an image to said semantic categories; and classifying said image as belonging to one of said semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in said image, said data processor storing image-related classification data in said database.
Claim 24
Dependent Claim
176 words
22 . A data processing system comprising a data processor, a graphical user interface and a memory that stores a database containing image information, where the image information comprises at least one of already classified images, network locations of already classified images and documents containing already classified images, said data processor operating in accordance with a stored program for classifying images by deriving a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories; said data processor, for each semantic category, forming a set of the perceptual features comprising required features and frequently occurring features; comparing an image to said semantic categories; and classifying said image as belonging to one of said semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in said image, said data processor storing image-related classification data in said database.
Claim 25
Dependent Claim
22 words
23 . A data processing system as in claim 22 , where said database is searched for images matching an input query.
Claim 26
Dependent Claim
42 words
24 . A data processing system as in claim 23 , where said input query comprises an input image, and where data processor extracts features from said input image and applies a metric to determine the semantic category for said input image.
Claim 27
Dependent Claim
81 words
25 . A data processing system as in claim 23 , where said input query comprises an input image, and where said data processor processes said input image and compares said input image to each semantic category using a metric that computes a similarity measure between the features used to describe the semantic category and corresponding features extracted from said input image; and assigns said input image to the semantic category that corresponds to a highest value of the similarity measure.
Claim 28
Dependent Claim
24 words
26 . A data processing system as in claim 22 , where said image information is obtained at least in part through the Internet.
Claim 29
Dependent Claim
21 words
27 . A data processing system as in claim 22 , where said image information is comprised of medical image information.
Claim 30
Dependent Claim
37 words
28 . A data processing system as in claim 22 , where said image information comprises data obtained from a plurality of autonomous searchers that execute in client computers and that return URLs corresponding to located images.
Claim 31
Dependent Claim
32 words
29 . A data processing system as in claim 23 , where said data processor searches said database for locating at least one image that satisfies thresholds established by said search query.
Claim 32
Dependent Claim
36 words
30 . A data processing system as in claim 23 , where the database is located at a remote location and is searched by issuing a query to a database server through a data communications network.
Claim 33
Dependent Claim
22 words
31 . A data processing system as in claim 30 , and further comprising retrieving an image through said data communications network.
Claim 34
Dependent Claim
52 words
32 . A data processing system as in claim 23 , where said database is located at a remote location and is searched by issuing a query to a database server, through a data communications network, in conjunction with a text-based search algorithm executed by said server to retrieve a multi-media object.
Claim 35
Dependent Claim
33 words
33 . A data processing system as in claim 23 , where said image information comprises medical image information and is searched by issuing a query to a database server through the Internet.
Claim 36
Dependent Claim
28 words
34 . A data processing system as in claim 23 , where said data processor segments an input image by employing texture segmentation, color segmentation and foreground/background segmentation.
Claim 37
Dependent Claim
19 words
35 . A data processing system as in claim 34 , where texture segmentation includes forming a texture map.
Claim 38
Dependent Claim
19 words
36 . A data processing system as in claim 35 , where color segmentation includes forming a region map.
Claim 39
Dependent Claim
42 words
37 . A data processing system as in claim 36 , where foreground/background segmentation includes using the texture map and the region map to determine if there is at least one dominant object in the image, and to form an object map.
Claim 40
Dependent Claim
66 words
38 . A data processing system as in claim 37 , where the input image, texture map, region map and object map are further processed to compute for each region in the region map and for each object in the object map, a set of local features comprising information concerning size, boundary, neighbors, boundary curvature, texture, mean color, color name, and shape properties for that region/object.
Claim 41
Dependent Claim
26 words
39 . A data processing system as in claim 34 , where local features are analyzed to compute regional features, indicating a presence of semantic cues.
Claim 42
Dependent Claim
33 words
40 . A data processing system as in claim 39 , where said semantic cues comprise at least one of skin, sky, grass, snow, nature, water, man-made objects, man-made blobs, flowers and texture.
Claim 43
Dependent Claim
28 words
41 . A data processing system as in claim 39 , where local features are combined to compute global features that describe a global appearance of the image.
Claim 44
Dependent Claim
56 words
42 . A data processing system as in claim 41 , where said global features comprise at least one of number of details, histogram of color names, description of color composition, number of regions, number of blobs, number of objects, number of straight lines, number of regular curves, energy, spatial activity, symmetry and measures of contrast.
Claim 45
Dependent Claim
136 words
43 . A computer program embodied on a computer readable media for directing a computer to execute a method for processing digitally represented images, comprising program instructions for classifying images by deriving a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories, where for each semantic category said program instructions form a set of the perceptual features comprising required features and frequently occurring features, compares an image to said semantic categories and classifies said image as belonging to one of said semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in said image.
Claim 46
Dependent Claim
136 words
43 . A computer program embodied on a computer readable media for directing a computer to execute a method for processing digitally represented images, comprising program instructions for classifying images by deriving a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories, where for each semantic category said program instructions form a set of the perceptual features comprising required features and frequently occurring features, compares an image to said semantic categories and classifies said image as belonging to one of said semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in said image.
Claim 47
Dependent Claim
42 words
44 . A computer program as in claim 43 , where classified images are stored in a database, and where classified images or network locations of classified images are added to said database after being assigned to one of said semantic categories.
Claim 48
Dependent Claim
22 words
45 . A computer program as in claim 43 , were said images are obtained at least in part through the Internet.
Claim 49
Dependent Claim
31 words
46 . A computer program as in claim 44 , further comprising searching said database for at least one image or image-containing document that satisfies thresholds established by a search query.
Claim 50
Dependent Claim
117 words
47 . An Internet searching tool, comprising: an Internet searching robot comprising at least one autonomous searcher that locates images and returns at least one of image location data and image-related document location data, said searcher executing in a client computer; a processor responsive to returned image location data and image-related document location data for accessing a located image for segmenting the located image and for extracting features from the located image for obtaining a set of image metadata; storage for storing the set of image metadata; and a user interface for entering a user query for use in searching the storage and for returning an image having image metadata that is relevant to the user query.
Claim 51
Dependent Claim
117 words
47 . An Internet searching tool, comprising: an Internet searching robot comprising at least one autonomous searcher that locates images and returns at least one of image location data and image-related document location data, said searcher executing in a client computer; a processor responsive to returned image location data and image-related document location data for accessing a located image for segmenting the located image and for extracting features from the located image for obtaining a set of image metadata; storage for storing the set of image metadata; and a user interface for entering a user query for use in searching the storage and for returning an image having image metadata that is relevant to the user query.
Claim 52
Dependent Claim
18 words
48 . An Internet searching tool as in claim 47 , where said user query comprises an image.
Claim 53
Dependent Claim
17 words
49 . An Internet searching tool as in claim 47 , where said user query comprises text.
Claim 54
Dependent Claim
38 words
50 . An Internet searching tool as in claim 47 , where there are a plurality of client computers and a corresponding plurality of searching robots individual ones of which execute in individual ones of said client computers.
Claim 55
Dependent Claim
54 words
51 . An Internet searching tool as in claim 47 , where said processor is responsive to the user query for segmenting an input query image and for extracting features from the input query image for obtaining a set of input image metadata, said processor using the input image metadata when searching the storage.
Claim 56
Dependent Claim
54 words
52 . An Internet searching tool as in claim 51 , where said processor derives a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories.
Claim 57
Dependent Claim
60 words
53 . A method for searching the Internet, comprising: providing at least one server accessible by a plurality of clients; downloading a program to a client, said program comprising an Internet searching program operable for locating at least one type of data; and executing said downloaded program on the client, and returning to said server URLs corresponding to located data.
Claim 58
Dependent Claim
60 words
53 . A method for searching the Internet, comprising: providing at least one server accessible by a plurality of clients; downloading a program to a client, said program comprising an Internet searching program operable for locating at least one type of data; and executing said downloaded program on the client, and returning to said server URLs corresponding to located data.
Claim 59
Dependent Claim
20 words
54 . A method as in claim 53 , where the at least one type of data comprises image data.
Claim 60
Dependent Claim
24 words
55 . A method as in claim 53 , where said URLs comprise URLs of located digital images, and URLs of image related documents.
Claim 61
Dependent Claim
86 words
56 . A system for obtaining information from the Internet, comprising at least one server accessible by a plurality of clients, said server operable for downloading a program to a client, said program comprising an Internet searching program operable for locating at least one type of data, said downloaded program being executable by said client and returning to said server URLs corresponding to located data, said server further comprising a database, searchable by the clients, for storing information corresponding to information returned by said downloaded program.
Claim 62
Dependent Claim
86 words
56 . A system for obtaining information from the Internet, comprising at least one server accessible by a plurality of clients, said server operable for downloading a program to a client, said program comprising an Internet searching program operable for locating at least one type of data, said downloaded program being executable by said client and returning to said server URLs corresponding to located data, said server further comprising a database, searchable by the clients, for storing information corresponding to information returned by said downloaded program.
Claim 63
Dependent Claim
20 words
57 . A system as in claim 56 , where the at least one type of data comprises image data.
Claim 64
Dependent Claim
24 words
58 . A system as in claim 56 , where said URLs comprise URLs of located digital images, and URLs of image related documents.
References Cited (Backward Citations)
US5915250A
Cited
US5899999A
Cited
US6484149B1
Cited
US6240423B1
Cited
US6721449B1
Cited
US20040221237A1
Cited
US20030053693A1
Cited
US20050010553A1
Cited
US6804684B2
Cited
USPTO Annuity & Maintenance Windows
Maintenance schedule is calculated for granted US utility patents.