Patent EP1589444A2 successfully refreshed from Google Patents.
EP1589444A2
Expired / Lapsed
EP
⏱️ 0.0 years remaining
Method, medium, and apparatus for detecting situation change of digital photos and method, medium, and apparatus for situation-based photo clustering in digital photo album
Assignee: Samsung Electronics Co Ltd
•
Filing: 2005-04-19
•
Granted: —
•
Expires: 2025-04-19
Abstract
Abstract An apparatus, medium, and method for detecting a situation change of a digital photo, and a method, medium, and apparatus for situation-based clustering in a digital photo album. The apparatus for situation-based clustering in a digital photo album includes: a photo description information generation unit generating photo description information describing a photo and including at least a photo descriptor; an albuming tool description information generation unit generating albuming tool description information including a predetermined parameter for situation-based clustering of digital photos; an albuming tool performing photo albuming through situation-based clustering by using at least the photo description information and the albuming tool description information; a photo group information generation unit generating predetermined photo group description information from the output of the albuming tool; and a photo albuming information generation unit generating photo albuming information by using photo description information and photo group description information.
Inventors
- 👤 Sangkyun Kim
- 👤 Youngsu Moon
- 👤 Jiyeun Kim
- 👤 Yongman Ro
- 👤 Seungji Yang
Bibliographic Details
Application Number
—
Jurisdiction
EP
Data Source
google_patents
Last Synchronized
2026-11-09 10:11
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/50; G06F16/58; G06F16/51; G06F16/583; G06F16/5838; G06F16/5854; G06F16/5862; G06F16/587
Claims Analysis
Total Claims: 30
Claim 1
Independent Claim
130 words
An apparatus for detecting a situation change in digital photos,
comprising: a photo sort unit sorting photos, desired to be situation-based clustered, in
order of time; a time feature value obtaining unit obtaining predetermined time feature
values from each of two contiguous photos among the sorted photos; a content-based feature value extraction unit extracting predetermined
content-based feature values from each of the two contiguous photos; a dissimilarity measuring unit measuring dissimilarity between the two
photos by making predetermined time feature value importances reflect respective
time feature values, and by making predetermined content-based feature value
importances reflect respective content-based feature values; and a situation change detection unit detecting a situation change by
determining the situation change if an amount of the dissimilarity is equal to or
greater than a predetermined threshold.
Claim 2
Dependent Claim
41 words
The apparatus of claim 1, wherein a predetermined content-based
feature value is generated based on pixel information of a photo, and includes a
visual descriptor including color, texture, and shape feature values, and an audio
descriptor including a voice feature value.
Claim 3
Dependent Claim
225 words
The apparatus of claim 1 or 2, wherein a time feature value
importance and a content-based feature value importance are determined by
referring to situation-based clustering hints including at least one of entire
brightness information of a photo (Brightness), complexity information of the photo
(Level of detail), homogeneous texture information of the photo (Homogeneous
texture), edge information of the photo (Heterogeneous texture), information on
whether the photo is monochrome (Monochromatic), information indicating a
degree of colorfulness of a color expression of the photo (colorfulness),
information indicating an entire color coherence shown in the photo (color
coherence), information indicating a color temperature of a color of the photo
(color temperature), information indicating whether a photo file of the photo
includes taken time information (Taken time), information indicating that, if the
photo and another photo are taken by different cameras in similar time bands and
are clustered together, time information of the photo overlaps time information of
the other photo and an importance of corresponding time information is lowered
when the photo is situation-based clustered (Time overlap), information indicating
whether voice information of a user is stored together with the photo when the
photo was taken and is included with the photo as an audio clip file (Audio clip),
and information indicating voice words and sentence strings recognized in an
audio file of the photo (Speech recognition).
Claim 4
Independent Claim
128 words
An apparatus for situation-based clustering of a digital photo album,
comprising: a photo description information generation unit generating photo description
information describing a photo and including at least a photo identifier; an albuming tool description information generation unit generating
albuming tool description information including a predetermined parameter for
situation-based clustering of digital photos; an albuming tool performing photo albuming through situation-based
clustering by using at least the photo description information and the albuming tool
description information; a photo group information generation unit generating predetermined photo
group description information from an output of the albuming tool; and a photo albuming information generation unit generating predetermined
photo albuming information by using the photo description information and the
predetermined photo group description information for situation-based clustering of
the digital photo album.
Claim 5
Dependent Claim
74 words
The apparatus of claim 4, wherein among the photo identifier,
information on an author of the photo, photo file information, camera information,
photographing information, and a content-based feature value, the photo
description information includes at least the photo identifier, with the content-based
feature value being generated by using pixel information of the photo, and includes
a visual descriptor including color, texture, and shape feature values, and/or an
audio descriptor including a voice feature value.
Claim 6
Dependent Claim
64 words
The apparatus of claim 4 or 5, wherein the albuming tool description
information generation unit comprises at least one of: a sort key generation unit generating items for sorting photos before
clustering the photos; a situation-based clustering hint generation unit generating a situation-based
clustering hint to help photo clustering; and an importance generation unit generating importances of information to be
used in photo clustering.
Claim 7
Dependent Claim
33 words
The apparatus of claim 6, wherein the photo sort items of the sort
key generation unit include at least one of a file name, a photographing time, and
a photo file creation time.
Claim 8
Dependent Claim
27 words
The apparatus of claim 6 or 7, wherein the photographing time
includes photographing date information and the photo file creation time includes
photo file creation date information.
Claim 9
Independent Claim
219 words
The apparatus of any one of claims 6 to 8, wherein the situation-based
clustering hint of the situation-based clustering hint unit includes at least
one of entire brightness information of the photo (Brightness), complexity
information of the photo (Level of detail), homogeneous texture information of the
photo (Homogeneous texture), edge information of the photo (Heterogeneous
texture), information on whether the photo is monochrome (Monochromatic),
information indicating a degree of colorfulness of a color expression of the photo
(colorfulness), information indicating an entire color coherence shown in the photo
(color coherence), information indicating a color temperature of a color of the
photo (color temperature), information indicating whether a photo file of the photo
includes taken time information (Taken time), information indicating that, if the
photo and another photo are taken by different cameras in similar time bands and
are clustered together, time information of the photo overlaps time information of
the other photo and an importances of corresponding time information is lowered
when the photo is situation-based clustered (Time overlap), information indicating
whether voice information of a user is stored together with the photo when the
photo was taken and is included with the photo as an audio clip file (Audio clip),
and information indicating voice words and sentence strings recognized in an
audio file of the photo (Speech recognition).
Claim 10
Independent Claim
57 words
The apparatus of any one of claims 6 to 9, wherein the importances
of the importance generation unit are based on at least one of: information (taken time) setting an importance of time information on a time
when the photo is taken; and information (low-level feature) setting an importance of content-based
feature value information of the photo.
Claim 11
Dependent Claim
45 words
The apparatus of claim 10, wherein the information (low-level
feature) setting the importance of content-based feature value information of the
photo comprises: information setting an importance of a moving picture experts group
(MPEG)-7 Visual Descriptor; and information setting an importance of a MPEG-7 Audio Descriptor.
Claim 12
Independent Claim
27 words
The apparatus of any one of claims 4 to 11, wherein the albuming
tool comprises: a situation-based photo clustering tool clustering digital photo data based
on situations.
Claim 13
Dependent Claim
169 words
The apparatus of claim 12, wherein the situation-based photo
clustering tool comprises: a photo sort unit sorting photos, desired to be situation-based clustered, in
order of time; a time feature value obtaining unit obtaining, from the photo description
information generation unit, time feature values from each of two contiguous
photos among the sorted photos; a content-based feature value extraction unit extracting, from the photo
description information generation unit, content-based feature values from each of
the two contiguous photos; a dissimilarity measuring unit measuring dissimilarity between the two
photos by making time feature value importances, obtained from the albuming tool
description information generation unit, reflect respective time feature values
obtained from the time feature value obtaining unit, and by making predetermined
content-based feature value importances, obtained from the albuming tool
description information generation unit, reflect respective content-based feature
values extracted in the content-based feature value extraction unit; and a situation change detection unit detecting a situation change by
determining the situation change based on an amount of the dissimilarity value.
Claim 14
Dependent Claim
36 words
The apparatus of claim 13, wherein the respective time feature value
importances and the respective predetermined content-based feature value
importances are determined by referring to situation-based clustering hints of the
albuming tool description information generation unit.
Claim 15
Independent Claim
80 words
The apparatus of any one of claims 4 to 14, wherein the photo group
description information of the photo group information generation unit includes at
least one of situation-based photo groups by clustering situation-based photos,
and a situation-based photo group comprises a situation identifier identifying a
situation, a series of photos formed with a plurality of photos determined by photo
identifiers, and a photo key identifier allowing identifying of one or more
representative photos among photos in a photo group.
Claim 16
Independent Claim
100 words
A method for detecting a situation change in digital photos,
comprising: sorting photos, desired to be situation-based clustered, in order of time; obtaining respective time feature values and respective predetermined
content-based feature values from each of two contiguous photos among the
sorted photos; measuring a dissimilarity between the two photos by making predetermined
time feature value importances reflect respective time feature values, and by
making predetermined content-based feature value importances reflect respective
content-based feature values; and detecting a situation change by determining the situation change if an
amount of the dissimilarity is equal to or greater than a predetermined threshold.
Claim 17
Dependent Claim
64 words
The method of claim 16, wherein the detecting of the situation
change comprises determining the situation change if an amount of change,
between a dissimilarity between one of the contiguous photos and a previous
photo, not same as another one of the contiguous photos, and a dissimilarity
between the other one of the contiguous photos and a subsequent photo, is
greater than a threshold.
Claim 18
Dependent Claim
43 words
The method of claim 16 or 17, wherein a predetermined content-based
feature value is generated by using pixel information of respective photos,
and includes a visual descriptor including color, texture, and shape feature values,
and/or an audio descriptor including a voice feature value.
Claim 19
Independent Claim
225 words
The method of any one of claims 16 to 18, wherein a time feature
value importance and a content-based feature value importance are determined by
referring to a situation-based clustering hint including at least one of entire
brightness information of a photo (Brightness), a complexity information of the
photo (Level of detail), homogeneous texture information of the photo
(Homogeneous texture), edge information of the photo (Heterogeneous texture),
information on whether the photo is monochrome (Monochromatic), information
indicating a degree of colorfulness of a color expression of the photo
(colorfulness), information indicating entire color coherence shown in the photo
(color coherence), information indicating a color temperature of a color of the
photo (color temperature), information indicating whether a photo file of the photo
includes taken time information (Taken time), information indicating that, if the
photo and another photo taken by different cameras in similar time bands and are
clustered together, time information of the photo overlaps time information of the
other photo and an importance of corresponding time information is lowered when
the photo is situation-based clustered (Time overlap), information indicating
whether voice information of a user is stored together with the photo when the
photo was taken and is included as an audio clip file (Audio clip), and information
indicating voice words and sentence strings recognized in an audio file of the
photo (Speech recognition).
Claim 20
Independent Claim
115 words
A method for situation-based clustering of a digital photo album,
comprising: generating photo description information by extracting at least one of
camera information on a camera taking a photo, photographing information of the
photo, and a content-based feature value of the photo; generating albuming tool description information including a predetermined
parameter for situation-based clustering of digital photos; performing photo albuming through situation-based clustering by using at
least the photo description information and the albuming tool description
information; generating photo group description information by using a result of the
situation-based clustering; and generating predetermined photo albuming information by using the photo
description information and the photo group description information to situation-based
cluster the digital photo album.
Claim 21
Dependent Claim
81 words
The method of claim 20, wherein in the generating of the photo
description information, among the photo identifier, information on an author of the
photo, photo file information, camera information, photographing information, and
content-based feature values, the photo description information includes at least
the photo identifier, with the content-based feature value being generated by using
pixel information of the photo, and includes a visual descriptor including color,
texture, and shape feature values, and/or an audio descriptor including a voice
feature value.
Claim 22
Dependent Claim
61 words
The method of claim 20 or 21, wherein in the generating of the
albuming tool description information, the albuming tool description information
generation comprises at least one of: sort key sorting photos before clustering of the photos; situation-based clustering hint generating a situation-based clustering hint
to help photo clustering; and importance generating importances of information to be used in photo
clustering.
Claim 23
Dependent Claim
31 words
The method of claim 22, wherein a sort key in the sort key sorting
includes at least one of a file name, a photographing time, and a photo file creation
time.
Claim 24
Dependent Claim
208 words
The method of claim 22 or 23, wherein the situation-based clustering
hint includes at least one of entire brightness information of the photo (Brightness),
complexity information of the photo (Level of detail), homogeneous texture
information of the photo (Homogeneous texture), edge information of the photo
(Heterogeneous texture), information on whether the photo is monochrome
(Monochromatic), information indicating a degree of colorfulness of a color
expression of the photo (colorfulness), information indicating an entire color
coherence shown in the photo (color coherence), information indicating a color
temperature of a color of the photo (color temperature), information indicating
whether a photo file of the photo includes taken time information (Taken time),
information indicating that; if the photo and another photo taken by different
cameras in similar time bands and are clustered together, time information of the
photo overlaps time information of the other photo and an importance of
corresponding time information is lowered when the photo is situation-based
clustered (Time overlap), information indicating whether voice information of a user
stored together with the photo when the photo was taken and is included with the
photo as an audio clip file (Audio clip), and information indicating voice words and
sentence strings recognized in an audio file of the photo (Speech recognition).
Claim 25
Independent Claim
52 words
The method of any one of claims 22 to 24, wherein the importances
are based on at least one of: information (taken time) setting an importance of time information on a time
when the photo is taken; and information (low-level feature) setting an importance of content-based
feature value information of the photo.
Claim 26
Dependent Claim
41 words
The method of claim 25, wherein the information (low-level feature)
setting the importance of content-based feature value information of the photo
comprises: information setting an importance of a MPEG-7 Visual Descriptor; and information setting an importance of a MPEG-7 Audio Descriptor.
Claim 27
Independent Claim
98 words
The method of any one of claims 20 to 26, wherein the performing of
the photo albuming comprises: sorting photos, desired to be situation-based clustered, in order of time; obtaining time feature values and predetermined content-based feature
values from each of two contiguous photos among the sorted photos; measuring a dissimilarity between the two photos by making predetermined
time feature value importances reflect respective time feature values, and by
making predetermined content-based feature value importances reflect respective
content-based feature values; and detecting a situation change by determining the situation change based on
an amount of the dissimilarity value.
Claim 28
Dependent Claim
25 words
The method of claim 27, wherein the time feature value importances
and the content-based feature value importances are determined by referring to
situation-based clustering hints.
Claim 29
Independent Claim
84 words
The method of any one of claims 20 to 28, wherein in the generating
of the predetermined photo albuming information, the photo group description
information includes at least one of situation-based photo groups by clustering
situation-based photos, and with the situation-based photo group comprising a
situation identifier identifying a situation, a series of photos formed with a plurality
of photos determined by photo identifiers, and a photo key identifier allowing
identifying of one or more representative photos among the photos in a photo
group.
Claim 30
Independent Claim
17 words
A medium comprising computer readable instructions implementing
the method of any one of claims 16 through 29.
References Cited (Backward Citations)
No citation records indexed.
USPTO Annuity & Maintenance Windows
Maintenance schedule is calculated for granted US utility patents.