Patent EP1045316A2 successfully refreshed from Google Patents.
EP1045316A2
Expired / Lapsed
EP
⏱️ 0.0 years remaining
Image processing method and apparatus
Assignee: Canon Inc
•
Filing: 2000-04-12
•
Granted: —
•
Expires: 2020-04-12
Abstract
Abstract In an image processing apparatus, a moving picture storage module stores moving picture data. A scene designation module designates at least one scene in the moving picture data, and sets a key frame and importance level of the designated scene. A frame feature extraction module acquires an image feature amount of the designated key frame. A scene feature information storage module generates scene feature information including the duration, key frame, importance level, and image feature amount of the key frame of the scene designated by the scene designation module, and stores the generated information in a memory in correspondence with the moving picture data. Based on the scene feature information, digest playback for playing back only principal part of the designated moving picture data is executed, or desired moving picture data is found by a similar image search using the key frame.
Inventors
- 👤 Kazuyo Ikeda
- 👤 Hirotaka Shiiyama
- 👤 Hiroshi Tojo
Bibliographic Details
Application Number
—
Jurisdiction
EP
Data Source
google_patents
Last Synchronized
2026-11-10 02:06
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/70; G06F16/78; G06F16/783; G06F16/7847; G06F16/785; G06F16/73; G06F16/738; G06F16/739
Claims Analysis
Total Claims: 58
Claim 1
Independent Claim
90 words
An image processing apparatus characterized by
comprising: specifying means (202, S11) for specifying at
least one desired scene in moving picture data, and a
frame serving as a key frame in the desired scene; setting means (202, S11) for setting an
importance level of the desired scene specified by said
specifying means; and storage means (204, S12) for storing information
indicating the scene and key frame specified by said
specifying means and the importance level set by said
setting means as scene information in association with
the moving picture data.
Claim 2
Dependent Claim
43 words
The apparatus according to claim 1, wherein the
information indicating the scene specified by said
specifying means includes frame identifiers of start
and end frames of the scene, and the information
indicating the key frame includes a frame identifier of
the key frame.
Claim 3
Dependent Claim
25 words
The apparatus according to claim 2, wherein the
frame identifier is a frame number assigned in turn
from the beginning of the moving picture data.
Claim 4
Dependent Claim
24 words
The apparatus according to claim 2, wherein the
frame identifier is a time from the beginning to that
frame of the moving picture data.
Claim 5
Dependent Claim
44 words
The apparatus according to claim 1, further
comprising: acquisition means for acquiring feature amount
information of the key frame, and wherein the scene information stored in said
storage means includes the feature amount information
of the key frame image acquired by said acquisition
means.
Claim 6
Dependent Claim
55 words
The apparatus according to claim 5, wherein the
feature amount information of the key frame is a label
sequence obtained by segmenting an image of the key
frame into a plurality of blocks, assigning labels in
accordance with feature amounts acquired from the
individual segmented blocks, and arranging the assigned
labels in a predetermined order.
Claim 7
Dependent Claim
29 words
The apparatus according to claim 6, wherein the
image is segmented into a plurality of vertical and
horizontal blocks, the numbers of which are given by
powers of 2.
Claim 8
Dependent Claim
37 words
The apparatus according to claim 6, further
comprising means for storing information that pertains
to the number of segmented blocks, a feature amount
extraction method, and a color expression of an image
together with the label sequence.
Claim 9
Dependent Claim
85 words
The apparatus according to claim 5, further
comprising: computation means for computing a similarity
value between images of a designated key frame and
another key frame on the basis of image feature amounts
of the key frames when one key frame is designated; presentation means for presenting key frames on
the basis of similarity values computed by said
computation means; and playback means for acquiring and playing back
moving picture data including the key frame selected
from the key frames presented by said presentation
means.
Claim 10
Dependent Claim
37 words
The apparatus according to claim 1, further
comprising generation means for generating extracted
moving picture data, which can be played back as a
moving picture, on the basis of the scene information
stored in said storage means.
Claim 11
Dependent Claim
30 words
The apparatus according to claim 10, wherein said
generation means generates the extracted moving picture
data by time-serially merging scenes indicated by the
scene information stored in said storage means.
Claim 12
Dependent Claim
43 words
The apparatus according to claim 10, wherein said
generation means generates the extracted moving picture
data by time-serially merging scenes indicated by the
scene information stored in said storage means, and
opening and ending scenes of the moving picture for a
predetermined time.
Claim 13
Dependent Claim
48 words
The apparatus according to claim 10, further
comprising: designation means for designating a playback time
of the extracted moving picture data; and adjustment means for adjusting durations of
scenes to be merged by said generation means on the
basis of the playback time designated by said
designation means.
Claim 14
Dependent Claim
105 words
The apparatus according to claim 13, wherein said
adjustment means comprises: frame extraction means for determining importance
levels of frames in a scene on the basis of an
importance level set in the scene, extracting frames
having importance levels that exceed a threshold value
from the scene, and providing the extracted frames as a
scene to be merged to said generation means; and threshold value control means for controlling the
threshold value used in said frame extraction means to
make the playback time of the extracted moving picture
data generated by said generation means fall within a
predetermined error range of the designated playback
time.
Claim 15
Dependent Claim
63 words
The apparatus according to claim 14, further
comprising extension adjustment means for extending the
playback time as needed using the key frame of each
scene as the center to fall within a predetermined
error range of the designated playback time when a
total time of all scenes designated by the scene
information is shorter than the playback time
designated by said designation means.
Claim 16
Dependent Claim
34 words
The apparatus according to claim 10, further
comprising removing means for removing a scene, a
playback time of which is shorter than a predetermined
time, from scenes that form the extracted moving
picture data.
Claim 17
Dependent Claim
30 words
The apparatus according to claim 16, further
comprising extending means for extending another scene
by a time corresponding to the playback time of the
scene removed by said removing means.
Claim 18
Dependent Claim
42 words
The apparatus according to claim 17, wherein said
extending means extends a time of a scene set with a
higher importance level of previous and next scenes
which neighbor the scene removed by said removing means
in the extracted moving picture data.
Claim 19
Dependent Claim
96 words
The apparatus according to claim 10, further
comprising: designation means for designating a playback time
of the extracted moving picture data; and extraction means for extracting scenes for a
predetermined time duration by selecting scenes
indicated by the scene information stored in said
storage means in descending order of importance levels,
and wherein said extraction means executes a process
until a total time of the extracted scenes exceeds the
playback time designated by said designation means, and said generation means generates the extracted
moving picture data by time-serially merging the scenes
extracted by said extraction means.
Claim 20
Dependent Claim
22 words
The apparatus according to claim 19, wherein the
predetermined time duration is a duration in which the
user can recognize an image.
Claim 21
Dependent Claim
63 words
The apparatus according to claim 20, further
comprising extension adjustment means for extending the
playback time as needed using the key frame of each
scene as the center to fall within a predetermined
error range of the designated playback time when a
total time of all scenes designated by the scene
information is shorter than the playback time
designated by said designation means.
Claim 22
Dependent Claim
117 words
The apparatus according to claim 10, wherein the
scene information includes an image feature amount of
an image of each key frame, and said apparatus further
comprises: computation means for computing a similarity
value between images of a designated key frame and
another key frame on the basis of image feature amounts
of the key frames when one key frame is designated; presentation means for presenting key frames on
the basis of similarity values computed by said
computation means; and playback means for playing back the extracted
moving picture data generated by said generation means
in association with the moving picture data which
includes the key frame selected from the key frames
presented by said presentation means.
Claim 23
Dependent Claim
55 words
The apparatus according to claim 22, wherein the
feature amount information of the key frame is a label
sequence obtained by segmenting an image of the key
frame into a plurality of blocks, assigning labels in
accordance with feature amounts acquired from the
individual segmented blocks, and arranging the assigned
labels in a predetermined order.
Claim 24
Dependent Claim
113 words
The apparatus according to claim 10, wherein the
scene information includes an image feature amount of
an image of each key frame, said apparatus further
comprises: first means for randomly displaying key frames; second means for playing back the extracted
moving picture data generated by said generation means; third means for making a similarity search by
computing a similarity value between images of the
designated key frame and another key frame on the basis
of image feature amounts of the key frames; and fourth means for playing back a moving picture on
the basis of the designated key frame, and said first to fourth means can operate to
function in an arbitrary combination.
Claim 25
Dependent Claim
20 words
The apparatus according to claim 1, further
comprising display means for displaying a list of
thumbnails of key frame images.
Claim 26
Dependent Claim
93 words
An image processing method characterized by
comprising: the specifying step (S11) of specifying at least
one desired scene in moving picture data, and a frame
serving as a key frame in the desired scene; the setting step (S11) of setting an importance
level of the desired scene specified in the specifying
step; and the storage step (S12) of storing information
indicating the scene and key frame specified in the
specifying step and the importance level set in the
setting step as scene information in storage means in
association with the moving picture data.
Claim 27
Dependent Claim
43 words
The method according to claim 26, wherein the
information indicating the scene specified in the
specifying step includes frame identifiers of start and
end frames of the scene, and the information indicating
the key frame includes a frame number of the key frame.
Claim 28
Dependent Claim
25 words
The method according to claim 27, wherein the
frame identifier is a frame number assigned in turn
from the beginning of the moving picture data.
Claim 29
Dependent Claim
24 words
The method according to claim 27, wherein the
frame identifier is a time from the beginning to that
frame of the moving picture data.
Claim 30
Dependent Claim
45 words
The method according to claim 26, further
comprising: the acquisition step of acquiring feature amount
information of the key frame, and wherein the scene information stored in the
storage step includes the feature amount information of
the key frame image acquired in the acquisition step.
Claim 31
Dependent Claim
55 words
The method according to claim 30, wherein the
feature amount information of the key frame is a label
sequence obtained by segmenting an image of the key
frame into a plurality of blocks, assigning labels in
accordance with feature amounts acquired from the
individual segmented blocks, and arranging the assigned
labels in a predetermined order.
Claim 32
Dependent Claim
29 words
The method according to claim 31, wherein the
image is segmented into a plurality of vertical and
horizontal blocks, the numbers of which are given by
powers of 2.
Claim 33
Dependent Claim
38 words
The method according to claim 31, further
comprising the step of storing information that
pertains to the number of segmented blocks, a feature
amount extraction method, and a color expression of an
image together with the label sequence.
Claim 34
Dependent Claim
88 words
The method according to claim 30, further
comprising: the computation step of computing a similarity
value between images of a designated key frame and
another key frame on the basis of image feature amounts
of the key frames when one key frame is designated; the presentation step of presenting key frames on
the basis of similarity values computed in the
computation step; and the playback step of acquiring and playing back
moving picture data including the key frame selected
from the key frames presented in the presentation step.
Claim 35
Dependent Claim
38 words
The method according to claim 26, further
comprising the generation step of generating extracted
moving picture data, which can be played back as a
moving picture, on the basis of the scene information
stored in the storage means.
Claim 36
Dependent Claim
34 words
The method according to claim 35, wherein the
generation step includes the step of generating the
extracted moving picture data by time-serially merging
scenes indicated by the scene information stored in the
storage means.
Claim 37
Dependent Claim
47 words
The method according to claim 35, wherein the
generation step includes the step of generating the
extracted moving picture data by time-serially merging
scenes indicated by the scene information stored in the
storage means, and opening and ending scenes of the
moving picture for a predetermined time.
Claim 38
Dependent Claim
50 words
The method according to claim 35, further
comprising: the designation step of designating a playback
time of the extracted moving picture data; and the adjustment step of adjusting durations of
scenes to be merged in the generation step on the basis
of the playback time designated in the designation step.
Claim 39
Dependent Claim
107 words
The method according to claim 38, wherein the
adjustment step comprises: the frame extraction step of determining
importance levels of frames in a scene on the basis of
an importance level set in the scene, extracting frames
having importance levels that exceed a threshold value
from the scene, and providing the extracted frames as a
scene to be merged to the generation step; and the threshold value control step of controlling
the threshold value used in the frame extraction step
to make the playback time of the extracted moving
picture data generated in the generation step fall
within a predetermined error range of the designated
playback time.
Claim 40
Dependent Claim
64 words
The method according to claim 39, further
comprising the extension adjustment step of extending
the playback time as needed using the key frame of each
scene as the center to fall within a predetermined
error range of the designated playback time when a
total time of all scenes designated by the scene
information is shorter than the playback time
designated in the designation step.
Claim 41
Dependent Claim
35 words
The method according to claim 35, further
comprising the removing step of removing a scene, a
playback time of which is shorter than a predetermined
time, from scenes that form the extracted moving
picture data.
Claim 42
Dependent Claim
31 words
The method according to claim 41, further
comprising the extending step of extending another
scene by a time corresponding to the playback time of
the scene removed in the removing step.
Claim 43
Dependent Claim
46 words
The method according to claim 42, wherein the
extending step includes the step of extending a time of
a scene set with a higher importance level of previous
and next scenes which neighbor the scene removed in the
removing step in the extracted moving picture data.
Claim 44
Dependent Claim
106 words
The method according to claim 35, further
comprising: the designation step of designating a playback
time of the extracted moving picture data; and the extraction step of extracting scenes for a
predetermined time duration by selecting scenes
indicated by the scene information stored in the
storage means in descending order of importance levels,
and wherein the extraction step includes the step of
executing a process until a total time of the extracted
scenes exceeds the playback time designated in the
designation step, and the generation step includes the step of
generating the extracted moving picture data by
time-serially merging the scenes extracted in the
extraction step.
Claim 45
Dependent Claim
22 words
The method according to claim 44, wherein the
predetermined time duration is a duration in which the
user can recognize an image.
Claim 46
Dependent Claim
64 words
The method according to claim 45, further
comprising the extension adjustment step of extending
the playback time as needed using the key frame of each
scene as the center to fall within a predetermined
error range of the designated playback time when a
total time of all scenes designated by the scene
information is shorter than the playback time
designated in the designation step.
Claim 47
Dependent Claim
120 words
The method according to claim 35, wherein the
scene information includes an image feature amount of
an image of each key frame, and said method further
comprises: the computation step of computing a similarity
value between images of a designated key frame and
another key frame on the basis of image feature amounts
of the key frames when one key frame is designated; the presentation step of presenting key frames on
the basis of similarity values computed in the
computation step; and the playback step of playing back the extracted
moving picture data generated in the generation step in
association with the moving picture data which includes
the key frame selected from the key frames presented in
the presentation step.
Claim 48
Dependent Claim
55 words
The method according to claim 47, wherein the
feature amount information of the key frame is a label
sequence obtained by segmenting an image of the key
frame into a plurality of blocks, assigning labels in
accordance with feature amounts acquired from the
individual segmented blocks, and arranging the assigned
labels in a predetermined order.
Claim 49
Dependent Claim
118 words
The method according to claim 35, wherein the
scene information includes an image feature amount of
an image of each key frame, and said method further
comprises: the first step of randomly displaying key frames; the second step of playing back the extracted
moving picture data generated in the generation step; the third step of making a similarity search by
computing a similarity value between images of the
designated key frame and another key frame on the basis
of image feature amounts of the key frames; and the fourth step of playing back a moving picture
on the basis of the designated key frame, and the first to fourth steps can operate to function
in an arbitrary combination.
Claim 50
Dependent Claim
21 words
The method according to claim 26, further
comprising the display step of displaying a list of
thumbnails of key frame images.
Claim 51
Dependent Claim
115 words
A storage medium for storing a control program
that makes a computer implement a moving picture data
storage process, said control program comprising: a code of the specifying step of specifying at
least one desired scene in moving picture data, and a
frame serving as a key frame in the desired scene; a code of the setting step of setting an
importance level of the desired scene specified in the
specifying step; and a code of the storage step of storing information
indicating the scene and key frame specified in the
specifying step and the importance level set in the
setting step as scene information in storage means in
association with the moving picture data.
Claim 52
Dependent Claim
52 words
The medium according to claim 51, wherein said
control program further comprises: a code of the acquisition step of acquiring
feature amount information of the key frame, and wherein the scene information stored in the
storage step includes the feature amount information of
the key frame image acquired in the acquisition step.
Claim 53
Dependent Claim
101 words
The medium according to claim 52, wherein said
control program further comprises: a code of the computation step of computing a
similarity value between images of a designated key
frame and another key frame on the basis of image
feature amounts of the key frames when one key frame is
designated; a code of the presentation step of presenting key
frames on the basis of similarity values computed in
the computation step; and a code of the playback step of acquiring and
playing back moving picture data including the key
frame selected from the key frames presented in the
presentation step.
Claim 54
Dependent Claim
45 words
The medium according to claim 51, wherein said
control program further comprises a code of the
generation step of generating extracted moving picture
data, which can be played back as a moving picture, on
the basis of the scene information stored in the
storage means.
Claim 55
Dependent Claim
130 words
The medium according to claim 54, wherein the
scene information includes an image feature amount of
an image of each key frame, and said control program
further comprises: a code of the computation step of computing a
similarity value between images of a designated key
frame and another key frame on the basis of image
feature amounts of the key frames when one key frame is
designated; a code of the presentation step of presenting key
frames on the basis of similarity values computed in
the computation step; and a code of the playback step of playing back the
extracted moving picture data generated in the
generation step in association with the moving picture
data which includes the key frame selected from the key
frames presented in the presentation step.
Claim 56
Dependent Claim
68 words
A storage medium for storing moving picture data,
said medium storing, in correspondence with the moving
picture data, scene information, which includes: scene specifying information for specifying at
least one desired scene from the stored moving picture
data; key frame information representing a key frame of
the scene specified by the scene specifying
information; and an importance level set for each scene specified
by the scene specifying information.
Claim 57
Dependent Claim
35 words
The medium according to claim 56, wherein the
scene specifying information includes frame numbers of
start and end frames of the scene, and the key frame
information includes a frame number of the key frame.
Claim 58
Dependent Claim
24 words
The medium according to claim 56, wherein the key
frame information includes a feature amount of a key
frame image in the scene information.
References Cited (Backward Citations)
No citation records indexed.
USPTO Annuity & Maintenance Windows
Maintenance schedule is calculated for granted US utility patents.