US9386235B2
Active / In Force
US
⏱️ 8.0 years remaining
Systems and methods for generating composite images of long documents using mobile video data
Assignee: Tungsten Automation Corp
•
Filing: 2014-11-14
•
Granted: 2016-07-05
•
Expires: 2034-11-14
Abstract
Abstract Systems, methods, and computer program products are disclosed and include: initiating a capture operation using an image capture component of the mobile device, the capture operation comprising; capturing video data; and estimating a plurality of motion vectors corresponding to motion of the image capture component during the capture operation. The systems, techniques, and computer program products also include detecting a document depicted in the video data; tracking a position of the detected document throughout the video data; selecting a plurality of images using the image capture component of the mobile device, wherein the selection is based at least in part on: the tracked position of the detected document; and the estimated motion vectors; and generating a composite image based on at least some of the selected plurality of images.
Inventors
- 👤 Jiyong Ma
- 👤 Anthony Macciola
- 👤 Jan W. Amtrup
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:06
CPC Classifications
H; H04; H04N; H04N5/00; H04N5/222; H04N5/262; H04N5/265; G06K9/00624; G06K9/00751; G; G06; G06Q; G06Q20/00; G06Q20/04; G06Q20/047; G06T; G06T11/00; G06T11/60; G06T3/00; G06T3/14; G06T7/00; G06T7/10; G06T7/11; G06V; G06V20/00; G06V20/40; G06V20/46; G06V20/47; G06V30/00; G06V30/40; G06V30/41; G06V30/413; H04N1/00; H04N1/387; H04N1/3876; H04N5/14; H04N5/144; H04N5/145; G06T2207/00; G06T2207/20; G06T2207/20212; G06T2207/20224
Claims Analysis
Total Claims: 22
Claim 1
Independent Claim
142 words
A computer program product comprising a non-transitory computer readable medium having stored thereon instructions executable by a mobile device, the instructions being configured to cause the mobile device, upon execution thereof, to: initiate a capture operation using an image capture component of the mobile device, the capture operation comprising; capturing video data; and estimating a plurality of motion vectors corresponding to motion of the image capture component during the capture operation; detect a document depicted in the video data; track a position of the detected document throughout the video data; select a plurality of images using the image capture component of the mobile device, wherein the selection is based at least in part on: the tracked position of the detected document; and the estimated motion vectors; and generate a composite image based on at least some of the selected plurality of images.
Claim 2
Dependent Claim
142 words
A computer program product comprising a non-transitory computer readable medium having stored thereon instructions executable by a mobile device, the instructions being configured to cause the mobile device, upon execution thereof, to: initiate a capture operation using an image capture component of the mobile device, the capture operation comprising; capturing video data; and estimating a plurality of motion vectors corresponding to motion of the image capture component during the capture operation; detect a document depicted in the video data; track a position of the detected document throughout the video data; select a plurality of images using the image capture component of the mobile device, wherein the selection is based at least in part on: the tracked position of the detected document; and the estimated motion vectors; and generate a composite image based on at least some of the selected plurality of images.
Claim 3
Dependent Claim
17 words
The computer program product as recited in claim 1 , wherein the document is a long document.
Claim 4
Dependent Claim
23 words
The computer program product as recited in claim 1 , wherein the tracking is based exclusively on the estimated plurality of motion vectors.
Claim 5
Dependent Claim
46 words
The computer program product as recited in claim 1 , the instructions configured to cause the mobile device to detect the document further comprising instructions configured to cause the mobile device to identify at least one edge of the document depicted in the captured video data.
Claim 6
Dependent Claim
35 words
The computer program product as recited in claim 1 , wherein each of the selected plurality of images depicts a portion of the document, and wherein the composite image depicts an entirety of the document.
Claim 7
Dependent Claim
54 words
The computer program product as recited in claim 1 , wherein the composite image is characterized by at least one of: an image resolution greater than an image resolution of any of the selected plurality of images; and an image size greater than an image size of any of the selected plurality of images.
Claim 8
Dependent Claim
66 words
The computer program product as recited in claim 1 , further comprising instructions configured to cause the mobile device to downsample the video data, and wherein the instructions configured to cause the mobile device to detect the document, track the position of the document, and select the plurality of images is configured to perform the detecting, the tracking, and the selecting using the downsampled video data.
Claim 9
Dependent Claim
105 words
The computer program product as recited in claim 1 , further comprising instructions configured to cause the mobile device to: determine at least one motion displacement based on some or all of the estimated plurality of motion vectors, each motion displacement corresponding to the image capture component during the capture operation; and either terminate or pause the capture operation in response to determining one of the motion displacement(s) is characterized by a value exceeding a predefined motion displacement threshold; and either initiate a new capture operation in response to terminating the capture operation; or resume the capture operation in response to pausing the capture operation.
Claim 10
Dependent Claim
30 words
The computer program product as recited in claim 8 , wherein the predefined motion displacement threshold has a value in a range from about 5 pixels to about 25 pixels.
Claim 11
Dependent Claim
41 words
The computer program product as recited in claim 1 , wherein each selected image depicts a portion of the document, and wherein the composite image depicts only portion(s) of the document that correspond to a financial transaction memorialized by the document.
Claim 12
Dependent Claim
91 words
The computer program product as recited in claim 10 , further comprising instructions configured to cause the mobile device to: identify, based on the composite image, one or more portions of the document depicting textual information; classify each identified portion of the document based on the textual information depicted therein; determine whether each classified portion is relevant to the financial transaction or irrelevant to the financial transaction, the determining being based on the portion classification; and remove each portion determined to be irrelevant to the financial transaction from the composite image.
Claim 13
Dependent Claim
104 words
The computer program product as recited in claim 11 , further comprising instructions configured to cause the mobile device to: align the portions determined to be relevant to the financial transaction; and generate a second composite image, wherein the second composite image is characterized by: approximately a same image size as an image size of the composite image; approximately a same image resolution as an image resolution of the composite image; excluding textual information irrelevant to the financial transaction; and including textual information relevant to the financial transaction, wherein a plurality of characters comprising the textual information relevant to the financial transaction are aligned.
Claim 14
Dependent Claim
68 words
The computer program product as recited in claim 1 , the instructions configured to cause the mobile device to select the plurality of images further comprising instructions configured to cause the mobile device to define a plurality of frame pairs, wherein each frame pair consists of a reference frame and a test frame, and wherein each reference frame and each test frame is selected from the video data.
Claim 15
Dependent Claim
106 words
The computer program product as recited in claim 13 , the instructions configured to cause the mobile device to generate the composite image further comprising instructions configured to cause the mobile device to: detect a skew angle in one or more of the reference frame and the test frame of at least one of the frame pairs, the skew angle corresponding to the document and having a magnitude of >0.0 degrees; and correct the skew angle in at least one of the reference frame and the test frame, wherein the document depicted in the composite image is characterized by a skew angle of approximately 0.0 degrees.
Claim 16
Dependent Claim
81 words
The computer program product as recited in claim 13 , the instructions configured to cause the mobile device to select the plurality of images further comprising instructions configured to cause the mobile device to: determine an amount of overlap between the reference frame and the test frame of each frame pair; and select an image corresponding to each frame pair for which the amount of overlap between the reference frame and the test frame is greater than a predetermined overlap threshold.
Claim 17
Dependent Claim
52 words
The computer program product as recited in claim 15 , wherein the amount of overlap corresponds to the document, wherein the amount of overlap does not correspond to a background depicted in the reference frame, and wherein the amount of overlap does not correspond to a background depicted in the test frame.
Claim 18
Dependent Claim
30 words
The computer program product as recited in claim 15 , wherein the predetermined overlap threshold corresponds to a distance of at least 40% of a length of the reference frame.
Claim 19
Dependent Claim
60 words
The computer program product as recited in claim 13 , the instructions configured to cause the mobile device to generate the composite image further comprising instructions configured to cause the mobile device to: detect textual information in each of the reference frame and the test frame of at least one frame pair, the textual information being depicted in the document.
Claim 20
Dependent Claim
139 words
The computer program product as recited in claim 18 , the instructions configured to cause the mobile device to detect textual information in each of the reference frame and the test frame of at least one frame pair further comprising instructions configured to cause the mobile device to: define, in the reference frame, at least one rectangular portion of the document depicting some or ail of the textual information; define, in the test frame, at least one corresponding rectangular portion of the document depicting some or all of the textual information; and align the document depicted in the test frame with the document depicted in the reference frame, the alignment being based on: the textual information depicted in at least one of the rectangular portion(s); and the textual information depicted in at least one of the corresponding rectangular portion(s).
Claim 21
Dependent Claim
220 words
The computer program product as recited in claim 19 , wherein the textual information comprises at least one of: an identity of one or more characters represented in the rectangular portion; an identity of one or more characters represented in the corresponding rectangular portion; a sequence of characters represented in the rectangular portion; a sequence of characters represented in the corresponding rectangular portion; a position of one or more characters represented in the rectangular portion; a position of one or more characters represented in the corresponding rectangular portion; an absolute size of one or more characters represented in the rectangular portion; an absolute size of one or more characters represented in the corresponding rectangular portion a size of one or more characters represented in the rectangular portion relative to a size of one or more characters represented in the corresponding rectangular portion; a size of one or more characters represented in the corresponding rectangular portion relative to a size of one or more characters represented in the rectangular portion; a color of one or more characters represented in the rectangular portion; a color of one or more characters represented in the corresponding rectangular portion; a shape of one or more characters represented in the rectangular portion; and a shape of one or more characters represented in the corresponding rectangular portion.
Claim 22
Dependent Claim
61 words
The computer program product as recited in claim 19 , the instructions configured to cause the mobile device to align the document depicted in the test frame with the document depicted in the reference frame further comprising instructions configured to cause the mobile device to perform optical character recognition (OCR) on at least the rectangular portion and the corresponding rectangular portion.
References Cited (Backward Citations)
US3069654A
Cited
US4836026A
Cited
US4656665A
Cited
US4903312A
Cited
US4992863A
Cited
US5124810A
Cited
US5159667A
Cited
US5020112A
Cited
US5181260A
Cited
US5063604A
Cited
US5344132A
Cited
US5202934A
Cited
US5220621A
Cited
US5430810A
Cited
US5375197A
Cited
US5282055A
Cited
US5313527A
Cited
US5467407A
Cited
US5563966A
Cited
EP0549329A2
Cited
US5355547A
Cited
US5317646A
Cited
US5668890A
Cited
US5268967A
Cited
US5848184A
Cited
US5717794A
Cited
US5629989A
Cited
US5602964A
Cited
US7082426B2
Cited
US5353673A
Cited
US6118552A
Cited
US6219773B1
Cited
US5721940A
Cited
US5546474A
Cited
US5839019A
Cited
US5473742A
Cited
US5832138A
Cited
US5699244A
Cited
WO1996004749A1
Cited
US5652663A
Cited
US5563723A
Cited
US5757963A
Cited
US5696611A
Cited
US6408094B1
Cited
US5822454A
Cited
US6104830A
Cited
JPH0962826A
Cited
US5781665A
Cited
US5825915A
Cited
US6073148A
Cited
EP0767578A2
Cited
JPH09116720A
Cited
US6009196A
Cited
US5987172A
Cited
US6009191A
Cited
US5937084A
Cited
US6356647B1
Cited
US6038348A
Cited
US6072916A
Cited
US6347162B1
Cited
US6016361A
Cited
US6512848B2
Cited
US6118544A
Cited
US6098065A
Cited
US6233059B1
Cited
US6347152B1
Cited
WO1998047098A1
Cited
US6154217A
Cited
US6005958A
Cited
US6067385A
Cited
US6229625B1
Cited
US6055968A
Cited
US5953388A
Cited
US6424742B2
Cited
US6525840B1
Cited
US7038713B1
Cited
US6011595A
Cited
US6473198B1
Cited
US6480624B1
Cited
US20050204058A1
Cited
US5867264A
Cited
TW401553B
Cited
EP0936804A2
Cited
US6873721B1
Cited
US6563531B1
Cited
US6327581B1
Cited
US6473535B1
Cited
US7194471B1
Cited
US6393147B2
Cited
US6789069B1
Cited
US20050228591A1
Cited
US6408105B1
Cited
US6748109B1
Cited
US6192360B1
Cited
JP2002519766A
Cited
WO1999067731A1
Cited
US7253836B1
Cited
JP2000103628A
Cited
US6219158B1
Cited
US6385346B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2019-07-05 | 2020-01-04 | 2020-07-05 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2023-07-05 | 2024-01-04 | 2024-07-05 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2027-07-05 | 2028-01-04 | 2028-07-05 | $7700.00 | Pending Window |