Patent US6865273B2 successfully refreshed from Google Patents.
US6865273B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus to detect watermark that are resistant to resizing, rotation and translation
Assignee: Sony Corp
•
Filing: 2002-06-05
•
Granted: 2005-03-08
•
Expires: 2022-06-05
Abstract
Abstract A method and/or apparatus for detecting a watermark in a two-dimensional frame of data, the frame of data including a plurality of data values representing the watermark embedded in a frame of content data, the method comprising: computing a filtered frame of data from the frame of data having at least some harmonic frequency components corresponding to the watermark that are emphasized as compared to at least some frequency components corresponding to the content data; computing a two-dimensional Fourier transform of the filtered frame of data to produce a two-dimensional frequency spectrum of the filtered frame of data; selecting a set of frequency components associated with the watermark from among the frequency components of the two-dimensional frequency spectrum; and computing at least one of a rotation value, a resizing value, and a translation value associated with the watermark as compared to a reference watermark using one or more of the frequency components of the selected set.
Inventors
- 👤 Peter D. Wendt
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 13:40
CPC Classifications
G; G06; G06T; G06T1/00; G06T1/0021; G06T1/005; G06T1/0064; G06F; G06F18/00; G06T7/00; G06T2201/00; G06T2201/005; G06T2201/0052; G06T2201/0065
Claims Analysis
Total Claims: 143
Claim 1
Independent Claim
154 words
A method of detecting a watermark in a two-dimensional frame of data, the frame of data including a plurality of data values representing the watermark embedded in a frame of content data, the method comprising: computing a filtered frame of data from the frame of data having at least some harmonic frequency components corresponding to the watermark that are emphasized as compared to at least some frequency components corresponding to the content data; computing a two-dimensional Fourier transform of the filtered frame of data to produce a two-dimensional frequency spectrum of the filtered frame of data; selecting a set of frequency components associated with the watermark from among the frequency components of the two-dimensional frequency spectrum; and computing at least one of a rotation value, a resizing value, and a translation value associated with the watermark as compared to a reference watermark using one or more of the frequency components of the selected set.
Claim 2
Independent Claim
154 words
A method of detecting a watermark in a two-dimensional frame of data, the frame of data including a plurality of data values representing the watermark embedded in a frame of content data, the method comprising: computing a filtered frame of data from the frame of data having at least some harmonic frequency components corresponding to the watermark that are emphasized as compared to at least some frequency components corresponding to the content data; computing a two-dimensional Fourier transform of the filtered frame of data to produce a two-dimensional frequency spectrum of the filtered frame of data; selecting a set of frequency components associated with the watermark from among the frequency components of the two-dimensional frequency spectrum; and computing at least one of a rotation value, a resizing value, and a translation value associated with the watermark as compared to a reference watermark using one or more of the frequency components of the selected set.
Claim 3
Dependent Claim
19 words
The method of claim 1 , wherein the frame of data are one of pixel data and audio data.
Claim 4
Dependent Claim
72 words
The method of claim 1 , wherein: the watermark substantially matches the reference watermark but for the at least one of the rotation value, the resizing value, and the translation value; the watermark and the reference watermark include a respective plurality of data blocks representing a message; and each data block includes an array of data values arranged in at least one of a first and second pattern defined by the message.
Claim 5
Dependent Claim
47 words
The method of claim 3 , wherein the first and second patterns may be defined by four quadrants of data values in a Cartesian system of coordinates; and the first and third quadrants have equal data values and the second and fourth quadrants have equal data values.
Claim 6
Dependent Claim
38 words
The method of claim 4 , wherein the data values of the first and third quadrants and the data values of the second and fourth quadrants of the first and second patterns consist of two opposite polarity numbers.
Claim 7
Dependent Claim
40 words
The method of claim 5 , wherein the data values of the first and third quadrants of the first pattern and the data values of the first and third quadrants of the second pattern consist of two opposite polarity numbers.
Claim 8
Dependent Claim
61 words
The method of claim 3 , wherein the filtered frame of data includes a plurality of filtered data values, each filtered data value corresponding with one of the data values of the frame of data and being equal to a sum-of-products of data values of a reference data block and a respective set of data values of the frame of data.
Claim 9
Dependent Claim
74 words
The method of claim 7 , wherein the data values of the reference data block are arranged in one of the at least first and second patterns such that a given one of the filtered values tends to peak when the respective set of data values of the frame of data includes a given one of the data blocks of the watermark that is in at least partial registration with the reference data block.
Claim 10
Dependent Claim
100 words
The method of claim 8 , wherein: the given filtered value tends to peak with a positive polarity when both the given one of the data blocks of the watermark and the reference data block include data values arranged in the first or second pattern; and the given filtered value tends to peak with a negative polarity when the given one of the data blocks of the watermark includes data values arranged in the one of the first and second patterns and the reference data block includes data values arranged in the other one of the first and second patterns.
Claim 11
Dependent Claim
65 words
The method of claim 9 , wherein the reference data block includes a center within the array of data values; and the respective set of data values of the frame of data are those data values covered by the reference data block when the center of the reference data block is aligned with the corresponding one of the data values of the frame of data.
Claim 12
Dependent Claim
47 words
The method of claim 7 , wherein the step of computing the filtered frame of data comprises computing, for each filtered value, an absolute value of the sum-of-products of the values of the reference data block and the respective set of values of the frame of data.
Claim 13
Dependent Claim
41 words
The method of claim 1 , wherein the step of computing the two-dimensional Fourier transform of the filtered frame of data includes computing a two-dimensional Fast Fourier transform (FFT) to produce the two-dimensional frequency spectrum of the filtered frame of data.
Claim 14
Dependent Claim
44 words
The method of claim 1 , wherein the step of selecting the set of frequency components associated with the watermark includes selecting at least some of the harmonic frequency components corresponding to the watermark from among the frequency components of the two-dimensional frequency spectrum.
Claim 15
Dependent Claim
26 words
The method of claim 13 , wherein the at least some of the harmonic frequency components include the second harmonic frequency components corresponding to the watermark.
Claim 16
Dependent Claim
109 words
The method of claim 13 , wherein the step of computing at least one of the rotation value, the resizing value, and the translation value associated with the watermark includes determining (i) one or more deviations of the geometric positions within the two-dimensional Fourier transform of at least one of the selected harmonic frequency components corresponding to the watermark as compared with expected geometric positions of those components defined by the reference watermark; or (ii) one or more deviations in the phases of at least one of the selected harmonic frequency components corresponding to the watermark as compared with expected phases of those components defined by the reference watermark.
Claim 17
Dependent Claim
66 words
The method of claim 15 , wherein an area containing the frequency components of the two-dimensional frequency spectrum (i) may be graphically represented in Cartesian coordinates according to an x-axis and a y-axis; (ii) includes a width (W) parallel to the x-axis; (iii) includes a height (H) parallel to the y-axis; and (iv) includes an origin substantially at an intersection of the x-axis and the y-axis.
Claim 18
Dependent Claim
63 words
The method of claim 16 , wherein the step of computing the rotation value corresponding to the watermark includes determining an angle between a first axis and a reference axis, the first axis being defined by a reference point and at least one of the selected harmonic frequency components corresponding to the watermark and the reference axis being defined by the reference watermark.
Claim 19
Dependent Claim
19 words
The method of claim 17 , wherein the reference axis lies along one of the x-axis and the y-axis.
Claim 20
Dependent Claim
48 words
The method of claim 18 , wherein the rotation value corresponding to the watermark is substantially proportional to: arctangent((Fy 1 *W)/(Fx 1 *H)), where Fy 1 and Fx 1 are the y-axis and x-axis coordinates, respectively, of one of the selected harmonic frequency components corresponding to the watermark.
Claim 21
Dependent Claim
24 words
The method of claim 19 , wherein the rotation value corresponding to the watermark is substantially equal to (180/n)*arctangent((Fy 1 *W)/(Fx 1 *H)) degrees.
Claim 22
Dependent Claim
70 words
The method of claim 17 , wherein the step of computing the rotation value corresponding to the watermark includes: determining two or more angles between (i) respective axes defined by the reference point and respective ones of the selected harmonic frequency components corresponding to the watermark, and (ii) one or more reference axes defined by the reference watermark; and determining the rotation value from among the two or more angles.
Claim 23
Dependent Claim
50 words
The method of claim 16 , wherein the step of computing the resizing value corresponding to the watermark includes comparing (i) a period (P) computed from at least one of the selected harmonic frequency components corresponding to the watermark; and (ii) a reference period (Pr) defined by the reference watermark.
Claim 24
Dependent Claim
29 words
The method of claim 22 , wherein the step of computing the resizing value corresponding to the watermark includes computing an x-axis resizing value and a y-axis resizing value.
Claim 25
Dependent Claim
55 words
The method of claim 23 , wherein the step of computing the x-axis resizing value includes: computing an x-axis period, Px, from one of the selected harmonic frequency components corresponding to the watermark; and computing a ratio of Px to an x-axis reference period, Prx, of the reference watermark to obtain the x-axis resizing value.
Claim 26
Dependent Claim
50 words
The method of claim 24 , wherein the x-axis period, Px, is substantially equal to (H*W)/√((Fx 1 *H) 2 +(Fy 1 *W) 2 ), where Fx 1 and Fy 1 are the x-axis and y-axis coordinates, respectively, of the one of the selected harmonic frequency components corresponding to the watermark.
Claim 27
Dependent Claim
50 words
The method of claim 25 , wherein the reference watermark includes a plurality of data blocks, each data block including a plurality of values arranged in rows and columns, the number of values in each of the rows being equal to the x-axis reference period, Prx, of the reference watermark.
Claim 28
Dependent Claim
36 words
The method of claim 26 , wherein the one of the selected harmonic frequency components corresponding to the watermark lies substantially along the x-axis or would lie substantially along the x-axis but for the rotation value.
Claim 29
Dependent Claim
55 words
The method of claim 23 , wherein the step of computing the y-axis resizing value includes: computing a y-axis period, Py, from one of the selected harmonic frequency components corresponding to the watermark; and computing a ratio of Py to a y-axis reference period, Pry, of the reference watermark to obtain the y-axis resizing value.
Claim 30
Dependent Claim
50 words
The method of claim 28 , wherein the y-axis period, Py, is substantially equal to (H*W)/√((Fx 2 *H) 2 +(Fy 2 *W) 2 ), where Fx 2 and Fy 2 are the x-axis and y-axis coordinates, respectively, of the one of the selected harmonic frequency components corresponding to the watermark.
Claim 31
Dependent Claim
50 words
The method of claim 29 , wherein the reference watermark includes a plurality of data blocks, each data block including a plurality of values arranged in rows and columns, the number of values in each of the columns being equal to the y-axis reference period, Pry, of the reference watermark.
Claim 32
Dependent Claim
36 words
The method of claim 30 , wherein the one of the selected harmonic frequency components corresponding to the watermark lies substantially along the y-axis or would lie substantially along the y-axis but for the rotation value.
Claim 33
Dependent Claim
35 words
The method of claim 16 , wherein the step of computing the translation value corresponding to the watermark includes obtaining phases from at least two of the selected harmonic frequency components corresponding to the watermark.
Claim 34
Dependent Claim
75 words
The method of claim 32 , wherein: the translation value is based on a first distance in a first direction and a second distance in a second direction; the first distance is proportional to a phase, θ 1 , of one of the selected harmonic frequency components corresponding to the watermark; and the second distance is proportional to a phase, θ 2 , of another of the selected harmonic frequency components corresponding to the watermark.
Claim 35
Dependent Claim
96 words
The method of claim 33 , wherein: the first distance is substantially equal to (θ 1 *H*W)/2n*√((Fx 1 *H) 2 +(Fy 1 *W) 2 ), where Fx 1 and Fy 1 are the x-axis and y-axis coordinates, respectively, of the one of the selected harmonic frequency components corresponding to the watermark; and the second distance is substantially equal to (θ 2 *H*W)/2n*√((Fx 2 *H) 2 +(Fy 2 *W) 2 ), where Fx 2 and Fy 2 are the x-axis and y-axis coordinates, respectively, of the other of the selected harmonic frequency components corresponding to the watermark.
Claim 36
Dependent Claim
32 words
The method of claim 34 , wherein the first direction is substantially proportional to arctangent((Fy 1 *W)/(Fx 1 *H)); and the second direction is substantially proportional to arctangent((Fy 2 *W)/(Fx 2 *H)).
Claim 37
Dependent Claim
18 words
The method of claim 33 , wherein the first and second directions are substantially perpendicular to one another.
Claim 38
Dependent Claim
50 words
The method of claim 3 , further comprising determining a plurality of locations within the frame of data corresponding to at least some of the data blocks of the watermark in accordance with at least one of the rotation value, the resizing value, and the translation value of the watermark.
Claim 39
Dependent Claim
22 words
The method of claim 37 , wherein the plurality of locations correspond to respective centers of the data blocks of the watermark.
Claim 40
Dependent Claim
70 words
The method of claim 37 , further comprising determining a filtered array of message values, each filtered message value corresponding with one of the at least some of the data blocks of the watermark and being equal to a sum-of-products of data values of a reference data block and a respective set of data values of the one of the at least some of the data blocks of the watermark.
Claim 41
Dependent Claim
81 words
The method of claim 39 , wherein the reference data block includes a center within the array of data values; and the respective set of data values of the one of the at least some of the data blocks of the watermark are those data values covered by the reference data block when the center of the reference data block is aligned with the respective location of the one of the at least some of the data blocks of the watermark.
Claim 42
Dependent Claim
53 words
The method of claim 40 , further comprising at least one of (i) rotating the reference data block by an amount proportional to the rotation value prior to taking the sum-of-products; and (ii) resizing the reference data block by one or more amounts proportional to the resizing value prior to taking the sum-of-products.
Claim 43
Dependent Claim
70 words
The method of claim 39 , further comprising determining a filtered array of reference message values, each filtered reference message value corresponding with one of at least some of the data blocks of the reference watermark and being equal to a sum-of-products of data values of the reference data block and the data values of the one of the at least some of the data blocks of the reference watermark.
Claim 44
Dependent Claim
79 words
The method of claim 42 , wherein the reference data block includes a center within the array of data values; and the data values of the one of the at least some of the data blocks of the reference watermark are those data values covered by the reference data block when the center of the reference data block is aligned with a center of the one of the at least some of the data blocks of the reference watermark.
Claim 45
Dependent Claim
16 words
The method of claim 42 , wherein the filtered array of reference message values is predetermined.
Claim 46
Dependent Claim
26 words
The method of claim 44 , further comprising computing a sum-of-products of the filtered array of message values and the filtered array of reference message values.
Claim 47
Dependent Claim
47 words
The method of claim 45 , further comprising determining that the watermark contains a message that corresponds to a message of the reference watermark when the sum-of-products of the filtered array of message values and the filtered array of reference message values meets or exceeds a threshold.
Claim 48
Dependent Claim
102 words
The method of claim 46 , further comprising: determining a set of filtered arrays of reference message values, each array of filtered reference message values being associated with a different reference watermark; computing a sum-of-products of the filtered array of message values and each of the filtered arrays of reference message values; and determining that the watermark contains a message that corresponds to a message of one of the reference watermarks when the sum-of-products of the filtered array of message values and the filtered array of reference message values associated with the one of the reference watermarks meets or exceeds a threshold.
Claim 49
Dependent Claim
100 words
The method of claim 45 , further comprising: translating the filtered array of message values with respect to the filtered array of reference message values by at least one message value location in at least one of a horizontal direction and a vertical direction; computing a sum-of-products of the filtered array of message values and the filtered array of reference message values; repeating the translating and computing steps one or more times; and determining that the watermark contains a message that corresponds to a message of the reference watermark when one of the sum-of-products computations meets or exceeds a threshold.
Claim 50
Dependent Claim
88 words
The method of claim 45 , further comprising: rotating the filtered array of message values with respect to the filtered array of reference message values in accordance with a multiple of 90°; computing a sum-of-products of the filtered array of message values and the filtered array of reference message values; repeating the rotating and computing steps one or more times; and determining that the watermark contains a message that corresponds to a message of the reference watermark when one of the sum-of-products computations meets or exceeds a threshold.
Claim 51
Dependent Claim
224 words
The method of claim 42 , further comprising obtaining a two-dimensional Fourier transform of the filtered array of reference message values; computing a two-dimensional Fourier transform of the filtered array of message values; computing a modified array by taking a product of the two-dimensional Fourier transform of the filtered array of reference message values and the two-dimensional Fourier transform of the filtered array of message values on a point-by-point basis; computing an inverse Fourier transform of the modified array; determining a maximum value from the inverse Fourier transform of the modified array; rotating the two-dimensional Fourier transform of the filtered array of reference message values relative to the two-dimensional Fourier transform of the filtered array of message values by a multiple of 90°; computing subsequent modified array by taking a product of the two-dimensional Fourier transform of the filtered array of reference message values and the two-dimensional Fourier transform of the filtered array of message values on a point-by-point basis; computing an inverse Fourier transform of the subsequent modified array; determining a subsequent maximum value from the inverse Fourier transform of the subsequent modified array; repeating the rotating, computing, and determining steps one or more times; and determining that the watermark contains a message that corresponds to a message of the reference watermark when one of the maximum values meets or exceeds a threshold.
Claim 52
Dependent Claim
173 words
A processing device operating under the control of a program, the program causing the processing device to carry out a method of detecting a watermark in a two-dimensional frame of data, the frame of data including a plurality of data values representing the watermark embedded in a frame of content data, the method comprising: computing a filtered frame of data from the frame of data having at least some harmonic frequency components corresponding to the watermark that are emphasized as compared to at least some frequency components corresponding to the content data; computing a two-dimensional Fourier transform of the filtered frame of data to produce a two-dimensional frequency spectrum of the filtered frame of data; selecting a set of frequency components associated with the watermark from among the frequency components of the two-dimensional frequency spectrum; and computing at least one of a rotation value, a resizing value, and a translation value associated with the watermark as compared to a reference watermark using one or more of the frequency components of the selected set.
Claim 53
Dependent Claim
173 words
A processing device operating under the control of a program, the program causing the processing device to carry out a method of detecting a watermark in a two-dimensional frame of data, the frame of data including a plurality of data values representing the watermark embedded in a frame of content data, the method comprising: computing a filtered frame of data from the frame of data having at least some harmonic frequency components corresponding to the watermark that are emphasized as compared to at least some frequency components corresponding to the content data; computing a two-dimensional Fourier transform of the filtered frame of data to produce a two-dimensional frequency spectrum of the filtered frame of data; selecting a set of frequency components associated with the watermark from among the frequency components of the two-dimensional frequency spectrum; and computing at least one of a rotation value, a resizing value, and a translation value associated with the watermark as compared to a reference watermark using one or more of the frequency components of the selected set.
Claim 54
Dependent Claim
74 words
The processing device of claim 51 , wherein: the watermark substantially matches the reference watermark but for the at least one of the rotation value, the resizing value, and the translation value; the watermark and the reference watermark each include a respective plurality of data blocks representing a message; and each data block includes an array of data values arranged in at least one of a first and second pattern defined by the message.
Claim 55
Dependent Claim
112 words
The processing device of claim 52 , wherein: the first and second patterns may be defined by four quadrants of data values in a Cartesian system of coordinates; and the first and third quadrants have equal data values and the second and fourth quadrants have equal data values; the data values of the first and third quadrants and the data values of the second and fourth quadrants of the first and second patterns consist of two opposite polarity numbers; the data values of the first and third quadrants of the first pattern and the data values of the first and third quadrants of the second pattern consist of two opposite polarity numbers.
Claim 56
Dependent Claim
62 words
The processing device of claim 52 , wherein the filtered frame of data includes a plurality of filtered data values, each filtered data value corresponding with one of the data values of the frame of data and being equal to a sum-of-products of data values of a reference data block and a respective set of data values of the frame of data.
Claim 57
Dependent Claim
75 words
The processing device of claim 54 , wherein the data values of the reference data block are arranged in one of the at least first and second patterns such that a given one of the filtered values tends to peak when the respective set of data values of the frame of data includes a given one of the data blocks of the watermark that is in at least partial registration with the reference data block.
Claim 58
Dependent Claim
101 words
The processing device of claim 55 , wherein: the given filtered value tends to peak with a positive polarity when both the given one of the data blocks of the watermark and the reference data block include data values arranged in the first or second pattern; and the given filtered value tends to peak with a negative polarity when the given one of the data blocks of the watermark includes data values arranged in the one of the first and second patterns and the reference data block includes data values arranged in the other one of the first and second patterns.
Claim 59
Dependent Claim
66 words
The processing device of claim 56 , wherein the reference data block includes a center within the array of data values; and the respective set of data values of the frame of data are those data values covered by the reference data block when the center of the reference data block is aligned with the corresponding one of the data values of the frame of data.
Claim 60
Dependent Claim
48 words
The processing device of claim 54 , wherein the step of computing the filtered frame of data comprises computing, for each filtered value, an absolute value of the sum-of-products of the values of the reference data block and the respective set of values of the frame of data.
Claim 61
Dependent Claim
42 words
The processing device of claim 51 , wherein the step of computing the two-dimensional Fourier transform of the filtered frame of data includes computing a two-dimensional Fast Fourier transform (FFT) to produce the two-dimensional frequency spectrum of the filtered frame of data.
Claim 62
Dependent Claim
45 words
The processing device of claim 51 , wherein the step of selecting the set of frequency components associated with the watermark includes selecting at least some of the harmonic frequency components corresponding to the watermark from among the frequency components of the two-dimensional frequency spectrum.
Claim 63
Dependent Claim
27 words
The processing device of claim 60 , wherein the at least some of the harmonic frequency components include the second harmonic frequency components corresponding to the watermark.
Claim 64
Dependent Claim
110 words
The processing device of claim 60 , wherein the step of computing at least one of the rotation value, the resizing value, and the translation value associated with the watermark includes determining (i) one or more deviations of the geometric positions within the two-dimensional Fourier transform of at least one of the selected harmonic frequency components corresponding to the watermark as compared with expected geometric positions of those components defined by the reference watermark; or (ii) one or more deviations in the phases of at least one of the selected harmonic frequency components corresponding to the watermark as compared with expected phases of those components defined by the reference watermark.
Claim 65
Dependent Claim
67 words
The processing device of claim 62 , wherein an area containing the frequency components of the two-dimensional frequency spectrum (i) may be graphically represented in Cartesian coordinates according to an x-axis and a y-axis; (ii) includes a width (W) parallel to the x-axis; (iii) includes a height (H) parallel to the y-axis; and (iv) includes an origin substantially at an intersection of the x-axis and the y-axis.
Claim 66
Dependent Claim
64 words
The processing device of claim 63 , wherein the step of computing the rotation value corresponding to the watermark includes determining an angle between a first axis and a reference axis, the first axis being defined by a reference point and at least one of the selected harmonic frequency components corresponding to the watermark and the reference axis being defined by the reference watermark.
Claim 67
Dependent Claim
20 words
The processing device of claim 64 , wherein the reference axis lies along one of the x-axis and the y-axis.
Claim 68
Dependent Claim
49 words
The processing device of claim 65 , wherein the rotation value corresponding to the watermark is substantially proportional to: arctangent((Fy 1 *W)/(Fx 1 *H)), where Fy 1 and Fx 1 are the y-axis and x-axis coordinates, respectively, of one of the selected harmonic frequency components corresponding to the watermark.
Claim 69
Dependent Claim
25 words
The processing device of claim 66 , wherein the rotation value corresponding to the watermark is substantially equal to (180/n)*arctangent((Fy 1 *W)/(Fx 1 *H)) degrees.
Claim 70
Dependent Claim
71 words
The processing device of claim 64 , wherein the step of computing the rotation value corresponding to the watermark includes: determining two or more angles between (i) respective axes defined by the reference point and respective ones of the selected harmonic frequency components corresponding to the watermark, and (ii) one or more reference axes defined by the reference watermark; and determining the rotation value from among the two or more angles.
Claim 71
Dependent Claim
30 words
The processing device of claim 63 , wherein the step of computing the resizing value corresponding to the watermark includes computing an x-axis resizing value and a y-axis resizing value.
Claim 72
Dependent Claim
59 words
The processing device of claim 69 , wherein the step of computing the x-axis resizing value includes: computing an x-axis period, Px, from the origin to one of the selected harmonic frequency components corresponding to the watermark; and computing a ratio of Px to an x-axis reference period, Prx, of the reference watermark to obtain the x-axis resizing value.
Claim 73
Dependent Claim
51 words
The processing device of claim 70 , wherein the x-axis period, Px, is substantially equal to (H*W)/√((Fx 1 *H) 2 +(Fy 1 *W) 2 ), where Fx 1 and Fy 1 are the x-axis and y-axis coordinates, respectively, of the one of the selected harmonic frequency components corresponding to the watermark.
Claim 74
Dependent Claim
51 words
The processing device of claim 71 , wherein the reference watermark includes a plurality of data blocks, each data block including a plurality of values arranged in rows and columns, the number of values in each of the rows being equal to the x-axis reference period, Prx, of the reference watermark.
Claim 75
Dependent Claim
37 words
The processing device of claim 72 , wherein the one of the selected harmonic frequency components corresponding to the watermark lies substantially along the x-axis or would lie substantially along the x-axis but for the rotation value.
Claim 76
Dependent Claim
59 words
The processing device of claim 69 , wherein the step of computing the y-axis resizing value includes: computing a y-axis period, Py, from the origin to one of the selected harmonic frequency components corresponding to the watermark; and computing a ratio of Py to a y-axis reference period, Pry, of the reference watermark to obtain the y-axis resizing value.
Claim 77
Dependent Claim
51 words
The processing device of claim 74 , wherein the y-axis period, Py, is substantially equal to (H*W)/√(Fx 2 *H) 2 +(Fy 2 *W) 2 ), where Fx 2 and Fy 2 are the x-axis and y-axis coordinates, respectively, of the one of the selected harmonic frequency components corresponding to the watermark.
Claim 78
Dependent Claim
51 words
The processing device of claim 75 , wherein the reference watermark includes a plurality of data blocks, each data block including a plurality of values arranged in rows and columns, the number of values in each of the columns being equal to the y-axis reference period, Pry, of the reference watermark.
Claim 79
Dependent Claim
37 words
The processing device of claim 76 , wherein the one of the selected harmonic frequency components corresponding to the watermark lies substantially along the y-axis or would lie substantially along the y-axis but for the rotation value.
Claim 80
Dependent Claim
76 words
The processing device of claim 63 , wherein: the translation value is based on a first distance in a first direction and a second distance in a second direction; the first distance is proportional to a phase, θ 1 , of one of the selected harmonic frequency components corresponding to the watermark; and the second distance is proportional to a phase, θ 2 , of another of the selected harmonic frequency components corresponding to the watermark.
Claim 81
Dependent Claim
97 words
The processing device of claim 78 , wherein: the first distance is substantially equal to (θ 1 *H*W)/2n*√((Fx 1 *H) 2 +(Fy 1 *W) 2 ), where Fx 1 and Fy 1 are the x-axis and y-axis coordinates, respectively, of the one of the selected harmonic frequency components corresponding to the watermark; and the second distance is substantially equal to (θ 2 *H*W)/2n*√((Fx 2 *H) 2 +(Fy 2 *W) 2 ), where Fx 2 and Fy 2 are the x-axis and y-axis coordinates, respectively, of the other of the selected harmonic frequency components corresponding to the watermark.
Claim 82
Dependent Claim
33 words
The processing device of claim 79 , wherein the first direction is substantially proportional to arctangent((Fy 1 *W)/(Fx 1 *H)); and the second direction is substantially proportional to arctangent((Fy 2 *W)/(Fx 2 *H)).
Claim 83
Dependent Claim
19 words
The processing device of claim 70 , wherein the first and second directions are substantially perpendicular to one another.
Claim 84
Dependent Claim
54 words
The processing device of claim 52 , wherein the method further comprises determining a plurality of locations within the frame of data corresponding to at least some of the data blocks of the watermark in accordance with at least one of the rotation value, the resizing value, and the translation value of the watermark.
Claim 85
Dependent Claim
23 words
The processing device of claim 82 , wherein the plurality of locations correspond to respective centers of the data blocks of the watermark.
Claim 86
Dependent Claim
74 words
The processing device of claim 82 , wherein the method further comprises determining a filtered array of message values, each filtered message value corresponding with one of the at least some of the data blocks of the watermark and being equal to a sum-of-products of data values of a reference data block and a respective set of data values of the one of the at least some of the data blocks of the watermark.
Claim 87
Dependent Claim
82 words
The processing device of claim 84 , wherein the reference data block includes a center within the array of data values; and the respective set of data values of the one of the at least some of the data blocks of the watermark are those data values covered by the reference data block when the center of the reference data block is aligned with the respective location of the one of the at least some of the data blocks of the watermark.
Claim 88
Dependent Claim
57 words
The processing device of claim 85 , wherein the method further comprises at least one of (i) rotating the reference data block by an amount proportional to the rotation value prior to taking the sum-of-products; and (ii) resizing the reference data block by one or more amounts proportional to the resizing value prior to taking the sum-of-products.
Claim 89
Dependent Claim
74 words
The processing device of claim 84 , wherein the method further comprises determining a filtered array of reference message values, each filtered reference message value corresponding with one of at least some of the data blocks of the reference watermark and being equal to a sum-of-products of data values of the reference data block and the data values of the one of the at least some of the data blocks of the reference watermark.
Claim 90
Dependent Claim
80 words
The processing device of claim 87 , wherein the reference data block includes a center within the array of data values; and the data values of the one of the at least some of the data blocks of the reference watermark are those data values covered by the reference data block when the center of the reference data block is aligned with a center of the one of the at least some of the data blocks of the reference watermark.
Claim 91
Dependent Claim
17 words
The processing device of claim 87 , wherein the filtered array of reference message values is predetermined.
Claim 92
Dependent Claim
30 words
The processing device of claim 89 , wherein the method further comprises computing a sum-of-products of the filtered array of message values and the filtered array of reference message values.
Claim 93
Dependent Claim
51 words
The processing device of claim 90 , wherein the method further comprises determining that the watermark contains a message that corresponds to a message of the reference watermark when the sum-of-products of the filtered array of message values and the filtered array of reference message values meets or exceeds a threshold.
Claim 94
Dependent Claim
106 words
The processing device of claim 91 , wherein the method further comprises: determining a set of filtered arrays of reference message values, each array of filtered reference message values being associated with a different reference watermark; computing a sum-of-products of the filtered array of message values and each of the filtered arrays of reference message values; and determining that the watermark contains a message that corresponds to a message of one of the reference watermarks when the sum-of-products of the filtered array of message values and the filtered array of reference message values associated with the one of the reference watermarks meets or exceeds a threshold.
Claim 95
Dependent Claim
103 words
The processing device of claim 90 , wherein the method further comprises: translating the filtered array of message values with respect to the filtered array of reference message values by at least one message location in at least one of a horizontal direction and a vertical direction; computing a sum-of-products of the filtered array of message values and the filtered array of reference message values; repeating the translating and computing steps one or more times; and determining that the watermark contains a message that corresponds to a message of the reference watermark when one of the sum-of-products computations meets or exceeds a threshold.
Claim 96
Dependent Claim
92 words
The processing device of claim 90 , wherein the method further comprises: rotating the filtered array of message values with respect to the filtered array of reference message values in accordance with a multiple of 90°; computing a sum-of-products of the filtered array of message values and the filtered array of reference message values; repeating the rotating and computing steps one or more times; and determining that the watermark contains a message that corresponds to a message of the reference watermark when one of the sum-of-products computations meets or exceeds a threshold.
Claim 97
Dependent Claim
228 words
The processing device of claim 87 , wherein the method further comprises: obtaining a two-dimensional Fourier transform of the filtered array of reference message values; computing a two-dimensional Fourier transform of the filtered array of message values; computing a modified array by taking a product of the two-dimensional Fourier transform of the filtered array of reference message values and the two-dimensional Fourier transform of the filtered array of message values on a point-by-point basis; computing an inverse Fourier transform of the modified array; determining a maximum value from the inverse Fourier transform of the modified array; rotating the two-dimensional Fourier transform of the filtered array of reference message values relative to the two-dimensional Fourier transform of the filtered array of message values by a multiple of 90°; computing subsequent modified array by taking a product of the two-dimensional Fourier transform of the filtered array of reference message values and the two-dimensional Fourier transform of the filtered array of message values on a point-by-point basis; computing an inverse Fourier transform of the subsequent modified array; determining a subsequent maximum value from the inverse Fourier transform of the subsequent modified array; repeating the rotating, computing, and determining steps one or more times; and determining that the watermark contains a message that corresponds to a message of the reference watermark when one of the maximum values meets or exceeds a threshold.
Claim 98
Dependent Claim
168 words
A storage medium containing a program for causing a processing device to carry out a method of detecting a watermark in a two-dimensional frame of data, the frame of data including a plurality of data values representing the watermark embedded in a frame of content data, the method comprising: computing a filtered frame of data from the frame of data having at least some harmonic frequency components corresponding to the watermark that are emphasized as compared to at least some frequency components corresponding to the content data; computing a two-dimensional Fourier transform of the filtered frame of data to produce a two-dimensional frequency spectrum of the filtered frame of data; selecting a set of frequency components associated with the watermark from among the frequency components of the two-dimensional frequency spectrum; and computing at least one of a rotation value, a resizing value, and a translation value associated with the watermark as compared to a reference watermark using one or more of the frequency components of the selected set.
Claim 99
Dependent Claim
168 words
A storage medium containing a program for causing a processing device to carry out a method of detecting a watermark in a two-dimensional frame of data, the frame of data including a plurality of data values representing the watermark embedded in a frame of content data, the method comprising: computing a filtered frame of data from the frame of data having at least some harmonic frequency components corresponding to the watermark that are emphasized as compared to at least some frequency components corresponding to the content data; computing a two-dimensional Fourier transform of the filtered frame of data to produce a two-dimensional frequency spectrum of the filtered frame of data; selecting a set of frequency components associated with the watermark from among the frequency components of the two-dimensional frequency spectrum; and computing at least one of a rotation value, a resizing value, and a translation value associated with the watermark as compared to a reference watermark using one or more of the frequency components of the selected set.
Claim 100
Dependent Claim
73 words
The storage medium of claim 96 , wherein: the watermark substantially matches the reference watermark but for the at least one of the rotation value, the resizing value, and the translation value; the watermark and the reference watermark include a respective plurality of data blocks representing a message; and each data block includes an array of data values arranged in at least one of a first and second pattern defined by the message.
Claim 101
Dependent Claim
112 words
The storage medium of claim 97 wherein: the first and second patterns may be defined by four quadrants of data values in a Cartesian system of coordinates; and the first and third quadrants have equal data values and the second and fourth quadrants have equal data values; the data values of the first and third quadrants and the data values of the second and fourth quadrants of the first and second patterns consist of two opposite polarity numbers; and the data values of the first and third quadrants of the first pattern and the data values of the first and third quadrants of the second pattern consist of two opposite polarity numbers.
Claim 102
Dependent Claim
62 words
The storage medium of claim 97 , wherein the filtered frame of data includes a plurality of filtered data values, each filtered data value corresponding with one of the data values of the frame of data and being equal to a sum-of-products of data values of a reference data block and a respective set of data values of the frame of data.
Claim 103
Dependent Claim
75 words
The storage medium of claim 99 , wherein the data values of the reference data block are arranged in one of the at least first and second patterns such that a given one of the filtered values tends to peak when the respective set of data values of the frame of data includes a given one of the data blocks of the watermark that is in at least partial registration with the reference data block.
Claim 104
Dependent Claim
101 words
The storage medium of claim 100 , wherein: the given filtered value tends to peak with a positive polarity when both the given one of the data blocks of the watermark and the reference data block include data values arranged in the first or second pattern; and the given filtered value tends to peak with a negative polarity when the given one of the data blocks of the watermark includes data values arranged in the one of the first and second patterns and the reference data block includes data values arranged in the other one of the first and second patterns.
Claim 105
Dependent Claim
66 words
The storage medium of claim 101 , wherein the reference data block includes a center within the array of data values; and the respective set of data values of the frame of data are those data values covered by the reference data block when the center of the reference data block is aligned with the corresponding one of the data values of the frame of data.
Claim 106
Dependent Claim
48 words
The storage medium of claim 99 , wherein the step of computing the filtered frame of data comprises computing, for each filtered value, an absolute value of the sum-of-products of the values of the reference data block and the respective set of values of the frame of data.
Claim 107
Dependent Claim
42 words
The storage medium of claim 96 , wherein the step of computing the two-dimensional Fourier transform of the filtered frame of data includes computing a two-dimensional Fast Fourier transform (FFT) to produce the two-dimensional frequency spectrum of the filtered frame of data.
Claim 108
Dependent Claim
45 words
The storage medium of claim 96 , wherein the step of selecting the set of frequency components associated with the watermark includes selecting at least some of the harmonic frequency components corresponding to the watermark from among the frequency components of the two-dimensional frequency spectrum.
Claim 109
Dependent Claim
27 words
The storage medium of claim 105 , wherein the at least some of the harmonic frequency components include the second harmonic frequency components corresponding to the watermark.
Claim 110
Dependent Claim
110 words
The storage medium of claim 105 , wherein the step of computing at least one of the rotation value, the resizing value, and the translation value associated with the watermark includes determining (i) one or more deviations of the geometric positions within the two-dimensional Fourier transform of at least one of the selected harmonic frequency components corresponding to the watermark as compared with expected geometric positions of those components defined by the reference watermark; or (ii) one or more deviations in the phases of at least one of the selected harmonic frequency components corresponding to the watermark as compared with expected phases of those components defined by the reference watermark.
Claim 111
Dependent Claim
67 words
The storage medium of claim 107 , wherein an area containing the frequency components of the two-dimensional frequency spectrum (i) may be graphically represented in Cartesian coordinates according to an x-axis and a y-axis; (ii) includes a width (W) parallel to the x-axis; (iii) includes a height (H) parallel to the y-axis; and (iv) includes an origin substantially at an intersection of the x-axis and the y-axis.
Claim 112
Dependent Claim
64 words
The storage medium of claim 108 , wherein the step of computing the rotation value corresponding to the watermark includes determining an angle between a first axis and a reference axis, the first axis being defined by a reference point and at least one of the selected harmonic frequency components corresponding to the watermark and the reference axis being defined by the reference watermark.
Claim 113
Dependent Claim
20 words
The storage medium of claim 109 , wherein the reference axis lies along one of the x-axis and the y-axis.
Claim 114
Dependent Claim
49 words
The storage medium of claim 110 , wherein the rotation value corresponding to the watermark is substantially proportional to: arctangent((Fy 1 *W)/(Fx 1 *H)), where Fy 1 and Fx 1 are the y-axis and x-axis coordinates, respectively, of one of the selected harmonic frequency components corresponding to the watermark.
Claim 115
Dependent Claim
25 words
The storage medium of claim 111 , wherein the rotation value corresponding to the watermark is substantially equal to (180/n)*arctangent((Fy 1 *W)/(Fx 1 *H)) degrees.
Claim 116
Dependent Claim
71 words
The storage medium of claim 109 , wherein the step of computing the rotation value corresponding to the watermark includes: determining two or more angles between (i) respective axes defined by the reference point and respective ones of the selected harmonic frequency components corresponding to the watermark, and (ii) one or more reference axes defined by the reference watermark; and determining the rotation value from among the two or more angles.
Claim 117
Dependent Claim
30 words
The storage medium of claim 108 , wherein the step of computing the resizing value corresponding to the watermark includes computing an x-axis resizing value and a y-axis resizing value.
Claim 118
Dependent Claim
59 words
The storage medium of claim 114 , wherein the step of computing the x-axis resizing value includes: computing an x-axis period, Px, from the origin to one of the selected harmonic frequency components corresponding to the watermark; and computing a ratio of Px to an x-axis reference period, Prx, of the reference watermark to obtain the x-axis resizing value.
Claim 119
Dependent Claim
51 words
The storage medium of claim 115 , wherein the x-axis period, Px, is substantially equal to (H*W)/√((Fx 1 *H) 2 +(Fy 1 *W) 2 ), where Fx 1 and Fy 1 are the x-axis and y-axis coordinates, respectively, of the one of the selected harmonic frequency components corresponding to the watermark.
Claim 120
Dependent Claim
51 words
The storage medium of claim 116 , wherein the reference watermark includes a plurality of data blocks, each data block including a plurality of values arranged in rows and columns, a number of values in each of the rows being equal to the x-axis reference period, Prx, of the reference watermark.
Claim 121
Dependent Claim
37 words
The storage medium of claim 117 , wherein the one of the selected harmonic frequency components corresponding to the watermark lies substantially along the x-axis or would lie substantially along the x-axis but for the rotation value.
Claim 122
Dependent Claim
59 words
The storage medium of claim 114 , wherein the step of computing the y-axis resizing value includes: computing a y-axis period, Py, from the origin to one of the selected harmonic frequency components corresponding to the watermark; and computing a ratio of Py to a y-axis reference period, Pry, of the reference watermark to obtain the y-axis resizing value.
Claim 123
Dependent Claim
51 words
The storage medium of claim 119 , wherein the y-axis period, Py, is substantially equal to (H*W)/√((Fx 2 *H) 2 +(Fy 2 *W) 2 ), where Fx 2 and Fy 2 are the x-axis and y-axis coordinates, respectively, of the one of the selected harmonic frequency components corresponding to the watermark.
Claim 124
Dependent Claim
51 words
The storage medium of claim 120 , wherein the reference watermark includes a plurality of data blocks, each data block including a plurality of values arranged in rows and columns, a number of values in each of the columns being equal to the y-axis reference period, Pry, of the reference watermark.
Claim 125
Dependent Claim
37 words
The storage medium of claim 121 , wherein the one of the selected harmonic frequency components corresponding to the watermark lies substantially along the y-axis or would lie substantially along the y-axis but for the rotation value.
Claim 126
Dependent Claim
76 words
The storage medium of claim 108 , wherein: the translation value is based on a first distance in a first direction and a second distance in a second direction; the first distance is proportional to a phase, θ 1 , of one of the selected harmonic frequency components corresponding to the watermark; and the second distance is proportional to a phase, θ 2 , of another of the selected harmonic frequency components corresponding to the watermark.
Claim 127
Dependent Claim
97 words
The storage medium of claim 123 , wherein: the first distance is substantially equal to (θ 1 *H*W)/2n*√((Fx 1 *H) 2 +(Fy 1 *W) 2 ), where Fx 1 and Fy 1 are the x-axis and y-axis coordinates, respectively, of the one of the selected harmonic frequency components corresponding to the watermark; and the second distance is substantially equal to (θ 2 *H*W)/2n*√((Fx 2 *H) 2 +(Fy 2 *W) 2 ), where Fx 2 and Fy 2 are the x-axis and y-axis coordinates, respectively, of the other of the selected harmonic frequency components corresponding to the watermark.
Claim 128
Dependent Claim
33 words
The storage medium of claim 124 , wherein the first direction is substantially proportional to arctangent((Fy 1 *W)/(Fx 1 *H)); and the second direction is substantially proportional to arctangent((Fy 2 *W)/(Fx 2 *H)).
Claim 129
Dependent Claim
19 words
The storage medium of claim 123 , wherein the first and second directions are substantially perpendicular to one another.
Claim 130
Dependent Claim
54 words
The storage medium of claim 97 , wherein the method further comprises determining a plurality of locations within the frame of data corresponding to at least some of the data blocks of the watermark in accordance with at least one of the rotation value, the resizing value, and the translation value of the watermark.
Claim 131
Dependent Claim
23 words
The storage medium of claim 127 , wherein the plurality of locations correspond to respective centers of the data blocks of the watermark.
Claim 132
Dependent Claim
74 words
The storage medium of claim 127 , wherein the method further comprises determining a filtered array of message values, each filtered message value corresponding with one of the at least some of the data blocks of the watermark and being equal to a sum-of-products of data values of a reference data block and a respective set of data values of the one of the at least some of the data blocks of the watermark.
Claim 133
Dependent Claim
82 words
The storage medium of claim 129 , wherein the reference data block includes a center within the array of data values; and the respective set of data values of the one of the at least some of the data blocks of the watermark are those data values covered by the reference data block when the center of the reference data block is aligned with the respective location of the one of the at least some of the data blocks of the watermark.
Claim 134
Dependent Claim
57 words
The storage medium of claim 130 , wherein the method further comprises at least one of (i) rotating the reference data block by an amount proportional to the rotation value prior to taking the sum-of-products; and (ii) resizing the reference data block by one or more amounts proportional to the resizing value prior to taking the sum-of-products.
Claim 135
Dependent Claim
74 words
The storage medium of claim 129 , wherein the method further comprises determining a filtered array of reference message values, each filtered reference message value corresponding with one of at least some of the data blocks of the reference watermark and being equal to a sum-of-products of data values of the reference data block and the data values of the one of the at least some of the data blocks of the reference watermark.
Claim 136
Dependent Claim
80 words
The storage medium of claim 132 , wherein the reference data block includes a center within the array of data values; and the data values of the one of the at least some of the data blocks of the reference watermark are those data values covered by the reference data block when the center of the reference data block is aligned with a center of the one of the at least some of the data blocks of the reference watermark.
Claim 137
Dependent Claim
17 words
The storage medium of claim 132 , wherein the filtered array of reference message values is predetermined.
Claim 138
Dependent Claim
30 words
The storage medium of claim 134 , wherein the method further comprises computing a sum-of-products of the filtered array of message values and the filtered array of reference message values.
Claim 139
Dependent Claim
51 words
The storage medium of claim 135 , wherein the method further comprises determining that the watermark contains a message that corresponds to a message of the reference watermark when the sum-of-products of the filtered array of message values and the filtered array of reference message values meets or exceeds a threshold.
Claim 140
Dependent Claim
106 words
The storage medium of claim 136 , wherein the method further comprises: determining a set of filtered arrays of reference message values, each array of filtered reference message values being associated with a different reference watermark; computing a sum-of-products of the filtered array of message values and each of the filtered arrays of reference message values; and determining that the watermark contains a message that corresponds to a message of one of the reference watermarks when the sum-of-products of the filtered array of message values and the filtered array of reference message values associated with the one of the reference watermarks meets or exceeds a threshold.
Claim 141
Dependent Claim
104 words
The storage medium of claim 135 , wherein the method further comprises: translating the filtered array of message values with respect to the filtered array of reference message values by at least one message value location in at least one of a horizontal direction and a vertical direction; computing a sum-of-products of the filtered array of message values and the filtered array of reference message values; repeating the translating and computing steps one or more times; and determining that the watermark contains a message that corresponds to a message of the reference watermark when one of the sum-of-products computations meets or exceeds a threshold.
Claim 142
Dependent Claim
92 words
The storage medium of claim 135 , wherein the method further comprises: rotating the filtered array of message values with respect to the filtered array of reference message values in accordance with a multiple of 90°; computing a sum-of-products of the filtered array of message values and the filtered array of reference message values; repeating the rotating and computing steps one or more times; and determining that the watermark contains a message that corresponds to a message of the reference watermark when one of the sum-of-products computations meets or exceeds a threshold.
Claim 143
Dependent Claim
228 words
The storage medium of claim 132 , wherein the method further comprises: obtaining a two-dimensional Fourier transform of the filtered array of reference message values; computing a two-dimensional Fourier transform of the filtered array of message values; computing a modified array by taking a product of the two-dimensional Fourier transform of the filtered array of reference message values and the two-dimensional Fourier transform of the filtered array of message values on a point-by-point basis; computing an inverse Fourier transform of the modified array; determining a maximum value from the inverse Fourier transform of the modified array; rotating the two-dimensional Fourier transform of the filtered array of reference message values relative to the two-dimensional Fourier transform of the filtered array of message values by a multiple of 90°; computing subsequent modified array by taking a product of the two-dimensional Fourier transform of the filtered array of reference message values and the two-dimensional Fourier transform of the filtered array of message values on a point-by-point basis; computing an inverse Fourier transform of the subsequent modified array; determining a subsequent maximum value from the inverse Fourier transform of the subsequent modified array; repeating the rotating, computing, and determining steps one or more times; and determining that the watermark contains a message that corresponds to a message of the reference watermark when one of the maximum values meets or exceeds a threshold.
References Cited (Backward Citations)
US4313984A
Cited
US5084790A
Cited
US5144658A
Cited
US6567533B1
Cited
US20030215112A1
Cited
US6047374A
Cited
EP0778566A2
Cited
WO1997026733A1
Cited
US6424725B1
Cited
US6381341B1
Cited
US6282299B1
Cited
US5809139A
Cited
US5915027A
Cited
US6680972B1
Cited
US6404926B1
Cited
US6108434A
Cited
US6556689B1
Cited
US6141441A
Cited
US6463162B1
Cited
US6442283B1
Cited
JP2001078010A
Cited
US6282300B1
Cited
US20010036292A1
Cited
US20020090107A1
Cited
US20030012402A1
Cited
US20030021439A1
Cited
US6563937B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2008-03-07 | 2008-09-06 | 2009-03-08 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2012-03-07 | 2012-09-06 | 2013-03-08 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2016-03-07 | 2016-09-06 | 2017-03-08 | $7700.00 | Window Closed / Lapsed |