Patent US6014669A successfully refreshed from Google Patents.
US6014669A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Highly-available distributed cluster configuration database
Assignee: Oracle America Inc
•
Filing: 1997-10-21
•
Granted: 2000-01-11
•
Expires: 2017-10-21
Abstract
Abstract The cluster configuration database is a distributed configuration database wherein a consistent copy of the configuration database is maintained on each active node of the cluster. Each node in the cluster maintains its own copy of the configuration database and configuration database operations can be performed from any node. Configuration database updates are automatically propagated to each node in a lock-step manner. If any node experiences a failure, the configuration database uses a reconfiguration protocol to insure consistent data in each node of the cluster. The cluster configuration database uses a two-level consistency update framework to insure consistent data among the nodes. Each local configuration database uses a self-contained consistency record to uniquely identify and stamp each copy of the configuration database. The consistency of each individual copy of the configuration database can be verified from the consistency record. Additionally, the cluster configuration database uses a two-phase commit protocol to guarantee the copies of the configuration database are consistent among the nodes. Cluster operations may be performed from any node in the cluster. Cluster operations include restore, checkpoint, update, query, and consistency checking. Because each node maintains a local copy of the configuration database, configuration database queries are local operations and can be performed in parallel among the nodes. Update operations are global operations that involve each active node of the cluster. The cluster configuration database uses a global locking mechanism that implements single write/multiple reader semantics.
Inventors
- 👤 Gregory L. Slaughter
- 👤 Bernard A. Traversat
- 👤 Robert J. Block
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-31 19:57
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/20; G06F16/27; G06F16/275; G06F2201/00; G06F2201/80; G06F2201/82; Y; Y10; Y10S; Y10S707/00; Y10S707/99931; Y10S707/99933; Y10S707/99937; Y10S707/99941; Y10S707/99942
Claims Analysis
Total Claims: 50
Claim 1
Independent Claim
199 words
A multi-node computer cluster comprising: a first node including a first storage device and a first cluster database server; a second node including a second storage device and a second cluster database server; and a communication interface coupled to said first node and said second node; wherein said first cluster database server and said second cluster database server store a local copy of a cluster database; wherein said cluster database includes an initialization database and a dynamic database, wherein said initialization database may be accessed before the cluster database is valid; wherein updates to said local copy of said cluster database stored in said first storage device are also made to said local copy of said cluster database stored in said second storage device, and a two-phase protocol ensures that first and second local copies of said databases are updated prior to confirming said update; and wherein when said multi-node computer cluster is restarted said local copy of said cluster database stored in said first storage device is compared to said local copy of said cluster database stored in second storage device and a most up-to-date copy of said cluster database is stored to said first and second nodes.
Claim 2
Dependent Claim
199 words
A multi-node computer cluster comprising: a first node including a first storage device and a first cluster database server; a second node including a second storage device and a second cluster database server; and a communication interface coupled to said first node and said second node; wherein said first cluster database server and said second cluster database server store a local copy of a cluster database; wherein said cluster database includes an initialization database and a dynamic database, wherein said initialization database may be accessed before the cluster database is valid; wherein updates to said local copy of said cluster database stored in said first storage device are also made to said local copy of said cluster database stored in said second storage device, and a two-phase protocol ensures that first and second local copies of said databases are updated prior to confirming said update; and wherein when said multi-node computer cluster is restarted said local copy of said cluster database stored in said first storage device is compared to said local copy of said cluster database stored in second storage device and a most up-to-date copy of said cluster database is stored to said first and second nodes.
Claim 3
Dependent Claim
33 words
The multi-node computer cluster of claim 1 wherein queries to said cluster database by said first node are made to said local copy of said cluster database stored in said first storage device.
Claim 4
Dependent Claim
43 words
The multi-node computer cluster of claim 2 wherein said two-phase protocol freezes said first and second nodes, updates said local copies of said cluster database, verifies the integrity of each local copy of said cluster database, and unfreezes said first and second nodes.
Claim 5
Dependent Claim
31 words
The multi-node computer cluster of claim 3 wherein said first and second nodes store a shadow copy of said cluster database prior to updating said local copy of said cluster database.
Claim 6
Dependent Claim
34 words
The multi-node computer cluster of claim 3 wherein steps of said two-phase commit protocol are run on all active nodes in a step-wise manner when a node joins or leaves said multi-node computer cluster.
Claim 7
Dependent Claim
31 words
The multi-node computer cluster of claim 4 wherein said first and second nodes rollback said shadow copies if the integrity of said local copies of said cluster database are not verified.
Claim 8
Dependent Claim
30 words
The multi-node computer cluster of claim 6 wherein said first and second nodes rollback said shadow copies if said updating of said local copy of said cluster database is unsuccessful.
Claim 9
Dependent Claim
30 words
The multi-node computer cluster of claim 7 wherein when a third node joins said cluster, said first node copies said local copy of said cluster database to said third node.
Claim 10
Dependent Claim
36 words
The multi-node computer cluster of claim 8 wherein said first cluster database server and said second cluster database server output a synchronization command when said two-phase protocol freezes or unfreezes said first and said second clusters.
Claim 11
Dependent Claim
28 words
The multi-node computer cluster of claim 8 wherein a cluster database server of either said first, second or third nodes can be designated as a master database server.
Claim 12
Dependent Claim
26 words
The multi-node computer cluster of claim 9 wherein a restore operation stores a consistent copy of said cluster database to said first and said second nodes.
Claim 13
Dependent Claim
29 words
The multi-node computer cluster of claim 11 wherein said first cluster database server and said second cluster database server output a synchronization command before or after a restore operation.
Claim 14
Dependent Claim
42 words
The multi-node computer cluster of claim 1 wherein each of said local copies of said cluster database includes a consistency record that stores a data value capable of verifying the integrity of said copy of said local copy of said cluster database.
Claim 15
Dependent Claim
36 words
The multi-node computer cluster of claim 13, wherein said consistency record includes a checksum of said local copy of said cluster database, the length of said local copy of said cluster database, and a time stamp.
Claim 16
Dependent Claim
41 words
The multi-node computer cluster of claim 14 wherein said comparing of said local copies of said cluster database comprises verifying the integrity of each local copy using said consistency record and comparing valid consistency records to determine said most up-to-date copy.
Claim 17
Dependent Claim
27 words
The multi-node computer cluster of claim 15 wherein said cluster database is only updated when a quorum majority of nodes of said multi-node computer cluster are active.
Claim 18
Dependent Claim
36 words
The multi-node computer cluster of claim 16 wherein said most up-to-date copy of said cluster database is determined when the lower bound of half or more of said nodes of said multi-node computer cluster are active.
Claim 19
Independent Claim
104 words
A method of storing and maintaining a consistent database in a cluster with multiple nodes comprising: storing a first copy of said database on a first node; storing a second copy of said database on a second node; wherein updates to said database are made to both said first copy and said second copy of said database, and a two-phase protocol ensures that said first copy and said second copy of said database are updated prior to confirming said update; and wherein said database includes an initialization database and a dynamic database, wherein said initialization database may be accessed before the database is valid.
Claim 20
Independent Claim
104 words
A method of storing and maintaining a consistent database in a cluster with multiple nodes comprising: storing a first copy of said database on a first node; storing a second copy of said database on a second node; wherein updates to said database are made to both said first copy and said second copy of said database, and a two-phase protocol ensures that said first copy and said second copy of said database are updated prior to confirming said update; and wherein said database includes an initialization database and a dynamic database, wherein said initialization database may be accessed before the database is valid.
Claim 21
Dependent Claim
42 words
The method of claim 18 wherein when said cluster is restarted, said first copy of said database is compared to said second copy of said database and a most up-to-date copy of said database is stored to said first and second nodes.
Claim 22
Dependent Claim
23 words
The method of claim 19 wherein queries to said database by said first node are made to said first copy of said database.
Claim 23
Dependent Claim
41 words
The method of claim 20 wherein said two-phase protocol freezes said first and second cluster, updates said first and second copies of said databases, verifies the integrity of each copy of said cluster database, and unfreezes said first and second nodes.
Claim 24
Dependent Claim
23 words
The method of claim 21 wherein said first and second nodes store a shadow copy of said database prior to updating said database.
Claim 25
Dependent Claim
30 words
The method of claim 21 wherein steps of said two-phase commit protocol are run on all active nodes in a step-wise manner when a node joins or leaves said cluster.
Claim 26
Dependent Claim
27 words
The method of claim 22 wherein said first and second nodes rollback said shadow copies if the integrity of said copies of said database are not verified.
Claim 27
Dependent Claim
23 words
The method of claim 24 wherein when a third node joins said cluster, said first copy of said database to said third node.
Claim 28
Dependent Claim
30 words
The method of claim 25 wherein said first node and said second node execute a synchronization command when said two-phase protocol freezes or unfreezes said first and said second clusters.
Claim 29
Dependent Claim
23 words
The method of claim 26 wherein a restore operation stores a consistent copy of said database to said first and said second nodes.
Claim 30
Dependent Claim
21 words
The method of claim 27 wherein said first node and said second node output a synchronization command during a restore operation.
Claim 31
Dependent Claim
20 words
The method of claim 25 wherein either said first, second or third nodes can be designated as a master server.
Claim 32
Dependent Claim
35 words
The method of claim 18 wherein each of said copies of said database include a consistency record that stores a data value capable of verifying the integrity of said local copy of said cluster database.
Claim 33
Dependent Claim
34 words
The method of claim 30 wherein said consistency record includes a checksum of said local copy of said cluster database, the length of said local copy of said cluster database, and a time stamp.
Claim 34
Dependent Claim
36 words
The method of claim 31 wherein said comparing of said copies of said database comprises verifying the integrity of each copy using said consistency record and comparing valid consistency records to determine said most up-to-date copy.
Claim 35
Dependent Claim
21 words
The method of claim 32 wherein said database is only update when a majority of nodes of said cluster are active.
Claim 36
Dependent Claim
31 words
The method of claim 33 wherein said most up-to-date copy of said cluster database is determined when lower bound of half or more of said nodes of said cluster are active.
Claim 37
Dependent Claim
292 words
A multi-node computer cluster comprising: a first node including a first storage device and a first cluster database server; a second node including a second storage device and a second cluster database server; and a communication interface coupled to said first node and said second node; wherein said first cluster database server and said second cluster database server store a local copy of a cluster database; wherein updates to said local copy of said cluster database stored in said first storage device are also made to said local copy of said cluster database stored in said second storage device, and a two-phase protocol ensures that first and second local copies of said databases are updated prior to confirming said update; wherein when said multi-node computer cluster is restarted said local copy of said cluster database stored in said first storage device is compared to said local copy of said cluster database stored in second storage device and a most up-to-date copy of said cluster database is stored to said first and second nodes; wherein each of said local copies of said cluster database includes a consistency record that stores a data value capable of verifying the integrity of said copy of said local copy of said cluster database; wherein said consistency record includes a checksum of said local copy of said cluster database, the length of said local copy of said cluster database, and a time stamp; wherein said comparing of said local copies of said cluster database comprises verifying the integrity of each local copy using said consistency record and comparing valid consistency records to determine said most up-to-date copy; and wherein said cluster database is only updated when a quorum majority of nodes of said multi-node computer cluster are active.
Claim 38
Dependent Claim
292 words
A multi-node computer cluster comprising: a first node including a first storage device and a first cluster database server; a second node including a second storage device and a second cluster database server; and a communication interface coupled to said first node and said second node; wherein said first cluster database server and said second cluster database server store a local copy of a cluster database; wherein updates to said local copy of said cluster database stored in said first storage device are also made to said local copy of said cluster database stored in said second storage device, and a two-phase protocol ensures that first and second local copies of said databases are updated prior to confirming said update; wherein when said multi-node computer cluster is restarted said local copy of said cluster database stored in said first storage device is compared to said local copy of said cluster database stored in second storage device and a most up-to-date copy of said cluster database is stored to said first and second nodes; wherein each of said local copies of said cluster database includes a consistency record that stores a data value capable of verifying the integrity of said copy of said local copy of said cluster database; wherein said consistency record includes a checksum of said local copy of said cluster database, the length of said local copy of said cluster database, and a time stamp; wherein said comparing of said local copies of said cluster database comprises verifying the integrity of each local copy using said consistency record and comparing valid consistency records to determine said most up-to-date copy; and wherein said cluster database is only updated when a quorum majority of nodes of said multi-node computer cluster are active.
Claim 39
Dependent Claim
36 words
The multi-node computer cluster of claim 35, wherein said most up-to-date copy of said cluster database is determined when the lower bound of half or more of said nodes of said multi-node computer cluster are active.
Claim 40
Dependent Claim
263 words
A multi-node computer cluster comprising: a first node including a first storage device and a first cluster database server; a second node including a second storage device and a second cluster database server; and a communication interface coupled to said first node and said second node; wherein said first cluster database server and said second cluster database server store a local copy of a cluster database; wherein updates to said local copy of said cluster database stored in said first storage device are also made to said local copy of said cluster database stored in said second storage device, and a two-phase protocol ensures that first and second local copies of said databases are updated prior to confirming said update; wherein when said multi-node computer cluster is restarted said local copy of said cluster database stored in said first storage device is compared to said local copy of said cluster database stored in second storage device and a most up-to-date copy of said cluster database is stored to said first and second nodes; wherein queries to said cluster database by said first node are made to said local copy of said cluster database stored in said first storage device; wherein said two-phase protocol freezes said first and second nodes, updates said local copies of said cluster database, verifies the integrity of each local copy of said cluster database, and unfreezes said first and second nodes; and wherein steps of said two-phase commit protocol are run on all active nodes in a step-wise manner when a node joins or leaves said multi-node computer cluster.
Claim 41
Dependent Claim
263 words
A multi-node computer cluster comprising: a first node including a first storage device and a first cluster database server; a second node including a second storage device and a second cluster database server; and a communication interface coupled to said first node and said second node; wherein said first cluster database server and said second cluster database server store a local copy of a cluster database; wherein updates to said local copy of said cluster database stored in said first storage device are also made to said local copy of said cluster database stored in said second storage device, and a two-phase protocol ensures that first and second local copies of said databases are updated prior to confirming said update; wherein when said multi-node computer cluster is restarted said local copy of said cluster database stored in said first storage device is compared to said local copy of said cluster database stored in second storage device and a most up-to-date copy of said cluster database is stored to said first and second nodes; wherein queries to said cluster database by said first node are made to said local copy of said cluster database stored in said first storage device; wherein said two-phase protocol freezes said first and second nodes, updates said local copies of said cluster database, verifies the integrity of each local copy of said cluster database, and unfreezes said first and second nodes; and wherein steps of said two-phase commit protocol are run on all active nodes in a step-wise manner when a node joins or leaves said multi-node computer cluster.
Claim 42
Dependent Claim
351 words
A multi-node computer cluster comprising: a first node including a first storage device and a first cluster database server; a second node including a second storage device and a second cluster database server; and a communication interface coupled to said first node and said second node; wherein said first cluster database server and said second cluster database server store a local copy of a cluster database; wherein updates to said local copy of said cluster database stored in said first storage device are also made to said local copy of said cluster database stored in said second storage device, and a two-phase protocol ensures that first and second local copies of said databases are updated prior to confirming said update; wherein when said multi-node computer cluster is restarted said local copy of said cluster database stored in said first storage device is compared to said local copy of said cluster database stored in second storage device and a most up-to-date copy of said cluster database is stored to said first and second nodes; wherein queries to said cluster database by said first node are made to said local copy of said cluster database stored in said first storage device; wherein said two-phase protocol freezes said first and second nodes, updates said local copies of said cluster database, verifies the integrity of each local copy of said cluster database, and unfreezes said first and second nodes; wherein said first and second nodes store a shadow copy of said cluster database prior to updating said local copy of said cluster database; wherein said first and second nodes rollback said shadow copies if the integrity of said local copies of said cluster database are not verified; wherein said first and second nodes rollback said shadow copies if said updating of said local copy of said cluster database is unsuccessful; wherein when a third node joins said cluster, said first node copies said local copy of said cluster database to said third node; and wherein a cluster database server of either said first, second or third nodes can be designated as a master database server.
Claim 43
Dependent Claim
351 words
A multi-node computer cluster comprising: a first node including a first storage device and a first cluster database server; a second node including a second storage device and a second cluster database server; and a communication interface coupled to said first node and said second node; wherein said first cluster database server and said second cluster database server store a local copy of a cluster database; wherein updates to said local copy of said cluster database stored in said first storage device are also made to said local copy of said cluster database stored in said second storage device, and a two-phase protocol ensures that first and second local copies of said databases are updated prior to confirming said update; wherein when said multi-node computer cluster is restarted said local copy of said cluster database stored in said first storage device is compared to said local copy of said cluster database stored in second storage device and a most up-to-date copy of said cluster database is stored to said first and second nodes; wherein queries to said cluster database by said first node are made to said local copy of said cluster database stored in said first storage device; wherein said two-phase protocol freezes said first and second nodes, updates said local copies of said cluster database, verifies the integrity of each local copy of said cluster database, and unfreezes said first and second nodes; wherein said first and second nodes store a shadow copy of said cluster database prior to updating said local copy of said cluster database; wherein said first and second nodes rollback said shadow copies if the integrity of said local copies of said cluster database are not verified; wherein said first and second nodes rollback said shadow copies if said updating of said local copy of said cluster database is unsuccessful; wherein when a third node joins said cluster, said first node copies said local copy of said cluster database to said third node; and wherein a cluster database server of either said first, second or third nodes can be designated as a master database server.
Claim 44
Independent Claim
187 words
A method of storing and maintaining a consistent database in a cluster with multiple nodes comprising: storing a first copy of said database on a first node; storing a second copy of said database on a second node; wherein updates to said database are made to both said first copy and said second copy of said database, and a two-phase protocol ensures that said first copy and said second copy of said database are updated prior to confirming said update; wherein each of said copies of said database include a consistency record that stores a data value capable of verifying the integrity of said local copy of said cluster database; wherein said consistency record includes a checksum of said local copy of said cluster database, the length of said local copy of said cluster database, and a time stamp; wherein said comparing of said copies of said database comprises verifying the integrity of each copy using said consistency record and comparing valid consistency records to determine said most up-to-date copy; and wherein said database is only updated when a majority of nodes of said cluster are active.
Claim 45
Independent Claim
187 words
A method of storing and maintaining a consistent database in a cluster with multiple nodes comprising: storing a first copy of said database on a first node; storing a second copy of said database on a second node; wherein updates to said database are made to both said first copy and said second copy of said database, and a two-phase protocol ensures that said first copy and said second copy of said database are updated prior to confirming said update; wherein each of said copies of said database include a consistency record that stores a data value capable of verifying the integrity of said local copy of said cluster database; wherein said consistency record includes a checksum of said local copy of said cluster database, the length of said local copy of said cluster database, and a time stamp; wherein said comparing of said copies of said database comprises verifying the integrity of each copy using said consistency record and comparing valid consistency records to determine said most up-to-date copy; and wherein said database is only updated when a majority of nodes of said cluster are active.
Claim 46
Dependent Claim
31 words
The method of claim 39, wherein said most up-to-date copy of said cluster database is determined when lower bound of half or more of said nodes of said cluster are active.
Claim 47
Independent Claim
197 words
A method of storing and maintaining a consistent database in a cluster with multiple nodes comprising: storing a first copy of said database on a first node; storing a second copy of said database on a second node; wherein updates to said database are made to both said first copy and said second copy of said database, and a two-phase protocol ensures that said first copy and said second copy of said database are updated prior to confirming said update; wherein when said cluster is restarted, said first copy of said database is compared to said second copy of said database and a most up-to-date copy of said database is stored to said first and second nodes; wherein queries to said database by said first node are made to said first copy of said database; wherein said two-phase protocol freezes said first and second cluster, updates said first and second copies of said databases, verifies the integrity of each copy of said cluster database, and unfreezes said first and second nodes; and wherein steps of said two-phase commit protocol are run on all active nodes in a step-wise manner when a node joins or leaves said cluster.
Claim 48
Independent Claim
197 words
A method of storing and maintaining a consistent database in a cluster with multiple nodes comprising: storing a first copy of said database on a first node; storing a second copy of said database on a second node; wherein updates to said database are made to both said first copy and said second copy of said database, and a two-phase protocol ensures that said first copy and said second copy of said database are updated prior to confirming said update; wherein when said cluster is restarted, said first copy of said database is compared to said second copy of said database and a most up-to-date copy of said database is stored to said first and second nodes; wherein queries to said database by said first node are made to said first copy of said database; wherein said two-phase protocol freezes said first and second cluster, updates said first and second copies of said databases, verifies the integrity of each copy of said cluster database, and unfreezes said first and second nodes; and wherein steps of said two-phase commit protocol are run on all active nodes in a step-wise manner when a node joins or leaves said cluster.
Claim 49
Independent Claim
245 words
A method of storing and maintaining a consistent database in a cluster with multiple nodes comprising: storing a first copy of said database on a first node; storing a second copy of said database on a second node; wherein updates to said database are made to both said first copy and said second copy of said database, and a two-phase protocol ensures that said first copy and said second copy of said database are updated prior to confirming said update; wherein when said cluster is restarted, said first copy of said database is compared to said second copy of said database and a most up-to-date copy of said database is stored to said first and second nodes; wherein queries to said database by said first node are made to said first copy of said database; wherein said two-phase protocol freezes said first and second cluster, updates said first and second copies of said databases, verifies the integrity of each copy of said cluster database, and unfreezes said first and second nodes; wherein said first and second nodes store a shadow copy of said database prior to updating said database; wherein said first and second nodes rollback said shadow copies if the integrity of said copies of said database are not verified; wherein when a third node joins said cluster, said first copy of said database to said third node; and wherein either said first, second or third nodes can be designated as a master server.
Claim 50
Independent Claim
245 words
A method of storing and maintaining a consistent database in a cluster with multiple nodes comprising: storing a first copy of said database on a first node; storing a second copy of said database on a second node; wherein updates to said database are made to both said first copy and said second copy of said database, and a two-phase protocol ensures that said first copy and said second copy of said database are updated prior to confirming said update; wherein when said cluster is restarted, said first copy of said database is compared to said second copy of said database and a most up-to-date copy of said database is stored to said first and second nodes; wherein queries to said database by said first node are made to said first copy of said database; wherein said two-phase protocol freezes said first and second cluster, updates said first and second copies of said databases, verifies the integrity of each copy of said cluster database, and unfreezes said first and second nodes; wherein said first and second nodes store a shadow copy of said database prior to updating said database; wherein said first and second nodes rollback said shadow copies if the integrity of said copies of said database are not verified; wherein when a third node joins said cluster, said first copy of said database to said third node; and wherein either said first, second or third nodes can be designated as a master server.
References Cited (Backward Citations)
US5257366A
Cited
EP0458564A2
Cited
US5371886A
Cited
GB2256069A
Cited
EP0578406A1
Cited
US5561797A
Cited
US5732071A
Cited
US5809296A
Cited
US5715250A
Cited
US5832222A
Cited
US5845292A
Cited
US5870763A
Cited
US5845279A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-01-10 | 2003-07-12 | 2004-01-11 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-01-10 | 2007-07-12 | 2008-01-11 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-01-10 | 2011-07-12 | 2012-01-11 | $7700.00 | Window Closed / Lapsed |