Generate a Dstore instance for a specific [[Datastore]] instance.
const dstore = Dstore(new Datastore())
You are encouraged to provide the second parameter to provide the ProjectID. This makes [[keySerialize]] more robust. Usually you can get this from the Environment:
const dstore = Dstore(new Datastore(), process.env.GCLOUD_PROJECT)
@param datastore A [[Datastore]] instance. Can be freely accessed by the client. Be aware that using this inside [[runInTransaction]] ignores the transaction.
@param projectId The `GCLOUD_PROJECT ID`. Used for Key generation during serialization.
Optional projectId: stringOptional logger: stringReadonly datastoreOptional Readonly loggerOptional Readonly projectPrivate Readonly urlreadKey() extracts the [[Key]] from an [[IDstoreEntry]].
Is is an alternative to entity[Datastore.KEY] which tends to fail in various contexts and also confuses older Typescript compilers.
It can extract the [[Key]] form a [[IDstoreEntry]] which has been serialized to JSON by leveraging the property _keyStr.
set() is addition to [[Datastore]]. It provides a classic Key-value Interface.
Instead providing a nested [[DstoreSaveEntity]] to [[save]] you can call set directly as set( key, value).
Observe that set takes a [[Key]] as first parameter. you call it like this;
const ds = Dstore()
ds.set(ds.key('kind', '123'), {props1: 'foo', prop2: 'bar'})
It returns the [[Key]] of the Object being written. If the Key provided was an incomplete [[Key]] it will return a completed [[Key]]. See [[save]] for more information on key generation.
createQuery() creates an "empty" [[Query]] Object.
Compatible to createQuery in the datastore.
Name of the [[Datastore]]Kind ("Table") which should be searched.
delete() is compatible to [Datastore.delete()].
Unfortunately currently (late 2021) there is no formal documentation from Google on [Datastore.delete()].
The single Parameter is a list of [[Key]]s.
If called within a transaction it returns undefined.
If not, it returns a [[CommitResponse]] which is not documented by Google.
delete() is idempotent. Deleting the same [[Key]] twice is no error.
[[DstoreError]]
get() reads a [[IDstoreEntry]] from the Datastore.
It returns [[IDstoreEntry]] or null if not found.
The underlying Datastore API Call is lookup.
It is in the Spirit of [Datastore.get()]. Unfortunately currently (late 2021) there is no formal documentation from Google on [Datastore.get()].
Differences between [[Dstore.get]] and [[Datastore.get]]:
getMulti() reads several [[IDstoreEntry]]s from the Datastore.
It returns a list of [[IDstoreEntry]]s or undefined if not found. Entries are in the same Order as the keys in the Parameter.
This is different from the @google-cloud/datastore where not found items are not present in the result and the order in the result list is undefined.
The underlying Datastore API Call is lookup.
It is in the Spirit of [Datastore.get()]. Unfortunately currently (late 2021) there is no formal documentation from Google on [Datastore.get()].
Differences between [[Dstore.getMulti]] and [[Datastore.get]]:
insert() is compatible to Datastore.insert().
The single Parameter is a list of [[DstoreSaveEntity]]s.
If called within a transaction it returns undefined.
If not, it returns a [[CommitResponse]] which is not documented by Google.
insert() seems to be like [[save]] where [[DstoreSaveEntity.method]] is set to 'insert'. It throws an [[DstoreError]] if there is already a Entity with the same [[Key]] in the Datastore.
For handling of incomplete [[Key]]s see [[save]].
This function can be completely emulated by using [[save]] with method: 'insert' inside each [[DstoreSaveEntity]].
await ds.insert([{key: ds.key(['testKind', 123]), entity: {data:' 123'}}])
[[DstoreError]]
key() creates a [[Key]] Object from a path.
Compatible to Datastore.key
If the Path has an odd number of elements, it is considered an incomplete Key. This can only be used for saving and will prompt the Datastore to auto-generate an (random) ID. See also [[save]].
keyFromSerialized() deserializes a string created with [[keySerialize]] to a [[Key]].
Compatible to keyFromLegacyUrlsafe.
keyFromSerialized() serializes [[Key]] to a string.
Compatible to keyToLegacyUrlSafe, but does not support the "locationPrefix" since the use for this parameter is undocumented and unknown. It seems to be an artifact from early App Engine days.
It can be a synchronous function because it does not look up the projectId. Instead it is assumed, that you give the projectId upon instantiation of [[Dstore]]. It also seems, that a wrong projectId bears no ill effects.
query() combined [[createQuery]] and [[runQuery]] in a single call.
Name of the [[Datastore]]Kind ("Table") which should be searched.
List of [[Query]] filter() calls.
Maximum Number of Results to return.
List of [[Query]] order() calls.
selectionList of [[Query]] select() calls.
Optional cursor: stringunsupported so far.
save() is compatible to Datastore.save().
The single Parameter is a list of [[DstoreSaveEntity]]s.
If called within a transaction it returns undefined.
If not, it returns a [[CommitResponse]] which is not documented by Google.
Different [DstoreSaveEntity]]s in the entities parameter can have different values in their [[DstoreSaveEntity.method]] property.
This allows you to do insert, update and upsert (the default) in a single request.
save() seems to basically be an alias to upsert.
If the [[Key]] provided in [[DstoreSaveEntity.key]] was an incomplete [[Key]] it will be updated by save() inside the [[DstoreSaveEntity]].
If the Datastore generates a new ID because of an incomplete [[Key]] on first save it will return an large integer as [[Key.id]].
On every subsequent save() an string encoded number representation is returned.
Dstore should normalizes that and always return an string encoded number representation.
Each [[DstoreSaveEntity]] can have an excludeFromIndexes property which is somewhat underdocumented.
It can use something like JSON-Path notation
(Source)
.*,
parent[],
parent.*,
parent[].*
and more complex patterns
seem to be supported patterns.
If the caller has not provided an excludeLargeProperties in a [[DstoreSaveEntity]] we will default it
to excludeLargeProperties: true. Without this you can not store strings longer than 1500 bytes easily
(source).
update() is compatible to Datastore.update().
The single Parameter is a list of [[DstoreSaveEntity]]s.
If called within a transaction it returns undefined.
If not, it returns a [[CommitResponse]] which is not documented by Google.
update() seems to be like [[save]] where [[DstoreSaveEntity.method]] is set to 'update'.
It throws an [[DstoreError]] if there is no Entity with the same [[Key]] in the Datastore. update() overwrites all existing data for that [[Key]].
There was an alpha functionality called merge() in the Datastore which read an Entity, merged it with the new data and wrote it back, but this was never documented.
update() is idempotent. Updating the same [[Key]] twice is no error.
This function can be completely emulated by using [[save]] with method: 'update' inside each [[DstoreSaveEntity]].
[[DstoreError]]
Allocate one ID in the Datastore.
Currently (late 2021) there is no documentation provided by Google for the underlying node function. Check the Documentation for the low-level function and the conceptual overview and this Stackoverflow post.
The ID is a string encoded large number. This function will never return the same ID twice for any given Datastore. If you provide a kindName the ID will be namespaced to this kind. In fact the generated ID is namespaced via an incomplete [[Key]] of the given Kind.
Private fixInternal
fixKeys() is called for all [[IDstoreEntry]]sa returned from [[Dstore]].
Is ensures that besides entity[Datastore.KEY] there is _keyStr to be leveraged by [[readKey]].
Private getGets the Datastore or the current Transaction.
This tries to give high level access to transactions.
So called "Gross Group Transactions" are always enabled. Transactions are never Cross Project. runInTransaction() works only if you use the same [[KvStore] instance for all access within the Transaction.
[[runInTransaction]] is modelled after Python 2.7 ndb's @ndb.transactional feature. This is based on node's AsyncLocalStorage.
Transactions frequently fail if you try to access the same data via in a transaction. See the Documentation on Locking for further reference. You are advised to use p-limit(1) to serialize transactions touching the same resource. This should work nicely with node's single process model. It is a much bigger problem on shared-nothing approaches, like Python on App Engine.
Transactions might be wrapped in p-retry to implement automatically retrying them with exponential back-off should they fail due to contention.
Be aware that Transactions differ considerable between Master-Slave Datastore (very old), High Replication Datastore (old, later called Google Cloud Datastore) and Firestore in Datastore Mode (current).
Most Applications today are running on "Firestore in Datastore Mode". Beware that the Datastore-Emulator fails with error: 3 INVALID_ARGUMENT: Only ancestor queries are allowed inside transactions. during [[runQuery]] while the Datastore on Google Infrastructure does not have such an restriction anymore as of 2022.
Generated using TypeDoc
Dstore implements a slightly more accessible version of the Google Cloud Datastore: Node.js Client
@google-cloud/datastore is a strange beast: The documentation is auto generated and completely shy of documenting any advanced concepts. (Example: If you ask the datastore to auto-generate keys during save: how do you retrieve the generated key?) Generally I suggest to look at the Python 2.x db and ndb documentation to get a better explanation of the workings of the datastore.
Also the typings are strange. The Google provided type
Entitiescan be the on disk representation, the same but including a key reference (Datastore.KEY- [[IDstoreEntry]]), a list of these or a structured object containing the on disk representation under thedataproperty and akeyproperty and maybe some configuration likeexcludeFromIndexes([[DstoreSaveEntity]]) or a list of these.KvStore tries to abstract away most surprises the datastore provides to you but als tries to stay as API compatible as possible to @google-cloud/datastore.
Main differences:
(key, value)and always returns the complete [[Key]] of the entity being written. Keys are normalized, numeric IDs are always encoded as strings.Symbol-based access toentity[Datastore.KEY]. If needed, it tries to deserialize_keyStrto createentity[Datastore.KEY]. This ist important when rehydrating an [[IDstoreEntry]] from a serializing cache.@ndb.transactionalfeature. This is implemented via node's AsyncLocalStorage.This documentation also tries to document the little known idiosyncrasies of the @google-cloud/datastore library. See the corresponding functions.