We have a peerstore that keeps all data for all observed peers in memory with no eviction.
This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.
We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.
It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.
This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives. It can be configured with a MemoryDatastore if desired.
It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.
BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
`interface-datastore` now only contains the interface definition,
`datastore-core` has the various implementations.
BREAKING CHANGE: datastore implementations provided to libp2p must be compliant with interface-datastore@6.0.0
BREAKING CHANGES:
top level types were updated, multiaddr@9.0.0 is used, dialer and keychain internal property names changed and connectionManager minPeers is not supported anymore
* chore: use ed25519 keys in tests
* fix: persisted keybook recheck keybook content for delete
* chore: only store if key not inline
* chore: update peer id
* chore: identify wait for closed streams
* fix: replace node buffers with uint8arrays
Upgrades all deps and replaces all use of node Buffers with Uint8Arrays
BREAKING CHANGES:
- All deps used by this module now use Uint8Arrays in place of node Buffers
* chore: browser fixes
* chore: remove .only
* chore: stringify uint8array before parsing
* chore: update interop suite
* chore: remove ts from build command
* chore: update deps
* fix: update records to use uint8array
* chore: fix lint
* chore: update deps
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
* chore: update interface datastore
Updates to v1.x.x to not have multiple versions of this module in the ipfs browser bundle.
* fix: let batch commits complete before continuing tests
Batch commits are async but the tests weren't waiting for them to complete,
mainly because they are triggered by events.
There's no way that I can see of waiting for the batch commit to finish so
I've added delays to the tests. Not great but a start.
* chore: use error log
* test: wait for commit spies to complete
* chore: bump interface-datastore
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
BREAKING CHANGE: all API methods with peer-info parameters or return values were changed. You can check the API.md document, in order to check the new values to use
* chore: deprecate old peer-store api
BREAKING CHANGE: the peer-store api changed. Check the API docs for the new specification.
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
* feat: address and proto books
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
* chore: minor fixes and initial tests added
* chore: integrate new peer-store with code using adapters for other modules
* chore: do not use peerstore.put on get-peer-info
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
* chore: add new peer store tests
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
* fix: include libp2p in the options passed to discovery creation
* fix: handle multiple peer addresses in get multiaddrs for peers
* test(peer-store): add test to verify returned relay multiaddrs
* docs: fix spelling in api
* fix: dont create peerstore twice
* feat: add support for dial coalescing
* doc(fix): add setPeerValue to API TOC
* docs: add more jsdocs to dialer
* chore: remove old comment
* fix: ensure connections are closed
* fix: registrar.getConnections returns first open conn
* fix: directly set the closed status
* chore: remove unneeded log
* refactor: peerStore.put takes an options object
