We have a peerstore that keeps all data for all observed peers in memory with no eviction.
This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.
We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.
It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.
This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives. It can be configured with a MemoryDatastore if desired.
It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.
BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
BREAKING CHANGES:
top level types were updated, multiaddr@9.0.0 is used, dialer and keychain internal property names changed and connectionManager minPeers is not supported anymore
* fix: catch pipe errors
There were some pipe errors not being caught. This can result in unhandled exceptions being thrown
* fix: catch pipe errors in identify push handler
* feat: address and proto books
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
* chore: minor fixes and initial tests added
* chore: integrate new peer-store with code using adapters for other modules
* chore: do not use peerstore.put on get-peer-info
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
* chore: add new peer store tests
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
* refactor: examples-browser
* chore: add information to use signalling server
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
* chore: update deps
* docs: refactor libp2p browser example
* docs(examples): add back websockets and boostrap nodes
docs(examples): redo the browser readme
* fix: handle edge case of connections closing early
* chore: fix lint
* chore: update example deps and readme
* chore: update webrtc-star
* chore: apply suggestions from code review
Co-Authored-By: Alan Shaw <alan.shaw@protocol.ai>
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
Co-authored-by: Alan Shaw <alan.shaw@protocol.ai>
* fix: upgrader should not need muxers
* chore: address review
* chore: apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
* docs: fix spelling in api
* fix: dont create peerstore twice
* feat: add support for dial coalescing
* doc(fix): add setPeerValue to API TOC
* docs: add more jsdocs to dialer
* chore: remove old comment
* fix: ensure connections are closed
* fix: registrar.getConnections returns first open conn
* fix: directly set the closed status
* chore: remove unneeded log
* refactor: peerStore.put takes an options object
* docs: add initial notes on stats
* feat: initial refactor of stats to metrics
* feat: add support for placeholder metrics
This is helpful for tracking metrics prior to knowing the remote peers id
* fix: add metrics tests and fix issues
* fix: always clear the dial timeout timer
* docs: add metrics to api doc
* chore: apply suggestions from code review
Co-Authored-By: Vasco Santos <vasco.santos@moxy.studio>
* docs: update metrics docs
* fix: call metrics.onDisconnect
* docs(config): add example headers so they appear in the TOC
* docs(config): add metrics configuration
* docs(relay): fix relay configuration docs
* chore(deps): update connection and multistream
* feat: add basic dial support for addresses and peers
* test: automatically require all node test files
* fix: dont catch and log in the wrong place
* test: add direct spec test
fix: improve dial error consistency
* feat: add dial timeouts and concurrency
Queue timeouts will result in aborts of the dials
* chore: fix linting
* test: verify dialer defaults
* feat: add initial upgrader
* fix: add more test coverage and fix bugs
* feat: libp2p creates the upgrader
* feat: hook up handle to the upgrader
* feat: hook up the dialer to libp2p
test: add node dialer libp2p tests
* feat: add connection listeners to upgrader
* feat: emit connect and disconnect events
* chore: use libp2p-interfaces
* fix: address review feedback
* fix: correct import
* refactor: dedupe connection creation code
