Ensure that we don't wait forever for upgrading an inbound connection
to occur.
Note that transports should return an AbortableSource when passed an
AbortSignal so outbound connections to not need the same fix.
Also adds default timeouts for the ping, fetch, and identify protocols.
Instead of trying to connect to every peer in the peer store when
we start a node, only connect to the peers that have been marked
with a `keep-alive` tag.
Uses peer tag values to select low-value connections to prune when we have too many connections open.
BREAKING CHANGE: `connectionManager.peerValue` has been removed, use `peerStore.tagPeer` instead
Converts this module to typescript.
- Ecosystem modules renamed from (e.g.) `libp2p-tcp` to `@libp2p/tcp`
- Ecosystem module now have named exports
- Configuration has been updated, now pass instances of modules instead of classes:
- Some configuration keys have been renamed to make them more descriptive. `transport` -> `transports`, `connEncryption` -> `connectionEncryption`. In general where we pass multiple things, the key is now plural, e.g. `streamMuxer` -> `streamMuxers`, `contentRouting` -> `contentRouters`, etc. Where we are configuring a singleton the config key is singular, e.g. `connProtector` -> `connectionProtector` etc.
- Properties of the `modules` config key have been moved to the root
- Properties of the `config` config key have been moved to the root
```js
// before
import Libp2p from 'libp2p'
import TCP from 'libp2p-tcp'
await Libp2p.create({
modules: {
transport: [
TCP
],
}
config: {
transport: {
[TCP.tag]: {
foo: 'bar'
}
},
relay: {
enabled: true,
hop: {
enabled: true,
active: true
}
}
}
})
```
```js
// after
import { createLibp2p } from 'libp2p'
import { TCP } from '@libp2p/tcp'
await createLibp2p({
transports: [
new TCP({ foo: 'bar' })
],
relay: {
enabled: true,
hop: {
enabled: true,
active: true
}
}
})
```
- Use of `enabled` flag has been reduced - previously you could pass a module but disable it with config. Now if you don't want a feature, just don't pass an implementation. Eg:
```js
// before
await Libp2p.create({
modules: {
transport: [
TCP
],
pubsub: Gossipsub
},
config: {
pubsub: {
enabled: false
}
}
})
```
```js
// after
await createLibp2p({
transports: [
new TCP()
]
})
```
- `.multiaddrs` renamed to `.getMultiaddrs()` because it's not a property accessor, work is done by that method to calculate announce addresses, observed addresses, etc
- `/p2p/${peerId}` is now appended to all addresses returned by `.getMultiaddrs()` so they can be used opaquely (every consumer has to append the peer ID to the address to actually use it otherwise). If you need low-level unadulterated addresses, call methods on the address manager.
BREAKING CHANGE: types are no longer hand crafted, this module is now ESM only
We have a peerstore that keeps all data for all observed peers in memory with no eviction.
This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.
We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.
It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.
This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives. It can be configured with a MemoryDatastore if desired.
It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.
BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
* fix: not dial all known peers on startup
* feat: connection manager should proactively connect to peers from peerStore
* chore: increase bundle size
* fix: do connMgr proactive dial on an interval
* chore: address review
* chore: use retimer reschedule
* chore: address review
* fix: use minConnections in default config
* chore: minPeers to minConnections everywhere
* refactor: initial refactor of the connection manager
* fix: start/stop issues
* fix: add tests and resolve pruning issues
* chore: fix lint
* test: move conn manager tests to node only for now
* chore: apply suggestions from code review
Co-Authored-By: Vasco Santos <vasco.santos@moxy.studio>
* fix: assert min max connection options
* test: fix assertion check for browser
* docs: add api and config docs for conn manager
* test: remove all tests for a clean slate
The refactor will require a large number of updates to the tests. In order
to ensure we have done a decent deduplication, and have a cleaner suite of tests
we've removed all tests. This will also allow us to more easily see tests
for the refactored systems.
We have a record of the latest test suites in master, so we are not losing any history.
* chore: update tcp and websockets
* chore: remove other transports until they are converted
* chore: use mafmt and multiaddr async versions
* chore: add and fix dependencies
* chore: clean up travis file
* feat: add new transport manager
* docs: add constructor jsdocs
* refactor(config): check that transports exist
This also removes the other logic, it can be added when those subsystems are refactored
* chore(deps): use async peer-id and peer-info
* feat: wire up the transport manager with libp2p
* chore: remove superstruct dep
* refactor: add js-libp2p-connection-manager to repo
Co-authored-by: David Dias <daviddias.p@gmail.com>
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
Co-authored-by: Pedro Teixeira <i@pgte.me>
Co-authored-by: Vasco Santos <vasco.santos@ua.pt>
* test(conn-mgr): only run in node
* refactor: add js-libp2p-identify to repo
Co-authored-by: David Dias <daviddias.p@gmail.com>
Co-authored-by: Friedel Ziegelmayer <dignifiedquire@gmail.com>
Co-authored-by: Hugo Dias <hugomrdias@gmail.com>
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
Co-authored-by: Maciej Krüger <mkg20001@gmail.com>
Co-authored-by: Richard Littauer <richard.littauer@gmail.com>
Co-authored-by: Vasco Santos <vasco.santos@moxy.studio>
Co-authored-by: Yusef Napora <yusef@protocol.ai>
Co-authored-by: ᴠɪᴄᴛᴏʀ ʙᴊᴇʟᴋʜᴏʟᴍ <victorbjelkholm@gmail.com>
* refactor: add libp2p-pnet to repo
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
Co-authored-by: Vasco Santos <vasco.santos@moxy.studio>
* refactor: add libp2p-ping to repo
Co-authored-by: David Dias <daviddias.p@gmail.com>
Co-authored-by: Francisco Baio Dias <xicombd@gmail.com>
Co-authored-by: Friedel Ziegelmayer <dignifiedquire@gmail.com>
Co-authored-by: Hugo Dias <mail@hugodias.me>
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
Co-authored-by: João Antunes <j.goncalo.antunes@gmail.com>
Co-authored-by: Richard Littauer <richard.littauer@gmail.com>
Co-authored-by: Vasco Santos <vasco.santos@moxy.studio>
Co-authored-by: Vasco Santos <vasco.santos@ua.pt>
Co-authored-by: ᴠɪᴄᴛᴏʀ ʙᴊᴇʟᴋʜᴏʟᴍ <victorbjelkholm@gmail.com>
* refactor: add libp2p-circuit to repo
Co-authored-by: David Dias <daviddias.p@gmail.com>
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Friedel Ziegelmayer <dignifiedquire@gmail.com>
Co-authored-by: Hugo Dias <mail@hugodias.me>
Co-authored-by: Jacob Heun <jacobheun@gmail.com>
Co-authored-by: Maciej Krüger <mkg20001@gmail.com>
Co-authored-by: Oli Evans <oli@tableflip.io>
Co-authored-by: Pedro Teixeira <i@pgte.me>
Co-authored-by: Vasco Santos <vasco.santos@ua.pt>
Co-authored-by: Victor Bjelkholm <victorbjelkholm@gmail.com>
Co-authored-by: Yusef Napora <yusef@napora.org>
Co-authored-by: dirkmc <dirk@mccormick.cx>
* test(switch): avoid using instanceof
* chore(switch): update bignumber dep
* refactor(circuit): clean up tests
* refactor(switch): consolidate get peer utils
* test(identify): do deep checks of addresses
* test(identify): bump timeout for identify test
* test(switch): tidy up limit dialer test
* refactor(switch): remove redundant circuit tests
* chore: add coverage script
* refactor(circuit): consolidate get peer info
* docs: reference original repositories in each sub readme
* docs: fix comment
* refactor: clean up sub package.json files and readmes
