* fix: add successful stream peer to protobook
If a protocol stream has been successfully negotiated and is to be
psased to the application, the peerstore should ensure that the peer is
registerd with that protocol.
* fix: upgrader test fix
Updates all deps needed to support passing lists of byte arrays where they have been created from multiple input buffers.
When reading multiplexed data, all messages arrive in length-prefixed buffers, which means the first few bytes tell the consumer how many bytes long next chunk will be.
One length prefixed chunk can be delivered in several payloads from the underlying network transport. The first payload can also include the length prefix and some or all of the data, so we stitch these together in a `Uint8ArrayList` to avoid having to concatenate `Uint8Array`s together.
Previously once we'd received enough bytes to satisfy the length prefix we'd concatenate the bytes together, but this is a potentially expensive operation where transports have small message sizes so instead just pass the `Uint8ArrayList` to the consumer and let them decide wether to concatenate or not as some consumers will be smart enough to operate on lists of `Uint8Array`s instead of always requiring a contiguous block of memory.
BREAKING CHANGE: Streams are now `Duplex<Uint8ArrayList, Uint8ArrayList | Uint8Array>`
- If a stream is opened that exceeds inbound/outbound limits, reset that stream (if it is incoming) or abort and throw (if it is outgoing)
- Make the error message more helpful (say which protocol has breached the limit)
- Increase the default stream limits so we don't trigger this by accident when a remote dials us with a protocol we don't support
Make sure we don't potentially wait forever during incoming circuit relay handshakes.
Adds a timeout option to the hop config to control how long we will wait.
If no `agentVersion` is provided for the Identify protocol, the default `AGENT_VERSION` will now be set to
* `js-libp2p/<libp2p.version> UserAgent=<process.version>` when running in Node.js
* `js-libp2p/<libp2p.version> UserAgent=<navigator.userAgent>` when running in the browser (also when running in a webworker)
Fixes#686
Supersedes #1240
Co-authored-by: Kevin Westphal <westphal@consider-it.de>
Co-authored-by: Kevin <56823591+6d7a@users.noreply.github.com>
Ensure that we don't wait forever for upgrading an inbound connection
to occur.
Note that transports should return an AbortableSource when passed an
AbortSignal so outbound connections to not need the same fix.
Also adds default timeouts for the ping, fetch, and identify protocols.
Instead of trying to connect to every peer in the peer store when
we start a node, only connect to the peers that have been marked
with a `keep-alive` tag.
Uses peer tag values to select low-value connections to prune when we have too many connections open.
BREAKING CHANGE: `connectionManager.peerValue` has been removed, use `peerStore.tagPeer` instead
* feat: limit protocol streams per-connection
Uses the `maxInboundStreams` and `maxOutboundStreams` of the `registrar.handle`
opts to limit the number of concurrent streams open on each connection
on a per-protocol basis.
Both values default to 1 so some tuning will be necessary to set
appropriate values for some protocols.
* chore: make error codes consistent
* chore: fix up examples
If an error is thrown during the initial stages of setting up a multiplexed stream, ensure we close the stream to free up any resources associated with it.
Adds a config option to specify a maximum message size we'll accept
for an Identify message.
The default is 8KB, the same as go-libp2p - previously we fell back
to the default `maxMessageLength` option of `it-length-prefixed`
which is 4MB.
Also adds a default timeout for reading responses to identify
requests which is used if an AbortSignal is not passed in.
The default timeout also aligns with go-libp2p.
There are a few places in the codebase where we send/receive data from the network without timeouts/abort controllers which means the user has to wait for the underlying socket to timeout which can take a long time depending on the platform, if at all.
This change ensures we can time out while running identify (both flavours), ping and fetch and adds tests to ensure there are no regressions.
**Motivation**
In lodestar, when we handle "peer:connect" event, we dial the peer which gives another "peer:connect" event and it causes other issues
**Motivation**
In `onConnect` function, "peer:connect" event should be emitted after we add connection to the `connections` map so that when app dial the peer in "peer:connect" event handler, it uses the same/existing connection
Instead of making the `.dht` and `.pubsub` properties optional, use dummy implementations that throw exceptions if they are not configured.
This way we don't have to null guard everywhere they are accessed.
Addresses PR comments from #1172 - fixes syntax of examples in docs, adds the transport manager to the exports map and renames fault tolerance enum for consistency.
Converts this module to typescript.
- Ecosystem modules renamed from (e.g.) `libp2p-tcp` to `@libp2p/tcp`
- Ecosystem module now have named exports
- Configuration has been updated, now pass instances of modules instead of classes:
- Some configuration keys have been renamed to make them more descriptive. `transport` -> `transports`, `connEncryption` -> `connectionEncryption`. In general where we pass multiple things, the key is now plural, e.g. `streamMuxer` -> `streamMuxers`, `contentRouting` -> `contentRouters`, etc. Where we are configuring a singleton the config key is singular, e.g. `connProtector` -> `connectionProtector` etc.
- Properties of the `modules` config key have been moved to the root
- Properties of the `config` config key have been moved to the root
```js
// before
import Libp2p from 'libp2p'
import TCP from 'libp2p-tcp'
await Libp2p.create({
modules: {
transport: [
TCP
],
}
config: {
transport: {
[TCP.tag]: {
foo: 'bar'
}
},
relay: {
enabled: true,
hop: {
enabled: true,
active: true
}
}
}
})
```
```js
// after
import { createLibp2p } from 'libp2p'
import { TCP } from '@libp2p/tcp'
await createLibp2p({
transports: [
new TCP({ foo: 'bar' })
],
relay: {
enabled: true,
hop: {
enabled: true,
active: true
}
}
})
```
- Use of `enabled` flag has been reduced - previously you could pass a module but disable it with config. Now if you don't want a feature, just don't pass an implementation. Eg:
```js
// before
await Libp2p.create({
modules: {
transport: [
TCP
],
pubsub: Gossipsub
},
config: {
pubsub: {
enabled: false
}
}
})
```
```js
// after
await createLibp2p({
transports: [
new TCP()
]
})
```
- `.multiaddrs` renamed to `.getMultiaddrs()` because it's not a property accessor, work is done by that method to calculate announce addresses, observed addresses, etc
- `/p2p/${peerId}` is now appended to all addresses returned by `.getMultiaddrs()` so they can be used opaquely (every consumer has to append the peer ID to the address to actually use it otherwise). If you need low-level unadulterated addresses, call methods on the address manager.
BREAKING CHANGE: types are no longer hand crafted, this module is now ESM only
When the node is shutting down, new connections can still be received.
If this happens we can end up writing into the datastore when it's
been closed which throws an error.
Instead, if we're not running, have the connection manager close new
incoming connections.
To allow us to shut down cleanly, we must wait the unhandling of protocols - this is because they write the new list of protocols into the datastore which might also be in the process of shutting down.
The order of the addresses can affect our success rate in dialing a
relay - if it's a loopback address or similar it won't work.
Instead try dialing every address.
We have a peerstore that keeps all data for all observed peers in memory with no eviction.
This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.
We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.
It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.
This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives. It can be configured with a MemoryDatastore if desired.
It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.
BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
The `AbortController` class is supported by browsers and node 14+ - we only support node 16+ (e.g. LTS+Current) so the `abort-controller` module isn't needed any more.
Implements the idea from #1060 - allows us to get some insight into what's happening in a libp2p node out side of just bandwidth stats.
Configures a few default metrics if metrics are enabled - current connections, the state of the dial queue, etc.
Also makes the `Metrics` class not depend on the `ConnectionManager` class, otherwise we can't collect simple metrics from the connection manager class due to the circular dependency.
Looks like this project stopped running the `test:node` npm script when it was migrated to gh actions.
Re-enable it and fix all the related test failures.
Do not abort all attempts to find peers when `findPeers` on one router throws synchronously
Co-authored-by: Robert Kiel <robert.kiel@hoprnet.io>
Co-authored-by: achingbrain <alex@achingbrain.net>
