54 Commits

Author SHA1 Message Date
Alex Potsides
750ed9c35f
fix: add timeout for incoming connections and build-in protocols (#1292)
Ensure that we don't wait forever for upgrading an inbound connection
to occur.

Note that transports should return an AbortableSource when passed an
AbortSignal so outbound connections to not need the same fix.

Also adds default timeouts for the ping, fetch, and identify protocols.
2022-07-14 13:35:12 +00:00
Alex Potsides
de30c2cec7
feat!: limit protocol streams per-connection (#1255)
* feat: limit protocol streams per-connection

Uses the `maxInboundStreams` and `maxOutboundStreams` of the `registrar.handle`
opts to limit the number of concurrent streams open on each connection
on a per-protocol basis.

Both values default to 1 so some tuning will be necessary to set
appropriate values for some protocols.

* chore: make error codes consistent

* chore: fix up examples
2022-06-17 15:46:31 +02:00
Alex Potsides
d4dd664071
feat!: update libp2p interfaces (#1252)
BREAKING CHANGE: uses new single-issue libp2p interface modules
2022-06-15 18:30:39 +01:00
Alex Potsides
824720fb8f
fix: reduce identify message size limit (#1230)
Adds a config option to specify a maximum message size we'll accept
for an Identify message.

The default is 8KB, the same as go-libp2p - previously we fell back
to the default `maxMessageLength` option of `it-length-prefixed`
which is 4MB.

Also adds a default timeout for reading responses to identify
requests which is used if an AbortSignal is not passed in.

The default timeout also aligns with go-libp2p.
2022-05-31 17:10:40 +01:00
Alex Potsides
a1220d22f5
fix: time out slow reads (#1227)
There are a few places in the codebase where we send/receive data from the network without timeouts/abort controllers which means the user has to wait for the underlying socket to timeout which can take a long time depending on the platform, if at all.

This change ensures we can time out while running identify (both flavours), ping and fetch and adds tests to ensure there are no regressions.
2022-05-25 18:15:21 +01:00
Alex Potsides
da3d19b309
fix: update interfaces (#1207)
Update to the latest interfaces version
2022-05-04 16:03:43 +01:00
Alex Potsides
a15254fdd4
fix: update to new interfaces (#1206)
Notably removes the `Dialer` interface as the `ConnectionManager` is now in charge of managing connections.
2022-05-04 10:19:04 +01:00
tuyennhv
d16817ca44
fix: emit peer:connect after all (#1171)
**Motivation**

In lodestar, when we handle "peer:connect" event, we dial the peer which gives another "peer:connect" event and it causes other issues

**Motivation**

In `onConnect` function, "peer:connect" event should be emitted after we add connection to the `connections` map so that when app dial the peer in "peer:connect" event handler, it uses the same/existing connection
2022-04-22 20:56:47 +01:00
Alex Potsides
fab4f1385c
fix: update pubsub interfaces (#1194)
Update to latest version of pubsub interface
2022-04-22 20:49:35 +01:00
Alex Potsides
c64a586a20
chore: update aegir to the latest version (#1186)
Removes boilerplate config that is no longer necessary
2022-04-09 09:26:25 +01:00
Alex Potsides
199395de4d
feat: convert to typescript (#1172)
Converts this module to typescript.

- Ecosystem modules renamed from (e.g.) `libp2p-tcp` to `@libp2p/tcp`
- Ecosystem module now have named exports
- Configuration has been updated, now pass instances of modules instead of classes:
- Some configuration keys have been renamed to make them more descriptive.  `transport` -> `transports`, `connEncryption` -> `connectionEncryption`.  In general where we pass multiple things, the key is now plural, e.g. `streamMuxer` -> `streamMuxers`, `contentRouting` -> `contentRouters`, etc.  Where we are configuring a singleton the config key is singular, e.g. `connProtector` -> `connectionProtector` etc.
- Properties of the `modules` config key have been moved to the root
- Properties of the `config` config key have been moved to the root
```js
// before
import Libp2p from 'libp2p'
import TCP from 'libp2p-tcp'

await Libp2p.create({
  modules: {
    transport: [
      TCP
    ],
  }
  config: {
    transport: {
      [TCP.tag]: {
        foo: 'bar'
      }
    },
    relay: {
      enabled: true,
      hop: {
        enabled: true,
        active: true
      }
    }
  }
})
```
```js
// after
import { createLibp2p } from 'libp2p'
import { TCP } from '@libp2p/tcp'

await createLibp2p({
  transports: [
    new TCP({ foo: 'bar' })
  ],
  relay: {
    enabled: true,
    hop: {
      enabled: true,
      active: true
    }
  }
})
```
- Use of `enabled` flag has been reduced - previously you could pass a module but disable it with config.  Now if you don't want a feature, just don't pass an implementation.   Eg:
```js
// before
await Libp2p.create({
  modules: {
    transport: [
      TCP
    ],
    pubsub: Gossipsub
  },
  config: {
    pubsub: {
      enabled: false
    }
  }
})
```
```js
// after
await createLibp2p({
  transports: [
    new TCP()
  ]
})
```
- `.multiaddrs` renamed to `.getMultiaddrs()` because it's not a property accessor, work is done by that method to calculate announce addresses, observed addresses, etc
- `/p2p/${peerId}` is now appended to all addresses returned by `.getMultiaddrs()` so they can be used opaquely (every consumer has to append the peer ID to the address to actually use it otherwise).  If you need low-level unadulterated addresses, call methods on the address manager.

BREAKING CHANGE: types are no longer hand crafted, this module is now ESM only
2022-03-28 14:30:27 +01:00
Alex Potsides
ff32eba6a0
feat: connection gater (#1142)
Port of https://github.com/libp2p/go-libp2p-core/blob/master/connmgr/gater.go

Adds a new configuration key `connectionGater` which allows denying the dialing of certain peers, individual multiaddrs and the creation of connections at certain points in the connection flow.

Fixes: https://github.com/libp2p/js-libp2p/issues/175
Refs: https://github.com/libp2p/js-libp2p/issues/744
Refs: https://github.com/libp2p/js-libp2p/issues/769

Co-authored-by: mzdws <8580712+mzdws@user.noreply.gitee.com>
2022-01-25 16:27:01 +00:00
Alex Potsides
4c3bf01f35
chore: fix flaky tests (#1137)
These tests are flaky in CI, probably due to differences in timing introduced by #1058

Fixes #1134
2022-01-21 08:44:13 +00:00
Alex Potsides
978eb3676f
feat: async peerstore backed by datastores (#1058)
We have a peerstore that keeps all data for all observed peers in memory with no eviction.

This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.

We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.

It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.

This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives.  It can be configured with a MemoryDatastore if desired.

It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.

BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
2022-01-20 12:03:35 +00:00
Vasco Santos
0f389a7828 chore: update uint8arrays 2021-08-20 09:13:21 +02:00
greenSnot
ef24fabf02
feat: custom protocol name (#962)
Co-authored-by: mzdws <8580712+mzdws@user.noreply.gitee.com>
2021-08-13 16:21:50 +02:00
Alex Potsides
818d2b2a98
fix: store remote agent and protocol version during identify (#943) 2021-05-12 19:11:17 +02:00
Vasco Santos
8506414ea1
chore: config types and dependencies update (#904)
BREAKING CHANGES:

top level types were updated, multiaddr@9.0.0 is used, dialer and keychain internal property names changed and connectionManager minPeers is not supported anymore
2021-04-15 09:40:02 +02:00
Alex Potsides
0a6bc0d101
feat: add UPnP NAT manager (#810)
* feat: add uPnP nat manager

Adds a really basic nat manager that attempts to use UPnP to punch
a hole through your router for any IPV4 tcp addresses you have
configured.

Adds any configured addresses to the node's observed addresses list
and adds observed addresses to `libp2p.multiaddrs` so we exchange
them with peers when performing `identify` and people can dial you.

Adds configuration options under `config.nat`

Hole punching is async to not affect start up time.

Co-authored-by: Vasco Santos <vasco.santos@moxy.studio>
2021-01-27 14:55:26 +01:00
Vasco Santos
169bb806a7 chore: add typedefs (#802) 2020-12-16 13:56:41 +01:00
Vasco Santos
ef9d3ca2c6 feat: custom announce filter 2020-12-16 13:56:41 +01:00
Vasco Santos
97e3633f47 chore: store self protocols in protobook (#760) 2020-12-16 13:56:41 +01:00
Vasco Santos
bb83cacb5a chore: address review 2020-12-16 13:56:41 +01:00
Vasco Santos
447d0ed0dd chore: add identify test for multiaddr change 2020-12-16 13:56:41 +01:00
Vasco Santos
43eda43f06 chore: create signed peer record on new listen addresses in transport manager 2020-12-16 13:56:41 +01:00
Vasco Santos
7b93ece7f2 chore: use listening events to create self peer record on updates 2020-12-16 13:56:41 +01:00
Vasco Santos
d0a9fada32
feat: custom and store self agent version + store self protocol version (#800)
* feat: custom and store self protocol and agent version

* fix: do not enable custom protocolVersion
2020-11-20 15:14:01 +01:00
Vasco Santos
60d437f595
fix: flakey identify test firefox (#774) 2020-10-06 15:37:01 +02:00
Alex Potsides
1e869717ff fix: replace node buffers with uint8arrays (#730)
* fix: replace node buffers with uint8arrays

Upgrades all deps and replaces all use of node Buffers with Uint8Arrays

BREAKING CHANGES:

- All deps used by this module now use Uint8Arrays in place of node Buffers

* chore: browser fixes

* chore: remove .only

* chore: stringify uint8array before parsing

* chore: update interop suite

* chore: remove ts from build command

* chore: update deps

* fix: update records to use uint8array

* chore: fix lint

* chore: update deps

Co-authored-by: Jacob Heun <jacobheun@gmail.com>
2020-08-27 15:38:01 +02:00
Vasco Santos
f574e82a5d chore: apply suggestions from code review 2020-08-27 15:38:01 +02:00
Vasco Santos
15613ccf19 fix: do not return self on peerstore.peers 2020-08-27 15:38:01 +02:00
Vasco Santos
74d414c21f chore: add certified peer records to persisted peer store 2020-08-27 15:38:01 +02:00
Vasco Santos
3158366579 fix: revert new identify protocol versions 2020-08-27 15:38:01 +02:00
Vasco Santos
1d318e12d8 chore: address review 2020-08-27 15:38:01 +02:00
Vasco Santos
8a97dded26 feat: create self peer record in identify 2020-08-27 15:38:01 +02:00
Vasco Santos
e50f0eeb7b feat: exchange signed peer records in identify 2020-08-27 15:38:01 +02:00
Jacob Heun
726a746479
fix(identify): make agentversion dynamic and add it to the peerstore (#724) 2020-08-04 18:39:05 +02:00
Vasco Santos
2b45fee0ed fix: onConnect should not add addr to the addressBook 2020-05-28 12:37:48 +02:00
Vasco Santos
65e87460b0 chore: add keys to keybook on connection upgraded 2020-05-28 12:37:48 +02:00
Vasco Santos
51474c334a chore: rename address functions and apply suggestions from code review
Co-Authored-By: Jacob Heun <jacobheun@gmail.com>
2020-05-28 12:37:48 +02:00
Vasco Santos
2a7967c1cc feat: address manager 2020-05-28 12:37:48 +02:00
Vasco Santos
d3a4bf0a3f chore: refactor connection manager and registrar 2020-05-28 12:37:48 +02:00
Vasco Santos
12e48adafa chore: remove peer-info usage
BREAKING CHANGE: all API methods with peer-info parameters or return values were changed. You can check the API.md document, in order to check the new values to use
2020-05-28 12:37:48 +02:00
Vasco Santos
e9d225c9dc feat: address and proto books (#590)
* feat: address and proto books

* chore: apply suggestions from code review

Co-Authored-By: Jacob Heun <jacobheun@gmail.com>

* chore: minor fixes and initial tests added

* chore: integrate new peer-store with code using adapters for other modules

* chore: do not use peerstore.put on get-peer-info

* chore: apply suggestions from code review

Co-Authored-By: Jacob Heun <jacobheun@gmail.com>

* chore: add new peer store tests

* chore: apply suggestions from code review

Co-Authored-By: Jacob Heun <jacobheun@gmail.com>

Co-authored-by: Jacob Heun <jacobheun@gmail.com>
2020-05-28 12:37:48 +02:00
Jacob Heun
f662fdcf36
fix: ensure identify streams are closed (#551)
* fix: ensure identify streams are closed

* fix: call connection.addStream properly

* chore: simplify stream closure

* test: improve durability of identify push test
2020-02-05 17:35:27 +01:00
Jacob Heun
c1038bec92 fix: use toB58String everywhere to be consistent (#537)
* chore: update deps

* fix: consistently use b58 peerid string

The migration to base32 will happen at a later date
2020-01-24 14:40:40 +01:00
Jacob Heun
15f7c2a974 feat: coalescing dial support (#518)
* docs: fix spelling in api

* fix: dont create peerstore twice

* feat: add support for dial coalescing

* doc(fix): add setPeerValue to API TOC

* docs: add more jsdocs to dialer

* chore: remove old comment

* fix: ensure connections are closed

* fix: registrar.getConnections returns first open conn

* fix: directly set the closed status

* chore: remove unneeded log

* refactor: peerStore.put takes an options object
2020-01-24 14:40:40 +01:00
Jacob Heun
e445a17278 feat: add token based dialer 2020-01-24 14:40:40 +01:00
Jacob Heun
12fc069873 fix: clean up peer discovery flow (#494)
* fix: clean up peer discovery flow

* test(fix): let libp2p start after connecting

* test(fix): dont auto dial in disco tests
2020-01-24 14:40:40 +01:00
Jacob Heun
f77ce39484 refactor: circuit relay to async (#477)
* refactor: add dialing over relay support

* chore: fix lint

* fix: dont clear listeners on close

* fix: if dial errors already have codes, just rethrow them

* fix: clear the registrar when libp2p stops

* fix: improve connection maintenance with circuit

* chore: correct feedback

* test: use chai as promised

* test(fix): reset multiaddrs on dial test
2020-01-24 14:40:40 +01:00