291 Commits

Author SHA1 Message Date
Alex Potsides
a1220d22f5
fix: time out slow reads (#1227)
There are a few places in the codebase where we send/receive data from the network without timeouts/abort controllers which means the user has to wait for the underlying socket to timeout which can take a long time depending on the platform, if at all.

This change ensures we can time out while running identify (both flavours), ping and fetch and adds tests to ensure there are no regressions.
2022-05-25 18:15:21 +01:00
Robert Kiel
35f9c0c793
fix: fix unintended aborts in dialer (#1185)
Fix a bug where `DialRequest` can abort wrong dial attempts.

Co-authored-by: Robert Kiel <robert.kiel@hoprnet.io>
2022-05-18 14:53:58 +01:00
Alex Potsides
d5386df684
fix: do upnp hole punch after startup (#1217)
The transport manager configures it's addresses during the `start`
phase so access them during `afterStart` so they'll be ready for use.
2022-05-18 13:19:31 +01:00
Alex Potsides
31480603f3
fix: simplify pnet exports (#1213)
Export the key generator from `libp2p/pnet`
2022-05-16 10:07:32 +01:00
Alex Potsides
4837430d8b
fix: encode enums correctly (#1210)
Updates protons and regenerates protobuf code to encode enums correctly
2022-05-10 12:35:33 +01:00
Alex Potsides
da3d19b309
fix: update interfaces (#1207)
Update to the latest interfaces version
2022-05-04 16:03:43 +01:00
Alex Potsides
a15254fdd4
fix: update to new interfaces (#1206)
Notably removes the `Dialer` interface as the `ConnectionManager` is now in charge of managing connections.
2022-05-04 10:19:04 +01:00
tuyennhv
d16817ca44
fix: emit peer:connect after all (#1171)
**Motivation**

In lodestar, when we handle "peer:connect" event, we dial the peer which gives another "peer:connect" event and it causes other issues

**Motivation**

In `onConnect` function, "peer:connect" event should be emitted after we add connection to the `connections` map so that when app dial the peer in "peer:connect" event handler, it uses the same/existing connection
2022-04-22 20:56:47 +01:00
Alex Potsides
fab4f1385c
fix: update pubsub interfaces (#1194)
Update to latest version of pubsub interface
2022-04-22 20:49:35 +01:00
Alex Potsides
5397137c65
fix: use placeholder dht/pubsub (#1193)
Instead of making the `.dht` and `.pubsub` properties optional, use dummy implementations that throw exceptions if they are not configured.

This way we don't have to null guard everywhere they are accessed.
2022-04-21 15:46:06 +01:00
Alex Potsides
147304449e
fix: expose getPublicKey (#1188)
This is used externally by IPFS so expose the method
2022-04-14 18:00:21 +01:00
Alex Potsides
c64a586a20
chore: update aegir to the latest version (#1186)
Removes boilerplate config that is no longer necessary
2022-04-09 09:26:25 +01:00
Alex Potsides
cc60cfde1a
fix: add transport manager to exports map and fix docs (#1182)
Addresses PR comments from #1172 - fixes syntax of examples in docs, adds the transport manager to the exports map and renames fault tolerance enum for consistency.
2022-03-29 15:39:50 +01:00
Alex Potsides
199395de4d
feat: convert to typescript (#1172)
Converts this module to typescript.

- Ecosystem modules renamed from (e.g.) `libp2p-tcp` to `@libp2p/tcp`
- Ecosystem module now have named exports
- Configuration has been updated, now pass instances of modules instead of classes:
- Some configuration keys have been renamed to make them more descriptive.  `transport` -> `transports`, `connEncryption` -> `connectionEncryption`.  In general where we pass multiple things, the key is now plural, e.g. `streamMuxer` -> `streamMuxers`, `contentRouting` -> `contentRouters`, etc.  Where we are configuring a singleton the config key is singular, e.g. `connProtector` -> `connectionProtector` etc.
- Properties of the `modules` config key have been moved to the root
- Properties of the `config` config key have been moved to the root
```js
// before
import Libp2p from 'libp2p'
import TCP from 'libp2p-tcp'

await Libp2p.create({
  modules: {
    transport: [
      TCP
    ],
  }
  config: {
    transport: {
      [TCP.tag]: {
        foo: 'bar'
      }
    },
    relay: {
      enabled: true,
      hop: {
        enabled: true,
        active: true
      }
    }
  }
})
```
```js
// after
import { createLibp2p } from 'libp2p'
import { TCP } from '@libp2p/tcp'

await createLibp2p({
  transports: [
    new TCP({ foo: 'bar' })
  ],
  relay: {
    enabled: true,
    hop: {
      enabled: true,
      active: true
    }
  }
})
```
- Use of `enabled` flag has been reduced - previously you could pass a module but disable it with config.  Now if you don't want a feature, just don't pass an implementation.   Eg:
```js
// before
await Libp2p.create({
  modules: {
    transport: [
      TCP
    ],
    pubsub: Gossipsub
  },
  config: {
    pubsub: {
      enabled: false
    }
  }
})
```
```js
// after
await createLibp2p({
  transports: [
    new TCP()
  ]
})
```
- `.multiaddrs` renamed to `.getMultiaddrs()` because it's not a property accessor, work is done by that method to calculate announce addresses, observed addresses, etc
- `/p2p/${peerId}` is now appended to all addresses returned by `.getMultiaddrs()` so they can be used opaquely (every consumer has to append the peer ID to the address to actually use it otherwise).  If you need low-level unadulterated addresses, call methods on the address manager.

BREAKING CHANGE: types are no longer hand crafted, this module is now ESM only
2022-03-28 14:30:27 +01:00
Alex Potsides
902f10d58d
fix: reject connections when not running (#1146)
When the node is shutting down, new connections can still be received.

If this happens we can end up writing into the datastore when it's
been closed which throws an error.

Instead, if we're not running, have the connection manager close new
incoming connections.
2022-01-26 10:52:23 +00:00
Alex Potsides
d44bd9094f
fix: await unhandle of protocols (#1144)
To allow us to shut down cleanly, we must wait the unhandling of protocols - this is because they write the new list of protocols into the datastore which might also be in the process of shutting down.
2022-01-25 19:56:56 +00:00
Alex Potsides
ff32eba6a0
feat: connection gater (#1142)
Port of https://github.com/libp2p/go-libp2p-core/blob/master/connmgr/gater.go

Adds a new configuration key `connectionGater` which allows denying the dialing of certain peers, individual multiaddrs and the creation of connections at certain points in the connection flow.

Fixes: https://github.com/libp2p/js-libp2p/issues/175
Refs: https://github.com/libp2p/js-libp2p/issues/744
Refs: https://github.com/libp2p/js-libp2p/issues/769

Co-authored-by: mzdws <8580712+mzdws@user.noreply.gitee.com>
2022-01-25 16:27:01 +00:00
Robert Kiel
9b22c6e2f9
fix: prevent auto-dialer from dialing self (#1104)
Co-authored-by: Robert Kiel <robert.kiel@hoprnet.io>
Co-authored-by: achingbrain <alex@achingbrain.net>
2022-01-24 17:59:14 +00:00
Spencer T Brody
d8ceb0bc66
feat: add fetch protocol (#1036)
Adds three methods to implement the `/libp2p/fetch/0.0.1` protocol:

* `libp2p.fetch(peerId, key) => Promise<Uint8Array>`
* `libp2p.fetchService.registerLookupFunction(prefix, lookupFunction)`
* `libp2p.fetchService.unRegisterLookupFunction(prefix, [lookupFunction])`

Co-authored-by: achingbrain <alex@achingbrain.net>
2022-01-24 17:07:11 +00:00
Alex Potsides
63aa480800
fix: try all peer addresses when dialing a relay (#1140)
The order of the addresses can affect our success rate in dialing a
relay - if it's a loopback address or similar it won't work.

Instead try dialing every address.
2022-01-21 11:57:09 -06:00
Alex Potsides
b7e87066a6
fix: make tests more reliable (#1139)
Try to use only public functions and properties to verify test behaviour
2022-01-21 12:58:31 +00:00
Alex Potsides
4c3bf01f35
chore: fix flaky tests (#1137)
These tests are flaky in CI, probably due to differences in timing introduced by #1058

Fixes #1134
2022-01-21 08:44:13 +00:00
Alex Potsides
978eb3676f
feat: async peerstore backed by datastores (#1058)
We have a peerstore that keeps all data for all observed peers in memory with no eviction.

This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.

We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.

It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.

This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives.  It can be configured with a MemoryDatastore if desired.

It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.

BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
2022-01-20 12:03:35 +00:00
Alex Potsides
0a4dc54d08
fix: remove abort-controller dep (#1095)
The `AbortController` class is supported by browsers and node 14+ - we only support node 16+ (e.g. LTS+Current) so the `abort-controller` module isn't needed any more.
2022-01-20 12:02:13 +00:00
Alex Potsides
5043cd5643
fix: cache build artefacts (#1091)
To speed up the build and make it more reliable, cache the node_modules
folder, dist, etc and re-use on each step.
2021-12-29 15:06:58 +01:00
Alex Potsides
2f0b311df7
feat: allow per-component metrics to be collected (#1061)
Implements the idea from #1060 - allows us to get some insight into what's happening in a libp2p node out side of just bandwidth stats.

Configures a few default metrics if metrics are enabled - current connections, the state of the dial queue, etc.

Also makes the `Metrics` class not depend on the `ConnectionManager` class, otherwise we can't collect simple metrics from the connection manager class due to the circular dependency.
2021-12-15 08:03:09 +00:00
Alex Potsides
1b46f47fdb
chore: run node tests in ci (#1057)
Looks like this project stopped running the `test:node` npm script when it was migrated to gh actions.

Re-enable it and fix all the related test failures.
2021-12-08 08:38:17 +00:00
Robert Kiel
3b683e7156
fix: fix uncaught promise rejection when finding peers (#1044)
Do not abort all attempts to find peers when `findPeers` on one router throws synchronously

Co-authored-by: Robert Kiel <robert.kiel@hoprnet.io>
Co-authored-by: achingbrain <alex@achingbrain.net>
2021-12-07 17:51:48 +00:00
Alex Potsides
b25e0fe531
fix: make error codes consistent (#1054)
Sometimes they are `NOT_FOUND`, sometimes `ERR_NOT_FOUND`, etc.

Move all error codes into `errors.js` and reference them from there.
2021-12-07 14:42:10 +00:00
Alex Potsides
9cbf36fcb5
chore: update peer id and libp2p crypto (#1042)
BREAKING CHANGE: requires node 15+
2021-12-02 10:11:23 +00:00
Alex Potsides
eacd7e8f76
chore: update deps (#1038) 2021-11-26 16:00:47 +00:00
Alex Potsides
2f598eba09
feat: update dht (#1009)
Changes dht creation to use factory function and updates docs

BREAKING CHANGE: libp2p-kad-dht has a new event-based API which is exposed as `_dht`
2021-11-25 16:32:19 +00:00
Alex Potsides
3bed7b4cb2
chore: update aegir (#1027)
Updates aegir, fixes all new linting errors.
2021-11-19 08:02:24 +00:00
Alex Potsides
83734ef520
chore: update datastore (#990)
`interface-datastore` now only contains the interface definition,
`datastore-core` has the various implementations.

BREAKING CHANGE: datastore implementations provided to libp2p must be compliant with interface-datastore@6.0.0
2021-09-24 10:24:29 +02:00
Vasco Santos
3d25ff7fd0 chore: use new libp2p-noise 2021-08-20 09:13:21 +02:00
Vasco Santos
0f389a7828 chore: update uint8arrays 2021-08-20 09:13:21 +02:00
greenSnot
ef24fabf02
feat: custom protocol name (#962)
Co-authored-by: mzdws <8580712+mzdws@user.noreply.gitee.com>
2021-08-13 16:21:50 +02:00
Vasco Santos
af723b355e
fix: do not allow dial to large number of multiaddrs (#954) 2021-07-09 08:46:24 +02:00
Alex Potsides
13cf476148
chore: update to new multiformats (#948)
BREAKING CHANGE: uses the CID class from the new multiformats module

Co-authored-by: Vasco Santos <vasco.santos@moxy.studio>
2021-07-09 08:43:34 +02:00
Vasco Santos
39b03586e8
chore: use libp2p-tcp with types (#952) 2021-06-16 09:09:26 +02:00
Vasco Santos
b291bc06ec
fix: dialer leaking resources after stopping (#947)
* fix: dialer leaking resources after stopping

* chore: add error code to test
2021-06-14 09:19:23 +02:00
Vasco Santos
50f7f32e53 chore: update branch 2021-06-13 21:42:36 +02:00
Vasco Santos
2c4b567b00 chore: restructure pubsub tests 2021-06-13 21:42:36 +02:00
Alex Potsides
2a6a635f13
chore: remove ipfs-utils dep (#953)
* chore: remove ipfs-utils dep

We only use it for the env detection, so use [wherearewe](https://www.npmjs.com/package/wherearewe)
instead which is that, but pulled out into a tiny module.

The `TextDecoder` class is global everywhere we support so we don't
need to pull it in from `ipfs-utils` and it's been removed from v8
anyway.

* chore: update ipfs-http-client
2021-06-11 10:01:40 +02:00
zeim839
478963ad2d
feat: keychain rotate passphrase (#944)
Co-authored-by: Vasco Santos <vasco.santos@ua.pt>
2021-05-27 10:30:19 +02:00
Alex Potsides
818d2b2a98
fix: store remote agent and protocol version during identify (#943) 2021-05-12 19:11:17 +02:00
zeim839
a79c6b50d7
fix: peerRouting.findPeer() trying to find self (#941)
* throw error if node attempts to find itself

Co-authored-by: Vasco Santos <vasco.santos@ua.pt>
2021-05-12 16:46:20 +02:00
Vasco Santos
ac370fc967
fix: address book guarantees no replicated entries are added (#927) 2021-04-27 17:14:48 +02:00
Vasco Santos
ef4393649f
chore: remove chai deps (#929) 2021-04-27 17:13:34 +02:00
Vasco Santos
cc1f4af879
fix: dht configuration selectors and validators (#919) 2021-04-21 13:28:37 +02:00