1136 Commits

Author SHA1 Message Date
Alex Potsides
13d45de376
chore: add missing cache dir (#1125)
Fixes ci config
2022-01-20 12:37:52 +00:00
Alex Potsides
978eb3676f
feat: async peerstore backed by datastores (#1058)
We have a peerstore that keeps all data for all observed peers in memory with no eviction.

This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.

We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.

It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.

This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives.  It can be configured with a MemoryDatastore if desired.

It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.

BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
2022-01-20 12:03:35 +00:00
Alex Potsides
0a4dc54d08
fix: remove abort-controller dep (#1095)
The `AbortController` class is supported by browsers and node 14+ - we only support node 16+ (e.g. LTS+Current) so the `abort-controller` module isn't needed any more.
2022-01-20 12:02:13 +00:00
Tim Daubenschütz
c3700f55d5
fix: import uint8arrays package in example (#1083) 2022-01-14 16:33:17 +01:00
dadepo
96d3461393
docs: Include loadkeychain in the index (#1087) 2022-01-10 11:56:08 +01:00
dadepo
5e5d11ec19
docs: fix import of datastore-level (#1086) 2022-01-08 17:57:37 +01:00
Vasco Santos
4cadbad102
chore: remove pubsub dev deps (#1107) 2022-01-07 11:40:05 +01:00
Alex Potsides
5043cd5643
fix: cache build artefacts (#1091)
To speed up the build and make it more reliable, cache the node_modules
folder, dist, etc and re-use on each step.
2021-12-29 15:06:58 +01:00
achingbrain
ef54e0a10e chore: release version v0.35.8 v0.35.8 2021-12-29 11:00:11 +01:00
achingbrain
61bf546c46 chore: update contributors 2021-12-29 11:00:11 +01:00
Alex Potsides
d2b7ec0f6b
fix: look for final peer event instead of peer response (#1092)
`FINAL_PEER` means we found the peer, `PEER_RESPONSE` means a peer
responded to our query.
2021-12-29 10:56:56 +01:00
Alex Potsides
79b3cfc6ad
fix: do not wait for autodial start (#1089)
When we've previously seen loads of peers and stored them in the
datastore we'll try to dial them as part of starting the autodial
component.

If we or our peers have bad network connections this can make
starting a libp2p node take ages so don't wait for a round of auto
dialing before considering the component started.
2021-12-29 10:55:48 +01:00
Alex Potsides
f18fc80b70
fix: increase listeners on any-signal (#1084)
Increase the number of listeners we allow on the actual signal we pass along, instead of the signal we pass into any-signal.
2021-12-29 10:51:26 +01:00
Alex Potsides
b4b432406e
fix: record tracked map clears (#1085)
Record the size of a map after we `.clear()` it.
2021-12-27 07:14:27 +01:00
Vasco Santos
bbdd559a02 chore: release version v0.35.7 v0.35.7 2021-12-24 14:51:51 +00:00
Vasco Santos
4070dcdf55 chore: update contributors 2021-12-24 14:51:51 +00:00
tuyennhv
cb0d7d6c99
fix: type definitions for big dialrequest and persistent peerstore (#1078)
Signed-off-by: Tuyen Nguyen <vutuyen2636@gmail.com>
2021-12-24 15:46:00 +01:00
Vasco Santos
d1c48dcbed
fix: main ci (#1079) 2021-12-24 11:18:09 +01:00
Vít Habada
c4a442788b
docs: update example config ipfs links (#1077) 2021-12-22 17:03:03 +01:00
Vít Habada
70a4bb9451
docs: peerstore configuration datastore fixed 2021-12-22 14:15:18 +01:00
Tim Daubenschütz
a0516ebc85
docs: update node and npm version badge according to package.json (#1074) 2021-12-22 14:01:54 +01:00
Alex Potsides
b425fa1230
fix: add tracked map (#1069)
Small refactor of the component stats - adds a `TrackedMap` which encapsulates updating the metrics and means we don't need to null guard on `this._metrics` everywhere.

If metrics are not enabled a regular `Map` is used.
2021-12-21 15:51:06 +01:00
achingbrain
0a485d07b3 chore: release version v0.35.6 v0.35.6 2021-12-18 07:55:27 +01:00
achingbrain
0c3ed0a4ac chore: update contributors 2021-12-18 07:55:26 +01:00
Alex Potsides
09a0f940df
fix: increase the maxlisteners for timeout controllers (#1065)
We use timeout controllers to ensure we're not dialling peers forever but we can end up registering lots of listeners for the `abort` event when peers have a lot of addresses.

In node this means we see an unhelpful `MaxListenersExceededWarning` in the console warning of a potential memory leak.

Increase the max number of listeners on the signal to silence the warning.
2021-12-18 07:34:27 +01:00
Alex Potsides
a642ad2a03
chore(deps-dev): bump libp2p-floodsub from 0.27.1 to 0.28.0 (#1062)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-15 09:30:49 +00:00
achingbrain
8ce2f08589 chore: release version v0.35.5 v0.35.5 2021-12-15 09:28:05 +00:00
achingbrain
fe0d9828bb chore: update contributors 2021-12-15 09:28:04 +00:00
achingbrain
c8e1b08c19 chore: typo 2021-12-15 09:25:40 +00:00
achingbrain
faf1f89d9e chore: release version v0.35.4 v0.35.4 2021-12-15 08:18:44 +00:00
achingbrain
76f4ea5e8a chore: update contributors 2021-12-15 08:18:44 +00:00
Alex Potsides
2f0b311df7
feat: allow per-component metrics to be collected (#1061)
Implements the idea from #1060 - allows us to get some insight into what's happening in a libp2p node out side of just bandwidth stats.

Configures a few default metrics if metrics are enabled - current connections, the state of the dial queue, etc.

Also makes the `Metrics` class not depend on the `ConnectionManager` class, otherwise we can't collect simple metrics from the connection manager class due to the circular dependency.
2021-12-15 08:03:09 +00:00
achingbrain
d172d0d952 chore: release version v0.35.3 v0.35.3 2021-12-13 09:03:54 +00:00
achingbrain
f8e8023aed chore: update contributors 2021-12-13 09:03:54 +00:00
Alex Potsides
bdc9f16d0c
fix: clean up pending dial targets (#1059)
If the `Promise.race` throws, execution of the function is terminated so the pending dial target is never removed from the map and we leak memory.

This can happen when there are invalid multiaddrs or when a peer reports more dialable addresses than the threshold.

Instead wrap the `Promise.race` in a `try/finally` which will always remove the pending dial target in the event of success or failure.
2021-12-10 12:42:09 +00:00
Alex Potsides
1b46f47fdb
chore: run node tests in ci (#1057)
Looks like this project stopped running the `test:node` npm script when it was migrated to gh actions.

Re-enable it and fix all the related test failures.
2021-12-08 08:38:17 +00:00
dependabot[bot]
b539f9b655
chore(deps-dev): bump libp2p-gossipsub from 0.11.4 to 0.12.1 (#1045)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-07 21:06:06 +00:00
dependabot[bot]
103818733e
chore(deps-dev): bump @chainsafe/libp2p-noise from 4.1.1 to 5.0.0 (#1053)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-07 21:05:53 +00:00
Marston Connell
1f1bbc0ee6
docs: naming error in the documentation (#1056)
Changed LevelStore to LevelDatastore
2021-12-07 21:05:33 +00:00
Robert Kiel
3b683e7156
fix: fix uncaught promise rejection when finding peers (#1044)
Do not abort all attempts to find peers when `findPeers` on one router throws synchronously

Co-authored-by: Robert Kiel <robert.kiel@hoprnet.io>
Co-authored-by: achingbrain <alex@achingbrain.net>
2021-12-07 17:51:48 +00:00
Alex Potsides
b25e0fe531
fix: make error codes consistent (#1054)
Sometimes they are `NOT_FOUND`, sometimes `ERR_NOT_FOUND`, etc.

Move all error codes into `errors.js` and reference them from there.
2021-12-07 14:42:10 +00:00
Alex Potsides
cbaa5a2ef3
chore: switch to nat api (#1052)
@motrix/nat-api is a fork, nat-api has the fix from https://github.com/alxhotel/nat-api/pull/25
2021-12-07 09:37:12 +00:00
Alan Smithee
51dabb1724
chore: pubsub example subscribe returns void (#1048)
Seems like the correct return type of `Libp2p.pubsub.subscribe` is `void`, so the `await` can be removed: ae21299ade/src/pubsub-adapter.js (L29)
2021-12-06 21:12:38 +01:00
Vasco Santos
b9339bccaa chore: release version v0.35.2 v0.35.2 2021-12-06 21:07:39 +01:00
Vasco Santos
9b21893b64 chore: update contributors 2021-12-06 21:07:38 +01:00
Alex Potsides
b70fb43427
fix: increase maxlisteners on event target (#1050)
Sometimes you encounter peers with lots of addresses. When this happens
you can attach more than 10x event listeners to the abort signal we
use to abort all the dials - this causes node to print a warning
which is misleading.

This PR increases the default number of listeners on the signal.

Fixes #900
2021-12-06 20:54:44 +01:00
achingbrain
ae21299ade chore: release version v0.35.1 v0.35.1 2021-12-03 16:24:14 +00:00
achingbrain
149120bebc chore: update contributors 2021-12-03 16:24:13 +00:00
Alex Potsides
91c2ec9856
fix: do not let closest peers run forever (#1047)
The DHT takes a `signal` not a timeout so if a timeout is passed,
create a `TimeoutController` that will abort the query after the
timeout.
2021-12-03 15:47:30 +00:00
achingbrain
6d0ac819f1 chore: release version v0.35.0 v0.35.0 2021-12-02 10:44:07 +00:00