Alex Potsides 978eb3676f
feat: async peerstore backed by datastores (#1058)
We have a peerstore that keeps all data for all observed peers in memory with no eviction.

This is fine when you don't discover many peers but when using the DHT you encounter a significant number of peers so our peer storage grows and grows over time.

We have a persistent peer store, but it just periodically writes peers into the datastore to be read at startup, still keeping them in memory.

It also means a restart doesn't give you any temporary reprieve from the memory leak as the previously observed peer data is read into memory at startup.

This change refactors the peerstore to use a datastore by default, reading and writing peer info as it arrives.  It can be configured with a MemoryDatastore if desired.

It was necessary to change the peerstore and *book interfaces to be asynchronous since the datastore api is asynchronous.

BREAKING CHANGE: `libp2p.handle`, `libp2p.registrar.register` and the peerstore methods have become async
2022-01-20 12:03:35 +00:00

57 lines
1.5 KiB
JavaScript

/* eslint-disable no-console */
'use strict'
const Libp2p = require('../../')
const TCP = require('libp2p-tcp')
const Mplex = require('libp2p-mplex')
const { NOISE } = require('@chainsafe/libp2p-noise')
const Gossipsub = require('@achingbrain/libp2p-gossipsub')
const { fromString: uint8ArrayFromString } = require('uint8arrays/from-string')
const { toString: uint8ArrayToString } = require('uint8arrays/to-string')
const createNode = async () => {
const node = await Libp2p.create({
addresses: {
listen: ['/ip4/0.0.0.0/tcp/0']
},
modules: {
transport: [TCP],
streamMuxer: [Mplex],
connEncryption: [NOISE],
pubsub: Gossipsub
}
})
await node.start()
return node
}
;(async () => {
const topic = 'news'
const [node1, node2] = await Promise.all([
createNode(),
createNode()
])
// Add node's 2 data to the PeerStore
await node1.peerStore.addressBook.set(node2.peerId, node2.multiaddrs)
await node1.dial(node2.peerId)
node1.pubsub.on(topic, (msg) => {
console.log(`node1 received: ${uint8ArrayToString(msg.data)}`)
})
node1.pubsub.subscribe(topic)
// Will not receive own published messages by default
node2.pubsub.on(topic, (msg) => {
console.log(`node2 received: ${uint8ArrayToString(msg.data)}`)
})
node2.pubsub.subscribe(topic)
// node2 publishes "news" every second
setInterval(() => {
node2.pubsub.publish(topic, uint8ArrayFromString('Bird bird bird, bird is the word!'))
}, 1000)
})()