Enable streams in http, and add support for preferredTransports as parameter to createReadStream (as it was in p_f_createReadStream)

This commit is contained in:
Mitra Ardron 2019-04-14 09:06:03 +10:00
parent 729bab2728
commit e7971a73df
5 changed files with 38 additions and 9 deletions

32
API.md
View File

@ -239,14 +239,26 @@ returns: Dictionary of Key:Value pairs, note take care if this could
```
### Transports - other functions
##### static async p_f_createReadStream(url, {wanturl})
##### static async p_f_createReadStream(url, {wanturl, preferredTransports=[] })
Provide a function of the form needed by <VIDEO> tag and renderMedia library etc
```
url Urls of stream
wanturl True if want the URL of the stream (for service workers)
preferredTransports: preferred order to select stream transports (usually determined by application)
returns f(opts) => stream returning bytes from opts.start || start of file to opts.end-1 || end of file
```
##### static createReadStream(urls, opts, cb)
Different interface, more suitable when just want a stream, now.
```
urls: Url or [urls] of the stream
opts{
start, end: First and last byte wanted (default to 0...last)
preferredTransports: preferred order to select stream transports (usually determined by application)
}
returns open readable stream from the net via cb or promise
```
##### supports(url, funcl)
Determines if the Transport supports urls of this form. For example TransportIPFS supports URLs starting ipfs:
```
@ -500,7 +512,8 @@ SupportFunctions (note YJS uses IPFS and supports some other functions):
SupportFeatures:
fetch.range Not supported (currently April 2018))
Currently there is code for p_f_createReadStream. It works but because of some other IPFS issues is disabled.
Currently there is code for p_f_createReadStream. It works but because IPFS cannot return an error even if it
cannot open the stream, IPFS is usually set as the last choice transport for streams.
## TransportYJS
A subclass of Transport for handling YJS connections.
@ -520,8 +533,10 @@ When used with a SW, it will attempt to retrieve from the http backup URL that i
In the SW it will also generate errors about trackers because the only reason to use trackers is to get the WebRTC links.
supportURLS = `magnet:*` (TODO: may in the future support `dweb:/magnet/*`)
supportFunctions:
`fetch, createReadStream`
`fetch`, `createReadStream`
supportFeatures:
fetch.range Not supported (currently April 2018)
@ -529,10 +544,17 @@ supportFeatures:
A subclass of Transport for handling GUN connections (decentralized database)
supportURLS = `gun:*` (TODO: may in the future support `dweb:/gun/*`)
supportFunctions
`add, list, listmonitor, newlisturls, connection, get, set, getall, keys, newdatabase, newtable, monitor`
supportFunctions = `add`, `list`, `listmonitor`, `newlisturls`, `connection`, `get`, `set`, `getall`, `keys`, `newdatabase`, `newtable`, `monitor`
supportFeatures:
## TransportWOLK
A subclass of Transport for handling the WOLK transport layer (decentralized, block chain based, incentivised storage)
supportURLs = ['wolk'];
supportFunctions = [ 'fetch', 'connection', 'get', 'set', ]; // 'store' - requires chunkdata; 'createReadStream' not implemented
## Naming
Independently from the transport, the Transport library can resolve names if provided an appropriate callback.
See p_resolveNames(urls) and resolveNamesWith(cb)

View File

@ -31,7 +31,7 @@ class TransportHTTP extends Transport {
this.options = options;
this.urlbase = options.urlbase;
this.supportURLs = ['contenthash', 'http','https'];
this.supportFunctions = ['fetch', 'store', 'add', 'list', 'reverse', 'newlisturls', "get", "set", "keys", "getall", "delete", "newtable", "newdatabase"]; //Does not support: listmonitor - reverse is disabled somewhere not sure if here or caller
this.supportFunctions = ['fetch', 'store', 'add', 'list', 'reverse', 'newlisturls', "get", "set", "keys", "getall", "delete", "newtable", "newdatabase", "createReadStream"]; //Does not support: listmonitor - reverse is disabled somewhere not sure if here or caller
// noinspection JSUnusedGlobalSymbols
this.supportFeatures = ['fetch.range'];
this.name = "HTTP"; // For console log etc
@ -164,6 +164,7 @@ class TransportHTTP extends Transport {
Node.js readable stream docs: https://nodejs.org/api/stream.html#stream_readable_streams
:param string url: URL of object being retrieved of form magnet:xyzabc/path/to/file (Where xyzabc is the typical magnet uri contents)
:param boolean wanturl True if want the URL of the stream (for service workers)
:resolves to: f({start, end}) => stream (The readable stream.)
:throws: TransportError if url invalid - note this happens immediately, not as a catch in the promise
*/

View File

@ -362,6 +362,7 @@ class TransportIPFS extends Transport {
:param string url: URL of object being retrieved of form:
magnet:xyzabc/path/to/file (Where xyzabc is the typical magnet uri contents)
ipfs:/ipfs/Q123
:param boolean wanturl True if want the URL of the stream (for service workers)
:resolves to: f({start, end}) => stream (The readable stream.)
:throws: TransportError if url invalid - note this happens immediately, not as a catch in the promise
*/

View File

@ -235,6 +235,7 @@ class TransportWEBTORRENT extends Transport {
Node.js readable stream docs: https://nodejs.org/api/stream.html#stream_readable_streams
:param string url: URL of object being retrieved of form magnet:xyzabc/path/to/file (Where xyzabc is the typical magnet uri contents)
:param boolean wanturl True if want the URL of the stream (for service workers)
:resolves to: f({start, end}) => stream (The readable stream.)
:throws: TransportError if url invalid - note this happens immediately, not as a catch in the promise
*/

View File

@ -299,6 +299,7 @@ class Transports {
static async p_f_createReadStream(urls, {wanturl=false, preferredTransports=[]}={}) { // Note options is options for selecting a stream, not the start/end in a createReadStream call
/*
urls: Url or [urls] of the stream
wanturl True if want the URL of the stream (for service workers)
returns: f(opts) => stream returning bytes from opts.start || start of file to opts.end-1 || end of file
*/
let tt = this.validFor(urls, "createReadStream", {}); //[ [Url,t],[Url,t]] // Can pass options TODO-STREAM support options in validFor
@ -331,16 +332,19 @@ class Transports {
debugtransports("Opening stream to %o failed on all transports", urls);
throw new errors.TransportError(errs.map((err)=>err.message).join(', ')); //Throw err with combined messages if none succeed
}
static createReadStream(urls, opts, cb) { //TODO-API
static createReadStream(urls, opts, cb) {
/*
Different interface, more suitable when just want a stream, now.
urls: Url or [urls] of the stream
opts{start, end}: First and last byte wanted (default to 0...last)
opts{
start, end: First and last byte wanted (default to 0...last)
preferredTransports: preferred order to select stream transports (usually determined by application)
}
cb(err, stream): Called with open readable stream from the net.
Returns promise if no cb
*/
if (typeof opts === "function") { cb = opts; opts = {start: 0}; } // Allow skipping opts
DwebTransports.p_f_createReadStream(urls)
DwebTransports.p_f_createReadStream(urls, {preferredTransports: (opts.preferredTransports || [])})
.then(f => {
let s = f(opts);
if (cb) { cb(null, s); } else { return(s); }; // Callback or resolve stream