From cc03e4334cefc3f84f43c2b906d811cff18ce701 Mon Sep 17 00:00:00 2001 From: Mitra Ardron Date: Mon, 13 Aug 2018 17:55:04 +1000 Subject: [PATCH] Replace verbose with debug package for logging - looks good --- API.md | 107 +- README.md | 4 +- Transport.js | 108 +- TransportGUN.js | 105 +- TransportHTTP.js | 128 +- TransportIPFS.js | 105 +- TransportWEBTORRENT.js | 69 +- TransportYJS.js | 116 +- Transports.js | 254 +- dist/dweb-transports-bundle.js | 6456 +++++++++++++++++++++++++++++++- example_block.html | 9 +- httptools.js | 21 +- package.json | 2 +- test.js | 6 +- test_ipfs.js | 2 +- 15 files changed, 6833 insertions(+), 659 deletions(-) diff --git a/API.md b/API.md index 227c1e4..0b0c3de 100644 --- a/API.md +++ b/API.md @@ -13,12 +13,10 @@ We use a naming convention that anything starting “p_” returns a promise so Ideally functions should take a String, Buffer or where applicable Object as parameters with automatic conversion. And anything that takes a URL should take either a string or parsed URL object. -The verbose parameter is a boolean that is an indicator of a need to output to the console. Normally it will be passed down to called functions and default to false. - Note that example_block.html collects this from the URL and passes it to the library, which is intended to be a good way to see what is happening. -Note: I am gradually (March2018) changing the API to take an opts {} dict which includes verbose as one field. This process is incomplete, but I’m happy to see it accelerated if there is any code built on this, just let mitra@archive.org know. +Note: I am gradually (March2018) changing the API to take an opts {} dict. This process is incomplete, but I’m happy to see it accelerated if there is any code built on this, just let mitra@archive.org know. ## Overview @@ -55,19 +53,18 @@ status|Numeric indication of transport status: Started(0); Failed(1); Starting(2 Transport setup is split into 3 parts, this allows the Transports class to do the first phase on all the transports synchronously, then asynchronously (or at a later point) try and connect to all of them in parallel. -##### static setup0 (options, verbose) +##### static setup0 (options) First part of setup, create obj, add to Transports but dont attempt to connect, typically called instead of p_setup if want to parallelize connections. In almost all cases this will call the constructor of the subclass Should be synchronous and leave `status=STATUS_LOADED` ``` options Object fields including those needed by transport layer -verbose boolean - true for debugging output Resolves to Instance of subclass of Transport ``` Default options should be set in each transport, but can be overwritten, for example to overwrite the options for HTTP call it with `options={ http: { urlbase: “https://dweb.me:443/” } }` -##### async p_setup1 (verbose, cb) +##### async p_setup1 (, cb) Setup the resource and open any P2P connections etc required to be done just once. Asynchronous and should leave `status=STATUS_STARTING` until it resolves, or `STATUS_FAILED` if fails. ``` @@ -75,7 +72,7 @@ cb (t)=>void If set, will be called back as status changes (so could be multip Resolves to the Transport instance ``` -##### async p_setup2 (verbose, cb) +##### async p_setup2 (, cb) Works like p_setup1 but runs after p_setup1 has completed for all transports. This allows for example YJS to wait for IPFS to be connected in TransportIPFS.setup1() and then connect itself using the IPFS object. @@ -84,7 +81,7 @@ cb (t)=>void If set, will be called back as status changes (so could be multip Resolves to the Transport instance ``` -##### async p_setup(options, verbose, cb) +##### async p_setup(options, cb) A deprecated utility to simply setup0 then p_setup1 then p_setup2 to allow a transport to be started in one step, normally `Transports.p_setup` should be called instead. @@ -95,7 +92,7 @@ in the paused state it will not be used for transport but, in some cases, will s cb(transport)=>void a callback called after this is run, may be used for example to change the UI ``` -##### async p_status (verbose) +##### async p_status () Check the status of the underlying transport. This may update the "status" field from the underlying transport. ``` returns: a numeric code for the status of a transport. @@ -110,15 +107,14 @@ Code|Name|Means 4|STATUS_PAUSED|It was launched, probably connected, but now paused so will be ignored by validfor() ### Transport: General storage and retrieval of objects -##### p_rawstore(data, {verbose}) +##### p_rawstore(data) Store a opaque blob of data onto the decentralised transport. ``` data string|Buffer data to store - no assumptions made to size or content -verbose boolean - True for debugging output Resolves to url of data stored ``` -##### p_rawfetch(url, {timeoutMS, start, end, relay, verbose}) +##### p_rawfetch(url, {timeoutMS, start, end, relay}) Fetch some bytes based on a url, no assumption is made about the data in terms of size or structure. Where required by the underlying transport it should retrieve a number if its "blocks" and concatenate them. @@ -129,13 +125,12 @@ url string url of object being retrieved in form returned by link or p_ timeoutMS Max time to wait on transports that support it (IPFS for fetch) start,end Inclusive byte range wanted (must be supported, uses a "slice" on output if transport ignores it. relay If first transport fails, try and retrieve on 2nd, then store on 1st, and so on. -verbose boolean - True for debugging output Resolves to string The object being fetched, (note currently (April 2018) returned as a string, may refactor to return Buffer) throws: TransportError if url invalid - note this happens immediately, not as a catch in the promise ``` ### Transport: Handling lists -##### p_rawadd(url, sig, {verbose}) +##### p_rawadd(url, sig) Store a new list item, it should be stored so that it can be retrieved either by "signedby" (using p_rawlist) or by "url" (with p_rawreverse). @@ -148,51 +143,46 @@ sig Signature data structure (see below - contains url, date, signedby, s urls - array of urls for the object being signed signature - verifiable signature of date+urls signedby - url of data structure (typically CommonList) holding public key used for the signature -verbose boolean - True for debugging output ``` -##### p_rawlist(url, {verbose}) +##### p_rawlist(url) Fetch all the objects in a list, these are identified by the url of the public key used for signing. Note this is the 'signedby' parameter of the p_rawadd call, not the 'url' parameter. List items may have other data (e.g. reference ids of underlying transport) ``` url String with the url that identifies the list (this is the 'signedby' parameter of the p_rawadd call, not the 'url' parameter -verbose boolean - True for debugging output Resolves to Array: An array of objects as stored on the list. Each of which is ….. ``` Each item of the list is a dict: {"url": url, "date": date, "signature": signature, "signedby": signedby} -##### p_rawreverse (url, {verbose}) +##### p_rawreverse (url) Similar to p_rawlist, but return the list item of all the places where the object url has been listed. (not supported by most transports) ``` url String with the url that identifies the object put on a list This is the “url” parameter of p_rawadd -verbose boolean - True for debugging output Resolves to Array objects as stored on the list (see p_rawlist for format) ``` -##### listmonitor (url, cb, {verbose, current}) +##### listmonitor (url, cb, { current}) Setup a callback called whenever an item is added to a list, typically it would be called immediately after a p_rawlist to get any more items not returned by p_rawlist. ``` url Identifier of list (as used by p_rawlist and "signedby" parameter of p_rawadd cb(obj) function(obj) Callback for each new item added to the list -verbose true for debugging output current true to send existing members as well as new obj is same format as p_rawlist or p_rawreverse ``` -##### async p_newlisturls(cl, {verbose}) +##### async p_newlisturls(cl) Obtain a pair of URLs for a new list. The transport can use information in the cl to generate this or create something random (the former is encouraged since it means repeat tests might not generate new lists). Possession of the publicurl should be sufficient to read the list, the privateurl should be required to read (for some transports they will be identical, and higher layers should check for example that a signature is signed. ``` cl CommonList instance that can be used as a seed for the URL -verbose boolean - True for debugging output Returns [privateurl, publicurl] ``` ### Transport: Support for KeyValueTable -##### async p_newdatabase(pubkey, {verbose}) { +##### async p_newdatabase(pubkey) { Create a new database based on some existing object ``` pubkey: Something that is, or has a pubkey, by default support Dweb.PublicPrivate, KeyPair @@ -200,7 +190,7 @@ pubkey: Something that is, or has a pubkey, by default support Dweb.PublicPr returns: {publicurl, privateurl} which may be the same if there is no write authentication ``` -##### async p_newtable(pubkey, table, {verbose}) { +##### async p_newtable(pubkey, table) { Create a new table, ``` pubkey: Is or has a pubkey (see p_newdatabase) @@ -208,7 +198,7 @@ table: String representing the table - unique to the database returns: {privateurl, publicurl} which may be the same if there is no write authentication ``` -##### async p_set(url, keyvalues, value, {verbose}) +##### async p_set(url, keyvalues, value) Set one or more keys in a table. ``` url: URL of the table @@ -216,7 +206,7 @@ keyvalues: String representing a single key OR dictionary of keys value: String or other object to be stored (its not defined yet what objects should be supported, e.g. any object ? ``` -##### async p_get(url, keys, {verbose}) +##### async p_get(url, keys) Get one or more keys from a table ``` url: URL of the table @@ -224,21 +214,21 @@ keys: Array of keys returns: Dictionary of values found (undefined if not found) ``` -##### async p_delete(url, keys, {verbose}) +##### async p_delete(url, keys) Delete one or more keys from a table ``` url: URL of the table keys: Array of keys ``` -##### async p_keys(url, {verbose}) +##### async p_keys(url) Return a list of keys in a table (suitable for iterating through) ``` url: URL of the table returns: Array of strings ``` -##### async p_getall(url, {verbose}) +##### async p_getall(url) Return a dictionary representing the table ``` url: URL of the table @@ -246,7 +236,7 @@ returns: Dictionary of Key:Value pairs, note take care if this could ``` ### Transports - other functions -##### static async p_f_createReadStream(url, {wanturl, verbose}) +##### static async p_f_createReadStream(url, {wanturl}) Provide a function of the form needed by