In the Q documentation there is this line:
"You can give the resolver to any number of producers and whoever resolves the promise first wins. Furthermore, none of the producers can observe that they lost unless you give them the promise part too."
I don't really get what that is saying. Is this for when you are constructing a promise by hand? Can someone give me an example please?
Is this for when you are constructing a promise by hand?
Exactly, you can create a promise with a deferred or the promise constructor, call resolve multiple times but only the first time is effectful. For example:
var p = new Q.Promise(function(resolve){
resolve(1);
resolve(2);
resolve(3);
resolve(4);
resolve(5); // can also pass `resolve` around.
});
p.then(function(result){
// always 1, all other calls had no effect.
});
This lets you build interesting things, for example let's build a race function that returns the result of the first to resolve of two promises (error handling omitted for brevity):
function race(p1, p2){
return new Promise(function(resolve){
p1.then(resolve);
p2.then(resolve);
});
}
Which would resolve with whichever promise is ready first, for example:
race(tryToGetFromFirstAPI(), tryToGetFromSecondAPI()).then(function(result){
// resolves as soon as the "fastest" resolved, with the result
});
Yes, I recently raised an eyebrow at that statement. Unfortunately you need already to understand Deferreds/Promises in order to understand what is actually meant.
For me, the problem starts with the introductory sentence :
Deferreds are cool because they separate the promise part from the resolver part. So:
This is misleading because Deferreds don't actually separate these components - it's more accurate to say that they combine them, but allow their separation if required.
Off the cuff, better wording would be :
Deferreds [are cool because they] include the means to derive a Promise and include, in the form of executable methods, the means to resolve or reject, all wrapped up in a single object. So:"
The sentence starting "You can give the promise to any number of consumers ..." is a bit wordy but otherwise fine.
But the sentence starting "You can give the resolver to any number of producers ..." would better read :
The resolve and/or reject methods are readily detachable allowing you, for example, to pass them to any number of producers and whoever resolves/rejects first wins; furthermore, by passing the detached methods and not the Deferred itself (and hence its Promise), none of the producers can observe whether they won or lost."
A third statement might also be included :
A Deferred can be passed in its entirety, complete with its means to resolve/reject and to derive a Promise."
In fact, all of the above can, with a little jiggery-pokery, also be achieved with the new Promise(synchronousSettlerFunction) construction and outer var(s).
It should also be said that statements like this can be reviewed and improved many times over. My attempts above took me 15 minutes so I wouldn't pretend they are the last word.
Related
The inconvenience I have is that there is no way, to my knowledge, using built-in Promise, to resolve a Promise within a synchronous block.
Specifically, the following code
var o = {};
function set(index, value) {
console.log(`setting ${index} to ${o.index = value}`);
}
Promise.resolve().then(()=>set('a', 1)).then(()=>set('b',2))
Promise.resolve().then(()=>set('c', 1)).then(()=>set('d',2))
console.log(`a is ${o.a}`);
Outputs :
a is undefined
setting a to 1
setting c to 1
setting b to 2
setting d to 2
Are there Promise like lib whose Promise would yield the following outputs:
setting a to 1
setting b to 2
setting c to 1
setting d to 2
a is 1
Obviously, no need to resort to Promises in the example above, but in my use case, setting 'a' leads to a pretreatment which only needs to be done once, whereas 'b' can be set multiple times, and is used within the same synchronous block it is set (no freedom on that).
I've looked up bluebird, and I have no clue how their promises work internally, but they seem to embrace using only promise chains.
Q Promise might be what I'm looking for, I'm not sure.
More generally, it feels daunting scouring the docs of the many Promise librairies out there querying some niche, depreciated need with likely the wrong keywords and concepts in mind.
This is definitely a case of ProblemXY where X could be solved in a variety of ways without Promises, for example by synchronous initializers, but I feel problem Y is interesting in and of itself (and likely a duplicate), so I'm still asking.
I'll get around giving problem X its own question, but long story short, I'd love to use, sometimes within a single js synchronous execution block, the expressive power of promises to pull values instead of pushing them and to flatten callbacks and try-catch pyramids.
Edit:
To clarify what I'm curious about, the way I would build the Promiselike with the desired properties is as follow :
let f = myPasync(function*(){ //equivalent to async function declaration
v = yield myPAwait(myPromiseInstance); //equivalent of v = await myPromiseInstance;
})
let pa = new myP( *(resolve, reject)=>{} ) //equivalent to new Promise((resolve, reject)=>{}) ; for simplicity, resolve and reject can be naked properties of pa
pa.myPthen((*()=>{})()) //equivalent to pa.then((v)=>{})
And then, instead of dynamically iterating over a task queue every tick, pushing thens and awaits of newly resolved promised to that queue in a FIFO manner,
I would push to the task queue in a LIFO manner and iterate over it upon any myP related function call.
So for example, when a promise is resolved, every routine awaiting it will resume before anything else, and in particular before the rest of the routine resolving the promise.
Other example when a routine awaits a promise, if that promise is already resolved, the routine is immediately resumed instead of letting other execute.
Question:
new Promise((resolve) => {
console.log(1)
resolve()
}).then(async () => {
console.log(2)
}).then(async () => {
console.log(3)
})
new Promise((resolve) => {
console.log('a')
resolve()
}).then(() => {
console.log('b')
}).then(() => {
console.log('c')
}).then(() => {
console.log('d')
}).then(() => {
console.log('e')
})
Answer:
1
a
2
b
c
d
3
e
But I think answer is.... 1 a 2 b 3 c d e
Please help to share how to think to get answer detailed. Thanks.
First I would explain that nobody should ever rely on precise timing between two separate promise chains. They run independently of one another and as soon as you insert any real world asynchronous operations in either of those promise chains (which all real-world programming would contain), then the timing of each chain is entirely unpredictable vs. the other.
Instead the timing in real code depends upon the timing of the asynchronous operations, not on anything else. Imagine that each of the steps in this promise chain was reading a file or doing some random delay. Those are entirely unpredictable operations so which chain does what first depends upon the timing of the actual asynchronous operations, not on what is shown in this example.
So, trying to dissect the details of exactly when each item goes into the promise job queue and when it gets serviced is a complete waste of programming time and not useful in any real programming problem.
Further, if you really need a specific ordering of operations, then you don't program with two independent promise chains. Instead, you use promise flow-of-control tools (like chaining everything into one chain or using Promise.all(), Promise.race(), etc...) to guide the execution/completion order into exactly what you want it to be regardless of the detailed inner workings of the promise implementation.
Then, I would explain the basics of how the promise queue works by walking through the first two links of one of the promise chains just to show that I understand how a promise gets resolved, gets added to the promise queue and then, when control is about to return to the event loop, the oldest item in the promise queue gets to run and call its .then() or .catch() handlers. This is just to illustrated that you understand the basics of how promises are scheduled and that they get serviced in LIFO order from their own job queue and before most other things in the event loop.
Then, I would explain that a few years ago, the spec was changed for some promise steps in the interest of improving performance and a JS engine before or after that spec change would likely generate different results for something like then. Yet another reason why you shouldn't rely on that level of implementation detail.
If the interviewer insisted on me trying to explain the precise order in this problem and wouldn't listen to my reasoning why that's a pointless exercise, even after hearing my explanation for how I'd code a real world situation where execution order does matter, then unless this was just some really junior interviewer trying to be clever (and outsmarting themselves), I'd have to conclude this is not a good place to work. No senior developer should insist that this is a super valuable or practical exercise beyond showing that you have a basic understanding of how the promise job queue works.
I'm a little confused about how the sequencing works in various documents I've come across. For instance, I have seen this sort of thing
let p = Promise.resolve();
for (let x in something)
{
/* do some hairy time consuming code */
p = p.then(dosomething(x));
}
return p.then(finalthing()).catch(pretendnobadthing());
What I really don't understand is that if you do something that takes a large amount of time in the main code, wouldn't that mean that one of the promises could complete before the main code actually got round to to setting up the .then.
Can you always use .then/.catch on a promise, and if its already reached completion, it'll carry on from the point it got to, so it conceptually looks like a big chain that will run to completion at some indeterminate point? And if so, doesn't that mean every time you create a promise chain it'll hang about for ever?
Can you always use .then/.catch on a promise, and if its already reached completion, it'll carry on from the point it got to, so it conceptually looks like a big chain that will run to completion at some indeterminate point?
Yes, you can always use .then/.catch on a promise. When you call p.then(), there are three possibilities.
The promise p is still pending (not fulfilled or rejected yet). If that's the case, then the function reference(s) you passed to .then() are registered as listeners for that promise. So, when a future state transition happens on the promise (either going from pending => fulfilled or from pending => rejected), then the appropriate registered listeners will be called.
The promise p is already fulfilled. If that's the case, then calling .then(f1, f2) will schedule f1 to be called on the next tick (after the current piece of Javascript finishes executing) and it will be passed the saved resolved value.
The promise p is already rejected. If that's the case, then calling .then(f1, f2) will schedule f2 to be called on the next tick (after the current piece of Javascript finishes executing) and it will be passed the saved reject reason.
So, it's perfectly safe to call .then() on a promise that is already fulfilled or rejected. The appropriate listener will just be scheduled to run on the next tick.
The same logic applies to .catch() except it is only interested in cases 1 and 3 above.
Here are
And if so, doesn't that mean every time you create a promise chain it'll hang about for ever?
Promises are just objects like any other objects in Javascript. They will hang around only while other code still has some live reference to them. As soon as there is no longer any way to reach that promise object, they will be eligible for garbage collection just like any other objects in Javascript.
So, if you do:
var p = somePromiseReturningFunction();
p.then(f1).then(f2).then(f3);
Then, p will remain around until somePromiseReturningFunction() is done with any references to the promise that it returned (usually, though not always, this occurs when the promise is finally fulfilled or rejected) and when the variable p goes out of scope. If p never goes out of scope (like when it's global or in some other lasting scope), then it will remain forever (just like any other Javascript object).
There are some misconceptions in your question so let me attempt to square those up.
You're using the construct p = p.then(dosomething(x)); which is likely not correct. You need to pass .then() a function reference. So, unless you want doSomething(x) to execute immediately and it also returns another function that is what you want called as then .then() handler (which seems unlikely here), then this is not the right construct. You probably meant to have:
p = p.then(result => dosomething(x));
or in ES5 syntax:
p = p.then(function(result) {
return dosomething(x)
});
You show the same issue in this too:
return p.then(finalthing()).catch(pretendnobadthing());
which should probably be:
return p.then(finalthing).catch(pretendnobadthing);
Remember, when you use f(), that means to execute f immediately. When you just pass f, that passes a function reference which the underlying function/method you are passing it to can then call later at the time of its choosing which is what you want for .then() and .catch() handlers.
What I really don't understand is that if you do something that takes a large amount of time in the main code, wouldn't that mean that one of the promises could complete before the main code actually got round to to setting up the .then.
First off, my original explanation at the beginning of my answer should explain that calling .then() on an already resolved promise is perfectly fine so this isn't an issue at all. It will just schedule action on the next tick of the event loop.
But, that isn't even the case here because Javascript in the browser and node.js is single-threaded so while your long-running code is running, that promise (who's async action was previously started) can't yet get resolved. Though the underlying async operation may be done and an event may be sitting in the Javascript event queue that will trigger a callback that will resolve the promise, that event in the event queue won't get processed until the current piece of Javascript that is executing is done and returns control to the system.
What I really don't understand is that if you do something that takes a large amount of time in the main code, wouldn't that mean that one of the promises could complete before the main code actually got round to to setting up the .then.
Since common implementations of JavaScript never run it in parallel, no. Nevertheless,
Can you always use .then/.catch on a promise, and if its already reached completion, it'll carry on from the point it got to, so it conceptually looks like a big chain that will run to completion at some indeterminate point?
yes! This is a big advantage of promises. Take this for example:
var promise = Promise.resolve(5); // already resolved here
setTimeout(function () {
promise.then(function (x) {
console.log(x); // still logs 5
});
}, 1000);
And if so, doesn't that mean every time you create a promise chain it'll hang about for ever?
Until it’s resolved, yes, but if the promise has already been resolved and there are no ways to reference it anymore, it can be disposed of like any other object.
I started using Twisted in a project that require asynchronous programming and the docs are pretty good.
So my question is, is a Deferred in Twisted the same as a Promise in Javascript? If not, what are the differences?
The answer to your question is both Yes and No depending on why you're asking.
Yes:
Both a Twisted Deferred and a Javascript Promise implement a mechanism for queuing synchronous blocks of code to be run in a given order while being decoupled from other synchronous blocks of code.
No:
So Javascript's Promise is actually more similar to Python's Future, and the airy-fairy way to explain this is to talk about the Promise and the Resolver being combined to make a Deferred, and to state that this affects what you can do with the callbacks.
This is all very well and good in that it's accurate, however it doesn't really make anything any clearer, and without typing thousands of words where I'm almost guaranteed to make a mistake, I'm probably better quoting someone who knows a little something about Python.
Guido van Rossum on Deferreds:
Here's my attempt to explain Deferred's big ideas (and there are a lot
of them) to advanced Python users with no previous Twisted experience.
I also assume you have thought about asynchronous calls before. Just
to annoy Glyph, I am using a 5-star system to indicate the importance
of ideas, where 1 star is "good idea but pretty obvious" and 5 stars
is "brilliant".
I am showing a lot of code snippets, because some ideas are just best
expressed that way -- but I intentionally leave out lots of details,
and sometimes I show code that has bugs, if fixing them would reduce
understanding the idea behind the code. (I will point out such bugs.)
I am using Python 3.
Notes specifically for Glyph: (a) Consider this a draft for a blog
post. I'd be more than happy to take corrections and suggestions for
improvements. (b) This does not mean I am going to change Tulip to a
more Deferred-like model; but that's for a different thread.
Idea 1: Return a special object instead of taking a callback argument
When designing APIs that produce results asynchronously, you find that
you need a system for callbacks. Usually the first design that comes
to mind is to pass in a callback function that will be called when the
async operation is complete. I've even seen designs where if you don't
pass in a callback the operation is synchronous -- that's bad enough
I'd give it zero stars. But even the one-star version pollutes all
APIs with extra arguments that have to be passed around tediously.
Twisted's first big idea then is that it's better to return a special
object to which the caller can add a callback after receiving it. I
give this three stars because from it sprout so many of the other good
ideas. It is of course similar to the idea underlying the Futures and
Promises found in many languages and libraries, e.g. Python's
concurrent.futures (PEP 3148, closely following Java Futures, both of
which are meant for a threaded world) and now Tulip (PEP 3156, using a
similar design adapted for thread-less async operation).
Idea 2: Pass results from callback to callback
I think it's best to show some code first:
class Deferred:
def __init__(self):
self.callbacks = []
def addCallback(self, callback):
self.callbacks.append(callback) # Bug here
def callback(self, result):
for cb in self.callbacks:
result = cb(result)
The most interesting bits are the last two lines: the result of each
callback is passed to the next. This is different from how things work
in concurrent.futures and Tulip, where the result (once set) is fixed
as an attribute of the Future. Here the result can be modified by each
callback.
This enables a new pattern when one function returning a Deferred
calls another one and transforms its result, and this is what earns
this idea three stars. For example, suppose we have an async function
that reads a set of bookmarks, and we want to write an async function
that calls this and then sorts the bookmarks. Instead of inventing a
mechanism whereby one async function can wait for another (which we
will do later anyway :-), the second async function can simply add a
new callback to the Deferred returned by the first one:
def read_bookmarks_sorted():
d = read_bookmarks()
d.addCallback(sorted)
return d
The Deferred returned by this function represents a sorted list of
bookmarks. If its caller wants to print those bookmarks, it must add
another callback:
d = read_bookmarks_sorted()
d.addCallback(print)
In a world where async results are represented by Futures, this same
example would require two separate Futures: one returned by
read_bookmarks() representing the unsorted list, and a separate Future
returned by read_bookmarks_sorted() representing the sorted list.
There is one non-obvious bug in this version of the class: if
addCallback() is called after the Deferred has already fired (i.e. its
callback() method was called) then the callback added by addCallback()
will never be called. It's easy enough to fix this, but tedious, and
you can look it up in the Twisted source code. I'll carry this bug
through successive examples -- just pretend that you live in a world
where the result is never ready too soon. There are other problems
with this design too, but I'd rather call the solutions improvements
than bugfixes.
Aside: Twisted's poor choices of terminology
I don't know why, but, starting with the project's own name, Twisted
often rubs me the wrong way with its choice of names for things. For
example, I really like the guideline that class names should be nouns.
But 'Deferred' is an adjective, and not just any adjective, it's a
verb's past participle (and an overly long one at that :-). And why is
it in a module named twisted.internet?
Then there is 'callback', which is used for two related but distinct
purposes: it is the preferred term used for a function that will be
called when a result is ready, but it is also the name of the method
you call to "fire" the Deferred, i.e. set the (initial) result.
Don't get me started on the neologism/portmanteau that is 'errback',
which leads us to...
Idea 3: Integrated error handling
This idea gets only two stars (which I'm sure will disappoint many
Twisted fans) because it confused me a lot. I've also noted that the
Twisted docs have some trouble explaining how it works -- In this case
particularly I found that reading the code was more helpful than the
docs.
The basic idea is simple enough: what if the promise of firing the
Deferred with a result can't be fulfilled? When we write
d = pod_bay_doors.open()
d.addCallback(lambda _: pod.launch())
how is HAL 9000 supposed to say "I'm sorry, Dave. I'm afraid I can't
do that" ?
And even if we don't care for that answer, what should we do if one of
the callbacks raises an exception?
Twisted's solution is to bifurcate each callback into a callback and
an 'errback'. But that's not all -- in order to deal with exceptions
raised by callbacks, it also introduces a new class, 'Failure'. I'd
actually like to introduce the latter first, without introducing
errbacks:
class Failure:
def __init__(self):
self.exception = sys.exc_info()[1]
(By the way, great class name. And I mean this, I'm not being
sarcastic.)
Now we can rewrite the callback() method as follows:
def callback(self, result):
for cb in self.callbacks:
try:
result = cb(result)
except:
result = Failure()
This in itself I'd give two stars; the callback can use
isinstance(result, Failure) to tell regular results apart from
failures.
By the way, in Python 3 it might be possible to do away with the
separate Failure class encapsulating exceptions, and just use the
built-in BaseException class. From reading the comments in the code,
Twisted's Failure class mostly exists so that it can hold all the
information returned by sys.exc_info(), i.e. exception class/type,
exception instance, and traceback but in Python 3, exception objects
already hold a reference to the traceback.There is some debug stuff
that Twisted's Failure class does which standard exceptions don't, but
still, I think most reasons for introducing a separate class have been
addressed.
But let's not forget about the errbacks. We change the list of
callbacks to a list of pairs of callback functions, and we rewrite the
callback() method again, as follows:
def callback(self, result):
for (cb, eb) in self.callbacks:
if isinstance(result, Failure):
cb = eb # Use errback
try:
result = cb(result)
except:
result = Failure()
For convenience we also add an errback() method:
def errback(self, fail=None):
if fail is None:
fail = Failure()
self.callback(fail)
(The real errback() function has a few more special cases, it can be
called with either an exception or a Failure as argument, and the
Failure class takes an optional exception argument to prevent it from
using sys.exc_info(). But none of that is essential and it makes the
code snippets more complicated.)
In order to ensure that self.callbacks is a list of pairs we must also
update addCallback() (it still doesn't work right when called after
the Deferred has fired):
def addCallback(self, callback, errback=None):
if errback is None:
errback = lambda r: r
self.callbacks.append((callback, errback))
If this is called with just a callback function, the errback will be a
dummy that passes the result (i.e. a Failure instance) through
unchanged. This preserves the error condition for a subsequent error
handler. To make it easy to add an error handler without also handling
a regular resullt, we add addErrback(), as follows:
def addErrback(self, errback):
self.addCallback(lambda r: r, errback)
Here, the callback half of the pair will pass the (non-Failure) result
through unchanged to the next callback.
If you want the full motivation, read Twisted's Introduction to
Deferreds; I'll just end by noting that an errback and substitute a
regular result for a Failure just by returning a non-Failure value
(including None).
Before I move on to the next idea, let me point out that there are
more niceties in the real Deferred class. For example, you can specify
additional arguments to be passed to the callback and errback. But in
a pinch you can do this with lambdas, so I'm leaving it out, because
the extra code for doing the administration doesn't elucidate the
basic ideas.
Idea 4: Chaining Deferreds
This is a five-star idea! Sometimes it really is necessary for a
callback to wait for an additional async event before it can produce
the desired result. For example, suppose we have two basic async
operations, read_bookmarks() and sync_bookmarks(), and we want a
combined operation. If this was synchronous code, we could write:
def sync_and_read_bookmarks():
sync_bookmarks()
return read_bookmarks()
But how do we write this if all operations return Deferreds? With the
idea of chaining, we can do it as follows:
def sync_and_read_bookmarks():
d = sync_bookmarks()
d.addCallback(lambda unused_result: read_bookmarks())
return d
The lambda is needed because all callbacks are called with a result
value, but read_bookmarks() takes no arguments.
If I create a single Deferred object, then it reports progress like I would expect.
var d1 = function() {
var d = $.Deferred();
d.notify('d1');
d.resolve('d1');
return d.promise();
};
d1().progress(function(a) {
log('Progress: ' + a);
});
However, if I chain two Deferred objects using then(), the progress callbacks are not called.
d1().then(d1).progress(function(a) {
log('Progress: ' + a);
});
It seems to me that then() should propagate the progress notification, but it doesn't appear to be the case. Am I missing something?
I have tested this with jQuery 1.8.0 and 1.8.1. A full working example is here: http://jsfiddle.net/eWQuG/13/
I can see what's going on but it's slightly tricky to explain. You may have to read this through a couple of times ...
In Test 1, the promise passed to test() is the "original promise" ie. the object returned by d1(). Simple.
In Test 2, the object passed to test() is a new promise resulting from d1().then(d2).then(d3). Less simple. This new promise will have inherited only some of the original promise's properties - not including its progressCallbacks (as put in place in the original promise with notify()). This is reasonable for a promise returned by .then(), because .then() is designed to put in place its own doneCallbacks, failCallbacks and progressCallbacks, not to propagate those of the deferred/promise to its immediate left. In short, .then() does exactly what its name implies - it specifies what is to be done after its preceding deferred/promise has been processed.
You can better see what's going on by keeping a reference to the original promise (ie. the output of d1()), thus allowing the simple Test 1 to be run both before and after Test 2.
Here is a composite Test 3 :
log('-- Test 3 --');
var p = d1();//p is a promise
test(p);//equivalent to Test 1
test(p.then(d2).then(d3));//equivalent to Test 2
test(p);//equivalent to Test 1 again
DEMO
In the log, you will see that the behaviour of test(p) is identical in both cases. However, test(p.then(d2).then(d3)) behaves differently because the object passed is no longer the original promise but a new promise with no progressCallbacks.
EDIT
Ryan, something maybe worth considering in achieving what you want with :
d1().then(d2).progress(function(a) {
log('Progress: ' + a);
});
is that a the chain can be rearranged as follows :
d1().progress(function(a) {
log('Progress: ' + a);
}).then(d2);
The progress callback is thus applied to the original promise and because .progress() propagates the original promise down the chain, .then() will still act as before.
Both versions are valid but would be applicable in different circumstances.
If I'm right then maybe this is the short answer to your question.
You are doing a few weird things in your tests that are not really logical.
One, you are resolving the promise before you even return it, which defeat the purpose of promises. Promise callbacks are not supposed to be called on previously resolved promises, simple as that.
var p = $.Deferred();
p.then(function(p){log("ok")}).progress(function(p){log("progress " + p)});
p.notify('n1'); // will call handler and be logged
p.resolve();
p.notify('n2'); // will not call handler and will not be logged
Two, you are calling "notify" before you have set the progress callback, but that is somehow still working properly to a degree - it seems jQuery's particular implementation of notify is somewhat async... Except it fails with chaining. How or why, I don't know, but I don't think it is important to debug as your usage is improper anyway.
See this update to your fiddle:
http://jsfiddle.net/eWQuG/15/