I have a NodeJS/Express web application that allows the user to upload a file, which I then parse using connect-busboy save to my database using Sequelize. Once that's done, I want to redirect the user to a given page. But Express is returning a status of 404 before my Promise resolves, even though I'm never calling next(), which I thought was mandatory in order to call the next handler in the middleware chain and thus result in a 404.
This is my code so far:
function uploadFormFile(req, res, next) {
var documentInstanceID = req.params.documentInstanceID;
// set up an object to hold my data
var data = {
file: null,
documentDate: null,
mimeType: null
};
// call the busboy middleware explicitly
// EDIT: this turned out to be the problem... of course this calls next()
// removing this line and moving it to an app.use() made everything work as expected
busboy(req, res, next);
req.pipe(req.busboy);
req.busboy.on('file', function (fieldName, file, fileName, encoding, mimeType) {
var fileData = [];
data.mimeType = mimeType;
file.on('data', function (chunk) {
fileData.push(chunk);
});
file.on('end', function () {
data.file = Buffer.concat(fileData);
});
});
req.busboy.on('finish', function () {
// api methods return promises from Sequelize
api.querySingle('DocumentInstance', ['Definition'], null, { DocumentInstanceID: documentInstanceID })
.then(function (documentInstance) {
documentInstance.RawFileData = data.file;
documentInstance.FileMimeType = data.mimeType;
// chaining promise
return api.save(documentInstance);
}).then(function () {
res.redirect('/app/page');
});
});
}
I can confirm that my data is being persisted correctly. But due to the race condition, the web page says 'can't POST' due to the 404 status being returned by Express, and the res.redirect is failing with an error setting the headers because it's trying to redirect after the 404 has been sent.
Can anyone help me figure out why Express is returning the 404?
The problem is coming from your internal call to busboy inside your handler. Rather than it executing and simply returning control to your handler, it would be calling the next which is passed to it before it returns control. So you code after the busboy call does execute, but the request has already advanced past that point.
In cases in which you want some middleware to only be executed for certain requests, you can chain middleware into those requests, such as:
router.post('/upload',busboy,uploadFromFile)
You can also separate them with .use() such as:
router.use('/upload', busboy);
router.post('/upload', uploadFromFile);
Either of the above will chain the middleware in the way you intended. In the case of .use() the middleware would also be applied to any applicable .METHOD() as Express refers to it in their documentation.
Also, note that you can pass in an arbitrary number of middleware this way, either as separate parameters or as arrays of middleware functions, such as:
router.post('/example', preflightCheck, logSomeStuff, theMainHandler);
// or
router.post('example', [ preflightCheck,logSomeStuff ], theMainHandler);
The execution behavior of either of the above examples will be equivalent. Speaking only for myself and not suggesting it is a best practice, I normally only use the array-based addition of middleware if i am building the middleware list at runtime.
Good luck with it. I hope you enjoy using Express as much as I have.
Related
I have an isomorphic react app and I would like to somehow pass state between express middleware.
I have the following express route that handles form submission:
export const createPaymentHandler = async (req: Request, res: Response, next: NextFunction) => {
const { field } = req.body;
if (!paymentType) {
res.locals.syncErrors = { field: 'some error.' };
next();
return;
}
try {
const { redirectUrl } = await makeRequest<CreatePaymentRequest, CreatePaymentResponse>({
body: { paymentType },
method: HttpMethod.POST
});
res.redirect(redirectUrl);
} catch (err) {
error(err);
res.locals.serverError = true;
next();
}
};
The next middleware is handling the rendering.
At the moment I am using res.locals, is there a better way or a recognised pattern?
IMO your question is more about passing some data to the next middleware. Since the rendering logic is handled by the next middleware, the express route shouldn't be concerned by how the data is being used. Your approach looks fine.
res.locals is the recommended way of passing data to the next middleware. From the docs:
This property is useful for exposing request-level information such as the request path name, authenticated user, user settings, and so on.
Also, since the variables added will be scoped to the current request, thus the data will only be available for the current request's lifecycle. Perhaps you can set a convention of adding a state key on the res.locals to store all your state variables, but the current approach would also work fine.
res.locals is a standard way to pass data to the next middleware in the scope of the current request. Since your use case is around the current request, it makes sense to do so.
At the same time, the standard way to handle errors is to pass the error to the next middleware.
next(err);
Then you can handle the error scenario from the error handler. However, for an isomorphic react app, this would make things harder. So if you decide to go down that path, I would suggest you to use a custom error like PaymentError by extending Error. This would make even more sense since you are already using Typescript.
However, when you actually think about this scenario, when the error is not a request error, from the point of view of the react app, it is a special state/property of rendering. Thus I suggest the following hybrid approach.
If the error is of high priority, that is, if the error should stop rendering the expected content and fallback to a special page, use the next(err) approach.
If the error should just be part of the state report, then use the res.locals approach.
Because your handler is async, you need to pass the err into next, like so:
next(err);
In order for your middleware to process the error, instead of it being picked up by the default error handler, you need to have four parameters:
app.use((err, req, res, next) => {
// handle the error
})
It's also worth noting that error handlers need to be specified after other middleware. For your case, it might make sense to use a normal "success" middleware alongside an error handler, rather than combining the two into one middleware.
Finally, keep in mind that passing err as a parameter is specific to error handlers. If you just want to pass some data into your next middleware, you would do that by modifying the req:
req.x = 'some data'
next()
Then, the next middleware's req parameter will have the data you set.
Further reading: https://expressjs.com/en/guide/using-middleware.html#middleware.error-handling
If it's passing lightweight information to the next middleware for rendering purposes then applying res.locals is fine. However, you might want to look into custom error-handling for general errors, such as internal error.
Consider the following error handling
function notFoundHandler(req, res, next) {
res.status(404).render('notFoundPage', {
error: '404 - not found'
});
}
function badRequestHandler(err, req, res, next) {
res.status(400).render('badRequestPage', {
error: 'Bad request'
});
}
function errorHandler(err, req, res, next) {
res.status(err.status || 500).render('errorPage', {
error: 'Internal server error'
});
}
app.use(notFoundHandler);
app.use(badRequestHandler);
app.use(errorHandler);
Now instead of passing error details to the next middleware you would simple let it flow to the error handlers, e.g.
export const createPaymentHandler = async (req: Request, res: Response, next:
NextFunction) => {
const { field } = req.body;
if (!paymentType) {
res.status(400);
return next(); // This will hit the Bad Request handler
}
try {
const { redirectUrl } = await makeRequest < CreatePaymentRequest, CreatePaymentResponse > ({
body: { paymentType },
method: HttpMethod.POST
});
res.redirect(redirectUrl);
} catch (err) {
res.status(500);
return next(err); // This will hit the Error Handler
}
};
The best way to pass a state between express middleware is to use the object res.locals what you already do.
You are on the correct and best way!
May be you have to look to the documentation from res.locals one time again:
Citate from documentation of res.locals
res.locals – an object that contains response local variables scoped to the
request, and therefore available only to the view(s) rendered during
that request / response cycle (if any). Otherwise, this property is
identical to app.locals.
This property is useful for exposing request-level information such as
the request path name, authenticated user, user settings, and so on.
app.use(function(req, res, next)
{
res.locals.user = req.user;
res.locals.authenticated = ! req.user.anonymous;
next();
});
And you can see – they recommend to use this object.
You are on the right way!
I'm trying to serve 500 pages (some generic HTML that says "500 - internal server error") from my Node.js server to requests that failed to resolve due to developer bugs, but can't find an elegant way to do this.
Lets say we have the following index.js, where a developer innocently made a mistake:
const http = require('http');
const port = 12345;
http.createServer(onHttpRequest).listen(port);
function onHttpRequest(req, res) {
var a = null;
var b = a.c; // this is the mistake
res.end('status: 200');
}
Trying to access property "c" of null throws an error, so "res.end" will never be reached. As a result, the requesting client will eventually get a timeout. Ideally, I my server to have code that can catch errors like this, and return 500 pages to the requesting client (as well as email an administrator and so on).
Using "try catch" in every single block is out of the question. Most Node.js code is async, and a lot of the code relies on external libraries with questionable error handling. Even if I use try-catch everywhere, there's a chance that an error would happen in an external library that didn't have a try-catch block inside of it, in a function that happens asynchronously, and thus my server will crash and the client would never get a response.
Shortest example I can provide:
/* my server's index.js */
const http = require('http');
const poorlyTestedNpmModule = require('some-npm-module');
const port = 12345;
http.createServer(onHttpRequest).listen(port);
function onHttpRequest(req, res) {
try {
poorlyTestedNpmModule(null, onResult);
}
catch(err) {
res.end('status: 500');
}
function onResult(err, expectedResult) {
if(err) {
res.end('status: 400');
}
else {
res.end('status: 200');
}
}
}
/* some-npm-module.js */
module.exports = function poorlyTestedNpmModule(options, callback) {
setTimeout(afterSomething, 100);
function afterSomething() {
var someValue = options.key; // here's the problem
callback(null, someValue);
}
}
Here, the server crashes, due to a function call that led to code that asynchronously throws an error. This code is not code that I control or wish to modify; I want my server to be able to handle all those errors on its own.
Now, I could, for instance, just use the global uncaughtException event, i.e.:
process.on('uncaughtException', doSomething);
but then I have no access to the (req, res) arguments, making it impossible to call res.end for the correct res instance; the only way to have access to them, is to store them in a higher-scope object for each incoming request, and then prune them on successful request resolutions, then mark existing [req, res] stored pairs as "potentially errored" whenever an uncaughtException triggers, and serve 500 pages to those requests whenever the count of currently-active requests matches the count of currently-unresolved-errors (and re-test that count per thrown uncaught expection and per successful res.end call).
Doing that works, but... it's ugly as hell. It means that request objects have to be leaked to the global scope, and it also means that my router module now has a dependency on the uncaughtException global event, and if any other code overwrites that event, everything breaks, or if I ever want to handle other uncaught exceptions for whatever reason, I'll run into cross dependency hell.
The root cause of this problem is that an unexpected error can happen anywhere, but I want to specifically catch whether an unexpected error originated from a stack trace that began from an incoming http request (and not, for example, from some interval I have running in the background, because then I get an unexpected error but obviously don't want to serve a 500 page to anyone, only email an admin with an error log), and on top of needing to know whether the error originated from an http request, I need to have access to the request+response objects that node server objects provide.
Is there no better way?
[Edit] The topic of this question is role distribution in modules.
i.e., one guy is making base code for a server, lets say a "router module". Other people will add new code to the server in the future, handling branches that are routed to.
The guy that writes the base server code has to write it in a way that it will serve 500 pages if any future code is written incorrectly and throws errors. Help him accomplish his goal.
Answers of the format "make sure all future people that add code never make mistakes and always write code that won't throw uncaught errors" will not be accepted.
At first, using uncaughtException in Nodejs is not safe. If you feel that there is no other option in your application, make sure that you exit the process in the handler of 'uncaughtException' and restart the process using pm2 or forever or someother modules. Below link can provide you its reference.
Catch all uncaughtException for Node js app
Coming to the process of error handling, as mentioned, you may always miss to handle errors with callback. To avoid, these we can use an exceptional advantage of promises in nodejs.
/* my server's index.js */
const http = require('http');
const poorlyTestedNpmModule = require('some-npm-module');
const port = 12345;
http.createServer(onHttpRequest).listen(port);
function onHttpRequest(req, res) {
try {
poorlyTestedNpmModule(null)
.then(result => {
res.end('status: 200');
})
.catch(err =>{
console.log('err is', err);
res.end('status: 400');
})
}
catch(err) {
res.end('status: 500');
}
}
/* some-npm-module.js */
module.exports = function poorlyTestedNpmModule(options, callback) {
setTimeout(afterSomething, 100);
afterSomthing = new Promise((resolve, reject)=> {
var someValue = options.key; // here's the problem
resolve(someValue);
})
}
If you see that some of the npm nodemodules are not present with promise, try to write wrappers to convert callback to promise model and use them in your application.
I'm just really new on Node and Express. Trying to pass a function instead of text on my route but it seems not working. I just looked up at documentation there, They mentioned only text with req.send() method. I'm trying to pass here function's but it's not working. and also the alert() not working like this req.send(alert('Hello world')) it say's alert isn't defined or something similar.
**Update: ** I'm trying to execute this library with express and node https://github.com/przemyslawpluta/node-youtube-dl
I'm trying to do here pass functions like this
function blaBla() {
var youtubedl = require('youtube-dl');
var url = 'http://www.youtube.com/watch?v=WKsjaOqDXgg';
// Optional arguments passed to youtube-dl.
var options = ['--username=user', '--password=hunter2'];
youtubedl.getInfo(url, options, function(err, info) {
if (err) throw err;
console.log('id:', info.id);
console.log('title:', info.title);
console.log('url:', info.url);
console.log('thumbnail:', info.thumbnail);
console.log('description:', info.description);
console.log('filename:', info._filename);
console.log('format id:', info.format_id);
});
}
app.get('/', (req, res) => {
res.send(blaBla());
})
**Instead of **
app.get('/', function (req, res) {
res.send('Hello World!')
})
I hope you guy's understood my question.
res.send() expects a string argument. So, you have to pass a string.
If you want the browser to execute some Javascript, then what you send depends upon what kind of request is coming in from the browser.
If it's a browser page load request, then the browser expects an HTML response and you need to send an HTML page string back. If you want to execute Javascript as part of that HTML page, then you can embed a <script> tag inside the page and then include Javascript text inside that <script> tag and the browser will execute that Javascript when the page is parsed and scripts are run.
If the route is in response to a script tag request, then you can return Javascript text as a string and you need to make sure the MIME type appropriately indicates that it is a script.
If the route is in response to an Ajax call, then it all depends upon what the caller of the Ajax call expects. If they expect a script and are going to execute the text as Javascript, then you can also just send Javascript text as a string. If they expect HTML and are going to process it as HTML, then you probably need to embed the <script> tag inside that HTML in order to get the Javascript executed.
In your example of:
response.send(blaBla());
That will work just fine if blaBla() synchronously returns a string that is formatted properly per the above comments about what the caller is expecting. If you want further help with that, then you need to show or describe for us how the request is initiated in the browser and show us the code for the blaBla() function because the issue is probably in the blaBla() function.
There are lots of issues with things you have in your question:
You show req.send(alert('Hello world')) in the text of your question. The .send() method belongs to the res object, not the req object (the second argument, not the first). So, that would be res.send(), not req.send().
In that same piece of code, there is no alert() function in node.js, but you are trying to execute it immediately and send the result with .send(). That won't work for a bunch of reasons.
Your first code block using blaBla() will work just fine as long as blaBla() returns a string of the right format that matches what the caller expects. If that doesn't work, then there's a problem with what blaBla() is doing so we need to see that code.
Your second code block works because you are send a string which is something the caller is equipped to handle.
Update now that you've shown the code for blaBla().
Your code for blaBla() does not return anything and it's asynchronous so it can't return the result. Thus, you cannot use the structure response.send(blaBla());. There is no way to make that work.
Instead, you will need to do something different like:
blaBla(response);
And, then modify blaBla() to call response.send(someTextValue) when the response string is known.
function blaBla(res) {
var youtubedl = require('youtube-dl');
var url = 'http://www.youtube.com/watch?v=WKsjaOqDXgg';
// Optional arguments passed to youtube-dl.
var options = ['--username=user', '--password=hunter2'];
youtubedl.getInfo(url, options, function(err, info) {
if (err) {
res.status(500).send("Internal Error");
} else {
console.log('id:', info.id);
console.log('title:', info.title);
console.log('url:', info.url);
console.log('thumbnail:', info.thumbnail);
console.log('description:', info.description);
console.log('filename:', info._filename);
console.log('format id:', info.format_id);
// construct your response here as a string
res.json(info);
}
});
}
Note also that the error handling does not use throw because that is really not useful inside an async callback.
No one just could help me with that and after finding things are alone I got to know how to do this. In express there is something called middleware we have to use that thing to get this kind of matter done. Those who are really expert or have working experience with express they know this thing.
to using functions with express you need to use middleware.
like below I'm showing
const express = require('express')
const youtubedl = require('youtube-dl');
const url = 'https://www.youtube.com/watch?v=quQQDGvEP10';
const app = express()
const port = 3000
function blaBla(req, res, next) {
youtubedl.getInfo(url, function(err, info) {
console.log('id:', info.id);
console.log('title:', info.title);
console.log('url:', info.url);
// console.log('thumbnail:', info.thumbnail);
// console.log('description:', info.description);
console.log('filename:', info._filename);
console.log('format id:', info.format_id);
});
next();
}
app.use(blaBla);
app.get('/', (request, response) => {
response.send('Hey Bebs, what is going on here?');
})
app.listen(port, (err) => {
if (err) {
return console.log('something bad happened', err)
}
console.log(`server is listening on ${port}`)
})
And remember that you must need to use app.use(blaBla); on top of getting your route. Otherwise this might not work.
I am working with node/express/passport/ looking at code that attempts to use a request like:
req._parsedUrl.pathname;
I cannot figure out where this variable is coming from. Is this a canonical variable name that is set in a common .js library? It doesn't seem exposed in any headers.
req._parsedUrl is created by the parseurl library which is used by Express' Router when handling an incoming request.
The Router doesn't actually intend to create req._parsedUrl. Instead parseurl creates the variable as a form of optimization through caching.
If you want to use req._parsedUrl.pathname do the following instead in order to ensure that your server doesn't crash if req._parsedUrl is missing:
var parseUrl = require('parseurl');
function yourMiddleware(req, res, next) {
var pathname = parseUrl(req).pathname;
// Do your thing with pathname
}
parseurl will return req._parsedUrl if it already exists or if not it does the parsing for the first time. Now you get the pathname in a save way while still not parsing the url more than once.
You can write a middleware to handle then set properties for req.
var myMiddleWare = function () {
return function (req, res, next) {
req._parsedUrl = 'SOME_THING';
next()
}
};
app.get('/', myMiddleWare, function (req, res) {
console.log(req._parsedUrl); // SOME_THING
res.end();
})
Express middleware document in here
Coming from a .net world where synchronicity is a given I can query my data from a back end source such as a database, lucene, or even another API, I'm having a trouble finding a good sample of this for node.js where async is the norm.
The issue I'm having is that a client is making an API call to my hapi server, and from there I need to take in the parameters and form an Elasticsearch query to call, using the request library, and then wait for the instance to return before populating my view and sending it back to the client, problem being is that the request library uses a callback once the data is returned, and the empty view has long been returned to the client by then.
Attempting to place the return within the call back doesn't work since the EOF for the javascript was already hit and null returned in it's place, what is the best way to retrieve data within a service call?
EX:
var request = require('request');
var options = {
url: 'localhost:9200',
path: {params},
body: {
{params}
}
}
request.get(options, function(error, response){
// do data manipulation and set view data
}
// generate the view and return the view to be sent back to client
Wrap request call in your hapi handler by nesting callbacks so that the async tasks execute in the correct logic order. Pseudo hapi handler code is as following
function (request, reply) {
Elasticsearch.query((err, results) => {
if (err) {
return reply('Error occurred getting info from Elasticsearch')
}
//data is available for view
});
}
As I said earlier in your last question, use hapi's pre handlers to help you do async tasks before replying to your client. See docs here for more info. Also use wreck instead of request it is more robust and simpler to use