We Keep Coding

How to send data from NodeJS server side to the JS client side, only when data is ready?

On my website, when the user clicks on a button, some user's data will be stored in a database and after that I want the server to send notification data to the Javascript frontend file to change the UI.
Right now, the Js file (index.js) receives data right after the website loads (always false). I want it to be received only when the data is ready on the server.
I searched a lot but couldn't find an answer to my problem?
I appreciate any help :)
server.js
var requestValidation = false;
app.post("/", function(req, res){
var name = req.body.personName;
var email = req.body.personEmail;
var collabTopic = req.body.collabTopic;
const newUser = new User({ //mongoDB schema
name: name,
email: email,
collabTopic: collabTopic
});
newUser.save(function(err){ //adding data to mongoDB
if(!err){
requestValidation = true;
}
});
});
app.get("/succ", function(req, res){
res.json(requestValidation);
});
index.js
const url = "http://localhost:3000/succ";
const getData = async (url) => {
try {
const response = await fetch(url);
const json = await response.json();
console.log(json);
} catch (error) {
console.log(error);
}
};
getData(url);

I'm not sure this is completely the answer you're looking for, but it's definitely a tool/feature to consider as you rework your approach.
app.post("/", async (req, res) => {
let result = await INSERT MONGODB UPDATE OR INSERT FUNCTION;
res.render("YOUR TEMPLATE", result);
});
You probably can't plug and play this, but when you finish a MongoDB operation, it returns a json object with some details on whether or not there was success. For example, a MongoDB insert operation returns something like this (stored in the variable result that I created)
{ "acknowledged" : true, "insertedId" : ObjectId("5fd989674e6b9ceb8665c57d") }
and then you can pass this value on as you wish.
Edit: This is what tkausl referred to in a comment.

Here is an example if you want to pass the content of a txt file to the client with express and jquery:
in express:
app.get('/get', (req, res) => {
fs.readFile('test.txt', (err, data) => {
if (err) throw err;
return res.json(JSON.parse(data));
})
})
jquery in client side:
$.getJSON( "http://localhost:3000/get", function( data ) {
geojsondata1 = JSON.stringify(data)
}
now you can do anything you want with the variable data

How to store data from a MariaDB in my node.js environment

I have a MariaDB that stores Energy-Data like voltage, frequency and so on. My aim is to visualize the data in a Web-application. Though i achieved to connect the MariaDB to node.js and log the data on a specific port thanks to the code below, i don't have a clue how to store this data for further mathematic operations or visualizations.
How can i store the data for further operations?
const express = require('express');
const pool = require('./db');
const app = express();
const port = 4999;
// expose an endpoint "persons"
app.get('/persons', async (req, res) => {
let conn;
try {
// make a connection to MariaDB
conn = await pool.getConnection();
// create a new query to fetch all records from the table
var query = "select * from Herget_Netz2_WirkleistungL1";
// run the query and set the result to a new variable
var rows = await conn.query(query);
console.log('Daten kommen');
// return the results
res.send(rows);
} catch (err) {
throw err;
} finally {
if (conn) return conn.release();
}
});
app.listen(port, () => console.log(`Listening on pfort ${port}`));

This question is quite broad.
It sounds like you need to set up a frontend and call fetch on your endpoint, something like:
fetch(<your-url>/persons)
.then(r => r.json())
.then(yourData => "<p>" + yourData "</p>")
Your data will be interpolated into HTML then. You will need to iterate over it.
The "storage" will take place in the variable you define in the second .then(yourData) of the promise for you to do further operations on.
You should search for tutorials like "set up frontend with maria db database and node backend".

retrieve data from forEach and send to front end node.js

I have this function:
app.get('/dashboard', async(req, res) => {
const customers = await stripe.customers.list();
customers.data.forEach(customer => {
console.log(customer.id);
});
// res.render('dashboard.ejs', {customer: customers})
})
what it does is returns all the customers from my Stripe db. However, now i want to pass the data returned from the loop, which is all of it lol, and access it on the front-end. How can i do this?

SQL to MongoDB Migration using nodejs script

Im new to nodejs and Im currently doing an sql to mongodb migration. I have created a script to load data to mongodb from sql queries. I created the script with the sample code from Google and it is working. But im facing below issue and need a workaround for this.
I have an sql query array and I don't need to run those queries if any of the queries has any syntax issues or any errors in the query result. (Say if the second query has syntax issue then no need to load the data of first query to mongo, currently its loading in my case). Basically if any of the query has any issue then no need to load the result in the mongo collection. And also if any issues from the mongo side no need to commit the transactions.
I have used the mongo transactions here to roll back the data if any errors. please find the below code and any help would be much appreciated.The sql and mongo credentials are mock data only.
config file code
var mongoCollection = 'collectionName';
exports.mongoCollection = mongoCollection;
var queryList = [
'sample query one',
'sample query two '
];
exports.queryList = queryList;
main script code
var MongoClient = require('mongodb').MongoClient;
var sql = require('mysql');
const config = require('./assets/config');
var sqlConfig = {
user: 'username',
password: 'password',
server: 'servername',
database: 'databasename',
port: 'portname',
multipleStatements: true
};
async function transaction() {
const mongodbUrl = 'mongourl';
const client = await MongoClient.connect(mongodbUrl, {useNewUrlParser: true}, {useUnifiedTopology:
true});
const db = client.db();
config.queryList.forEach(query => {
new sql.ConnectionPool(sqlConfig).connect().then(pool => {
return pool.request().query(query)
}).then(result => {
(async()=>{
const session = client.startSession();
session.startSession({
readConcers: {level: 'snapshot'},
writeConcern: {w: 'majority'}
});
try {
const collection = client.db('mongodbName').collection(config.mongoCollection);
await collection.insertMany(result.recordset, {session});
await session.commitTransaction();
session.emdSession();
console.log('transaction completed');
}catch(error){
await session.abortTransaction();
session.endSession();
console.log('transaction aborted');
throw error;
}
});
sql.close();
}).catch(error => {
sql.close();
throw error;
})
});
};
transaction();

Depending on the volume of data, you might look at breaking the process into two parts
Get the data from mySql
If no errors, load into Mongo
That would save you having to roll back the mongo writes
You can also take advantage of the default mongo pool size (5) and use pool on the mySQL side too.
Currently, this code is creating a pool for every select, which isn't optimal
config.queryList.forEach(query => {
new sql.ConnectionPool(sqlConfig).connect().then(pool => {//<-New pool per query?
return pool.request().query(query)
})
})
Instead, you can set up a pool once, per the mySql documentation
It looks like that driver only has a callback api, but you can promisfy the query to make it easier to work with.
So to put it all together, you could try something like this (this isn't working/tested code, just a suggestion)
var MongoClient = require('mongodb').MongoClient;
var sql = require('mysql');
const config = require('./assets/config');
var pool = sql.createPool({
connectionLimit : 5,
host : 'servername',
user : 'username',
password : 'password',
database : 'databasename'
});
async function transaction() {
try{
const mongodbUrl = 'mongourl';
const client = await MongoClient.connect(mongodbUrl, {useNewUrlParser: true}, {useUnifiedTopology: true});
const db = client.db();
const collection = client.db('mongodbName').collection(config.mongoCollection);
//Map your query list to an array of runSql promises
//this will complete when all queries return, and jump to the catch if any fail
let results = await Promise.all(config.queryList.map(runSql))
//Map the results to an array of mongo inserts
let inserts = await Promise.all(results.map(r=>collection.insertMany(r.recordset)))
//Close all connections
pool.end((err)=>err?console.err(err):console.log('MySQL Closed'))
client.close((err)=>err?console.err(err):console.log('MongoDB Closed'))
}
catch(err){
console.error(err)
}
};
transaction();
function runSql(queryStr){
return new Promise((resolve, reject)=>{
pool.query(queryStr, function (error, results, fields){
error?reject(error):resolve(results)
})
})
}
If data volume is a concern, you might want to look at getting streams from your mySql selects instead of just running them

Why doesn't push move results from DB query into variable?

Intent
Establishing a connection with a DB in my route, running a query, capturing the result, and making it accessible to my index view to render in the DOM.
Problem
DB connects and returns results successfully in the console. However, fails to push this result into the "finalResult" array.
var express = require('express');
var router = express.Router();
router.get('/', function(req, res) {
// Establish Connection with Database
var mysql = require('mysql');
var connection = mysql.createConnection({
host : '127.0.0.1',
user : 'root',
password : 'xxxxxxx',
database : 'my_db'
});
connection.connect();
// // Query and post result to console
connection.query('SELECT * FROM products', function(error, results, fields) {
if (error) throw error;
doSomething(results[0]);
});
connection.end();
var finalResult = ['hello'];
function doSomething(results) {
console.log(results.name);
finalResult.push(results.name);
}
console.log(finalResult);
// Render/make accessible to views
res.render('index', {
pageTitle: 'Home',
result: finalResult
});
});
module.exports = router;
doSomething(results[0]); successfully pushes the result into the doSomething() function because console.log(results.name) returns the correct query result. Issue occurs with finalResult.push(results.name) because console.log(finalResult) only returns an array with "hello" when it should have "hello" plus the query result.

Please note when you query database from mysql module is synchronous (the query takes function as argument -- callback). That's mean you cannot take result value as variable outside the query function. Try to use async/await or promise to get returned value.

Try to write console.log(finalResult) inside doSomething function, after you pushed the new data to finalResult array.
I think that you see only "helo" because console.log(finalResult) execute before the doSomething function finished (before you add the data to finalResult array).