Importing text file contents to database - javascript

I'm working with sensor units outfield that spit out data in the form of native text files and have a database created with pre-defined tags. E.G mm, level, voltage. The text file I'm using is pretty unstructured with the data being on the right side of the header separated by semicolon delimiters.
I want to try and import the content into the database where each header matches the tag name and the values are inserted into that tag consecutively. I'm wondering if there's a possible solution or maybe even some tips on how i can achieve this?
Currently i have been working with PHP but haven't gotten to far, is it the best language to use for such a method? Or would javascript be preferred?
Text file is delimited by semicolons:
L;MINVi;Min voltage;V;PTi;Processor temperature;C;AVGVi;Average voltage;V;SDB;Network signal dB;dB;WL02;waterlevel 2m;cm;RSSI;Network signal indication;RSSI;OCi;Operating cycle;sec;SCNT;Satellites;;LAT;Latitude;deg;LON;Longitude;deg
S;170427000428;ERR;SERVER_LOGIN;+CME ERROR: Bad or no response from server
S;170427000428;ERR;FTP
S;170427000450;ALARM_SEND_OK
S;170427000510;WDT;GPS
D;170427000510;SCNT;0*T;LAT;0*T;LON;0*T
S;170427000518;ERR;SERVER_LOGIN;+CME ERROR: Bad or no response from server
S;170427000518;ERR;FTP
S;170427000647;ERR;SERVER_LOGIN;+CME ERROR: Bad or no response from server
S;170502171807;POWER_ON;ML-315;V2.7B1
S;170502171807;SYS_START;BHSDemo 5170991
D;170502171817;MINVi;3.66;PTi;25.8;AVGVi;3.71;WL02;2.86*A;OCi;9.95
S;170502171822;WDT;MODEM_INIT
D;170502171823;SDB;0*T;RSSI;0*T
S;170502171823;WDT;Network signal
database table Tag_data Structure

You can do like this
LOAD DATA INFILE '/yourtext.txt' INTO TABLE TABLENAME;

Your text will be pretty hard to explode every value with it's parameter because every single word end with ; so, first try to make your text file like parameter:value; or parameter=value; or parameter_value; then you can read the content of this file with this php function $content = file_get_contents("path/text_file_name.txt"); now the value of $content variable is equal to the entire text of your file hence, you can split this variable into an array with $splittedcontent = explode(";" , $content); every parameter:value is a parameter in this array like $splittedcontent[0] = parameter0:value0, $splittedcontent[1] = parameter1:value1 and so on, now you can use for loop throw this array and make what your need in you database..
I hope this will help you.

Related

How to create a function that returns an array of image paths in javascript?

How can I create a function that loops through a folder in my directory called "data." It contains only image files, and I will keep adding more image files to it. Previously, I was using the following function that returns an array of URLs:
function _image_urls(){return(
[
"https://images.pexels.com/photos/4050284/pexels-photo-4050284.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/1323550/pexels-photo-1323550.jpeg?auto=compress&cs=tinysrgb&w=600",
"https://images.pexels.com/photos/2002719/pexels-photo-2002719.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/919606/pexels-photo-919606.jpeg?auto=compress&cs=tinysrgb&w=600",
"https://images.pexels.com/photos/1983038/pexels-photo-1983038.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/1702624/pexels-photo-1702624.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/3631430/pexels-photo-3631430.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/5011647/pexels-photo-5011647.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/135018/pexels-photo-135018.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1",
"https://images.pexels.com/photos/161154/stained-glass-spiral-circle-pattern-161154.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1"
]
)}
I'm trying to create a function that returns an array of paths for all the images in the data folder. I have been trying the following approach:
function _image_urls(){
const image_folder = 'data';
const image_extension = '.jpg';
let image_urls = [];
for (let i = 0; i < 10; i++) {
let image_url = image_folder + i + image_extension;
image_urls.push(image_url);
}
return image_urls;
}
It seems like this will just return an array like:
[
"data0.jpg",
"data1.jpg",
"data2.jpg",
"data3.jpg",
"data4.jpg",
"data5.jpg",
"data6.jpg",
"data7.jpg",
"data8.jpg",
"data9.jpg"
]
If that's what you're getting, then you need to use i as the index for an array that contains the file names.
The bigger question is how are you getting that list of files in the first place? This is generally not something that JavaScript can do on its own. If the files exist on the server, you'd need some server-side script to actually access the folder and output the array of file names - this can then be put into an array several ways (either writing it directly to the code if you allow your server side code to process the JS file or probably more likely using an XHR to request the file names and then populate the array when you get the response.)
If you write this server side script such that it formats the output as JSON, then it could simply be a matter of using JSON.parse() to convert the output to an array directly without any need to iterate over the response such as the function in the question.
EDIT/UPDATE after comment from OP:
Since you're using PHP on the server side, I would create a server side script that readers the contents of the "Data" folder and outputs a JSON formatted string which can then be parsed by the JS on the front end.
In general, this is done using the scandir function. See https://www.php.net/manual/en/function.scandir.php for details.
and the steps would be as follows:
Use scandir to get an array of files in the Data folder
Remove the first two items in the array (. and ..)
Use the json_encode function to convert the array to a JSON formatted string
Echo that string
Then on the page where you have your JS you have two options:
Include the PHP script described above such that it becomes a JS array using JSON.parse().
Use an XHR to request the PHP script, and when you get a response use JSON.parse() to set it as an array variable.
The first method is outdated, but very simple - though it does require that your JS code is parsed by PHP which may or may not be possible/advisable depending on your server configuration.
The second method is probably what you should do, as long as you're fine with the array being populated after the page loads and that you wait for the XHR to complete before calling any functions that rely on the array.
The main thing to know here is that what you want to do is not possible using only JavaScript because JS cannot read the contents of a folder on the server. Your JS will need to interact with some server side code in order to read the contents of a folder into any array.

How to put fully string content in js

so i have this block of sql query code here for sending user description to db in node js.
const sqlAddDescription = (desc, id) => {return `UPDATE \`Memcon\`.\`users_list\` SET \`description\` = '${desc}' WHERE (\`id\` = '${id}')`}
it's working completely fine, in the client user inputs a text and the db process goes on with no problem.
BUT if the user sends an input with backticks or quotes or even brackets, the process gonna fail because their text head on goes to ${desc} and it replaces it, so it creates an error.
How can i tell js to fully stringify the text, no matter the inputs.
(i also tried JSON.stringify but that serves a different purpose )

How to display 100K XML based DOM data on a JSP client page?

I need some help in understanding what to do next.
I need to write a web based search function to find medical records from an XML file.
The operator can enter either part or all of a patient name and Hit Search on the JSP web page.
The server is suppose to then return a list of possible patient names with the opportunity for the operator to go to next page until a possible patient is found. They can then select the person and view more details.
On the Server side
I have an XML file with about 100,000 records. There are five different types of records in the file. (This is roughly about 20,000 x 5 = 100,000).
I have a java class to source the xml file and create a DOM to traverse the data elements found on the file.
-- XML File Begin
100k - XML file outline
<hospital>
<infant key="infant/0002DC15" diagtype="general entry" mdate="2015-02-18">
<patient>James Holt</patient>
<physician>Michael Cheng</physician>
<physician>David Long</physician>
<diagnosisCode>IDC9</diagnosisCode>
..
</infant>
<injury key="injury/0002IC15" diagtype="general entry" mdate="2015-03-14">
<patient>Sara Lee</patient>
<physician>Michael Cheng</physician>
<diagnosisCode>IEC9</diagnosisCode>
..
</injury>
<terminal key="terminal/00X2IC15" diagtype="terminal entry" mdate="2015-05-14">
<patient>Jason Man</patient>
<physician>John Hoskin</physician>
<diagnosisCode>FEC9</diagnosisCode>
<diagnosisCode>FXC9</diagnosisCode>
..
</terminal>
<aged key= xxxx ... >
...
</aged>
<sickness key= xxxx ... >
...
</sickness>
</hospital>
approx 5 ( )x 20,000 = 100K records.
Key and patient are the only mandatory fields. The rest of the elements are Optional or multiple elements.
-- XML File End
Here is where I need help
Once I have the DOM how do I go forward in letting the client know what was found in the XML file?
Do I create a MAP to hold the element node links and then forward say 50 links at a time to the JSP and then wait to send some more links when the user hits next page?
Is there an automated way of displaying the links, either via a Java Script, Jquery, XSLT or do I just create a table in HTML and place patient links inside the rows? Is there some rendering specific thing I have to do in order to display the data depending on the browser used by client?
Any guidance, tutorials, examples or books I can refer to would be greatly appreciated.
Thank you.
I don't know an automatic way to match the type in jQuery, but you can test the attributes, something like verify if a non optional attribute in the object is present:
// Non optional Infant attribute
if(obj.nonOptionalAttribute) {
// handle Infant object
}
Or you may add an attribute to differentiate the types (something like a String or int attribute to test in your Javascript).
if(obj.type == 'infant') {
// handle Infant object
}
#John.west,
You can try to bind the XML to a list of objects (something like Injure implements MyXmlNodeMapping, Terminal implements MyXmlNodeMapping, Infant implements MyXmlNodeMapping and go on and have a List) to iterate and search by the value at the back end or you can pass this XML file to a Javascript (if you are using jQuery you can use a get or a post defining the result type as XML) and iterate over the objects to find what the user is trying to find...
Your choice may be based on the preference to use processor time in the server side or in the client side...

Getting all of the .json files from a directory

I'm creating an android app which takes in some json data, is there a way to set up a directory such as;
http://......./jsons/*.json
Alternatively, a way to add into a json file called a.json, and extend its number of containing array data, pretty much add more data into the .json file this increase its size.
It could be by PHP or Javascript.
Look into Parsing JSON, you can use the JSON.parse() function, in addition, I'm not sure about getting all your JSON files from a directory call, maybe someone else will explain that.
var data ='{"name":"Ray Wlison",
"position":"Staff Author",
"courses":[
"JavaScript & Ajax",
"Buildinf Facebook Apps"]}';
var info = JSON.parse(data);
//var infostoring = JSON.stringify(info);
One way to add to a json file is to parse it, add to it, then save it again. This might not be optimal if you have large amounts of data but in that case you'll probably want a proper database anyway (like mongo).
Using PHP:
$json_data = json_decode(file_get_contents('a.json'));
array_push($json_data, 'some value');
file_put_contents('a.json', json_encode($json_data));

JQuery and JSON

Here's something I want to learn and do. I have a JSON file that contains my product and details (size, color, description). In the website I can't use PHP and MySQL, I can only use Javascript and HTML. Now what I want to happen is using JQuery I can read and write a JSON file (JSON file will serve as my database). I am not sure if it can be done using only JQuery and JSON.
First thing, How to query a JSON file? (Example: I would search for the name and color of the product.)
How to parse the JSON datas that were searched into an HTML?
How to add details, product to the JSON file?
It will also be great if you can point me to a good tutorial about my questions.
I'm new to both JQuery and JSON.
Thanks!
Since Javascript is client side, you won't be able to write to the JSON file on the server using only Javascript. You would need some server side code in order to do that.
Reading and parsing the JSON file is not a problem though. You would use the jQuery.getJSON function. You would supply both a url and a callback parameter (data isn't needed, because you're reading a file, so no need to send data). The url would be the path to your JSON file, and the callback would be a function that uses the data.
Here's an example of what your code might look like. I don't know exactly what your JSON is, but if you have a set called "products" containing a set of objects with the details "name" and "price", this code would print those out:
$.getJSON("getProductJSON.htm",
function(data) {
$.each(data.products, function(i, item) {
var name = item.name;
var price = item.price;
// now display the name and price on the page here!
});
},
);
Basically, the data variable in $.getJSON makes the entire contents of the JSON available to you, very easily. And the $.each is used to loop over a set of JSON objects.

Categories