Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix empty sql statement error #64

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Fix empty sql statement error #64

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
59198e5
Fix empty sql statement error
Oct 26, 2020
d2f6465
Checking in again so that the CI build can pass
Oct 27, 2020
24646cb
Removing logging statements for empty sql and reset the flag
Oct 28, 2020
6f0e2ea
Fix to return callback and not execute further lines of code
Oct 29, 2020
f09bf3e
Changed the logic to return callback if error condition is met
Oct 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions lib/connections/snowflake.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,11 +153,15 @@ connection.prototype.query = function(query, data, callback, skipTransactions){
sqlText: query,
binds: data,
complete: function(err, stmt, rows) {

isEmptySQLStatementError=false;
if (err) {
//In SQL Queries if there are empty lines between sql queries , snowflake throws a exception. This statement provides a workaround to not get that error.
if(String(err.message).includes('Empty SQL statement')) {
callback(null);
if(String(err.message).includes('Empty SQL statement')) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@manasibarate - I remember adding this statement because there were already some comments in some sql files because of which we get this error, and we should technically not make this as a failure.

Why are we then printing Stack trace for these things ?

self.book.logger.log('Error in : ' + String(query), 'error');
self.book.logger.log('Error message : ' + err.message, 'error');
self.book.logger.log('Stack Trace : ' + err.stack, 'error');
// set the flag so that callback will be called only once
isEmptySQLStatementError=true;
} else if(!(String(query).startsWith('SHOW COLUMNS') && String(query).endsWith('_TMP'))){
self.book.logger.log('Error in : ' + String(query), 'error');
self.book.logger.log('Error message : ' + err.message, 'error');
Expand All @@ -170,7 +174,13 @@ connection.prototype.query = function(query, data, callback, skipTransactions){
process.exit(1);
}
}
callback(err);
if(isEmptySQLStatementError){
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For statements where we get "Empty SQL statement" , the "isEmptySQLStatementError=true;" will always be correct , then it would never go into the "else" part , right ? Am I missing anything here ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rajpalparyani I had added the logging statements so that we have the information about the error in the logs, removed it now. Also, you are correct, this was missed, the flag would've always remained true. I have reset the flag back to false after the condition is met.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i still don't understand how this change will help us in this case. In both this PR and current master , we are just calling "callback(null)" in both cases .. The only addition is the flag , which I don't understand how it will help us not get this error.

let's talk more in the scrum as a breakout tomorrow about this.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rajpalparyani : I did some more investigation on this today, adding the if-else flag ensures that the remaining block of code doesnt execute after a callback is executed. The same can be achieved by the above changes, we want to "callback(null)" and "return" once the error condition of "empty SQL statement" is met and not continue execution till the "callback(err)" line.
This solution works too, and is the general way used even in NodeJS source. Example from filesystem module-
Notice the "return callback(...)" -->

fs.fstat(fd, function(er, st) {
if (er) return callback(er);
size = st.size;
if (size === 0) {
// the kernel lies about many files.
// Go ahead and try to read some bytes.
buffers = [];
return read();
}

buffer = new Buffer(size);
read();
});
Following link of stackoverflow summarizes the exact same issue we have-
https://stackoverflow.com/questions/19072228/javascript-callback-which-doesnt-stop-execution-of-lines-below

We can discuss more on this in the breakout.

callback(null);
}
else{
callback(err);
}

} else {
if(String(query).includes('INSERT INTO EMPUJAR') ) {
self.book.logger.log('Empujar insert , Query Id ' + stmt.getStatementId() + ' Payload Info : ' + String(data), 'debug');
Expand Down
2 changes: 1 addition & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "empujar",
"version": "0.2.23",
"version": "0.2.24" ,
"description": "When you need to push data around, you push it. Push it real good. An ETL and Operations tool.",
"main": "index.js",
"engines": {
Expand Down