-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix empty sql statement error #64
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me know if you want to setup a meeting to discuss the intention of this change .
I am having a hard time understanding how this change will help in this case.
lib/connections/snowflake.js
Outdated
if (err) { | ||
//In SQL Queries if there are empty lines between sql queries , snowflake throws a exception. This statement provides a workaround to not get that error. | ||
if(String(err.message).includes('Empty SQL statement')) { | ||
callback(null); | ||
if(String(err.message).includes('Empty SQL statement')) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@manasibarate - I remember adding this statement because there were already some comments in some sql files because of which we get this error, and we should technically not make this as a failure.
Why are we then printing Stack trace for these things ?
lib/connections/snowflake.js
Outdated
@@ -170,7 +174,13 @@ connection.prototype.query = function(query, data, callback, skipTransactions){ | |||
process.exit(1); | |||
} | |||
} | |||
callback(err); | |||
if(isEmptySQLStatementError){ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For statements where we get "Empty SQL statement" , the "isEmptySQLStatementError=true;" will always be correct , then it would never go into the "else" part , right ? Am I missing anything here ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rajpalparyani I had added the logging statements so that we have the information about the error in the logs, removed it now. Also, you are correct, this was missed, the flag would've always remained true. I have reset the flag back to false after the condition is met.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i still don't understand how this change will help us in this case. In both this PR and current master , we are just calling "callback(null)" in both cases .. The only addition is the flag , which I don't understand how it will help us not get this error.
let's talk more in the scrum as a breakout tomorrow about this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rajpalparyani : I did some more investigation on this today, adding the if-else flag ensures that the remaining block of code doesnt execute after a callback is executed. The same can be achieved by the above changes, we want to "callback(null)" and "return" once the error condition of "empty SQL statement" is met and not continue execution till the "callback(err)" line.
This solution works too, and is the general way used even in NodeJS source. Example from filesystem module-
Notice the "return callback(...)" -->
fs.fstat(fd, function(er, st) {
if (er) return callback(er);
size = st.size;
if (size === 0) {
// the kernel lies about many files.
// Go ahead and try to read some bytes.
buffers = [];
return read();
}
buffer = new Buffer(size);
read();
});
Following link of stackoverflow summarizes the exact same issue we have-
https://stackoverflow.com/questions/19072228/javascript-callback-which-doesnt-stop-execution-of-lines-below
We can discuss more on this in the breakout.
Fix to return callback and not execute further lines of code
Changed the logic to return callback if error condition is met
Problem
Error in /home/deploy/www/tr_empujar/shared/node_modules/async/lib/async.js:43
Context : If we have a query commented in the sql “transformations” , we get this error.
We need to change snowflake connector to handle this gracefully.
Solution
Added check for empty sql statement error so that process continues on this error after logging the error details to log.
Link to story
https://taskrabbit.atlassian.net/browse/DATA-1411