SNOW-802269-Jsontuple-cbrt-fromjson-datesub-trycast#161
SNOW-802269-Jsontuple-cbrt-fromjson-datesub-trycast#161sfc-gh-sjayabalan wants to merge 6 commits intomainfrom
Conversation
| * The column argument must be a string column in Snowflake. | ||
| */ | ||
| def try_cast(e: Column, targetType: DataType): Column = { | ||
| e.cast(targetType) |
There was a problem hiding this comment.
Do you have tests cases to validate that this returns null if cast fails? I think this might fail and that you might need to use try_cast built int
| } | ||
|
|
||
| @Test | ||
| public void json_tuple1() { |
There was a problem hiding this comment.
use a more representative test name like json_tuple_
maybe json_tuple_single_parameter ? for example
| } | ||
|
|
||
| @Test | ||
| public void json_tuple2() { |
There was a problem hiding this comment.
Same as with preivous
| } | ||
|
|
||
| @Test | ||
| public void try_cast() { |
There was a problem hiding this comment.
need more test cases. You need to test several combinarions
df = spark.createDataFrame(
[(2, "123"), (5, "Bob"), (3, None)], ["age", "name"])
df.select(df.name.try_cast(LongType())).show()
+----+
|name|
+----+
| 123|
|NULL|
|NULL|
+----+
Structure types should be consider as well:
df.filter(F.array_contains(F.lit('Key'), F.col('KEY_UNSTRCUTURED_ARRAY ').cast('ARRAY(VARCHAR)')).show()
or
df.filter(F.array_contains(F.lit('Key'), F.col('KEY_UNSTRCUTURED_ARRAY ').cast(T.ArraryType(T.StringType()))).show()
|
|
||
| @Test | ||
| public void date_sub() { | ||
| DataFrame df = |
There was a problem hiding this comment.
We need more tests, what happens with null, what happens with large values
|
|
||
| @Test | ||
| public void from_json() { | ||
| DataFrame df = |
There was a problem hiding this comment.
positive an negative tests needed.
3 participants
Please answer these questions before submitting your pull requests. Thanks!
Added following functions in java and scala.
a. json_tuple()
b.from_json()
c.date_sub()
d.try_cast()
e.cbrt()
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
Added following functions as wrapper to java and scala functions.
Pre-review checklist
(For Snowflake employees)