some more basic examples with tablecloth

Author: kirahowe

book/chapter_2_input_output/2_1_loading_data.clj

@@ -1,4 +1,6 @@
-(ns book.chapter-2-input-output.2-1-loading-data)
+(ns book.chapter-2-input-output.2-1-loading-data
+  (:require
+    [tablecloth.api :as tc]))
 
 ;; # 2.1 How to get data into the notebook
 
@@ -21,6 +23,7 @@
 ;; TODO: Link to useful explainer on lazy seqs
 
 ;; #### With tablecloth
+
 ;; For most work involving tabular/columnar data, you'll use tablecloth, Clojure's go-to data
 ;; wrangling library. These all return a `tech.ml.dataset Dataset` object. The implementation
 ;; details aren't important now, but `tech.ml.dataset` is the library that allows for efficient
@@ -69,8 +72,57 @@
 
 ;; ##### Specify file encoding
 
-;;
+;; TODO: does this really matter? test out different file encodings..
 
 ;; ##### Normalize values into consistent formats and types
 
-;; Tablecloth makes it easy to apply arbitrary transformations to all values in a given column:
+;; Tablecloth makes it easy to apply arbitrary transformations to all values in a given column
+
+;; We can inspect the column metadata with tablecloth:
+
+(-> dataset
+    (tc/info :columns))
+
+;; Certain types are built-in (it knows what to do convert them, e.g. numbers:)
+
+(-> dataset
+    (tc/convert-types "CO2" :double)
+    (tc/info :columns))
+
+;; The full list of magic symbols representing types tablecloth supports comes from the underlying
+;; `tech.ml.dataset` library:
+(require '[tech.v3.datatype.casting :as casting])
+@casting/valid-datatype-set
+
+;; More details on [supported types here](https://github.com/techascent/tech.ml.dataset/blob/master/topics/supported-datatypes.md).
+
+;; You can also process multiple columns at once, either by specifying a map of columns to data types:
+
+(-> dataset
+    (tc/convert-types {"CO2" :double
+                       "adjusted CO2" :double})
+    (tc/info :columns))
+
+;; Or by changing all columns of a certain type to another:
+
+(-> dataset
+    (tc/convert-types :type/numerical :double)
+    (tc/info :columns))
+
+;; The supported types of columns are:
+
+;; :type/numerical - any numerical type
+;; :type/float - floating point number (:float32 and :float64)
+;; :type/integer - any integer
+;; :type/datetime - any datetime type
+
+;; Also the magical `:!type` qualifier exists, which will select the complement set -- all columns that
+;; are _not_ the specified type
+
+;; For others you need to provide a casting function yourself, e.g. parsing strings:
+(-> dataset
+    ;; (tc/convert-types "Date" :local-date-time)
+    (tc/info :columns))
+
+;; For full details on all the possible options for type conversion of columns see the
+;; [tablecloth API docs](https://scicloj.github.io/tablecloth/index.html#Type_conversion)