Expose PDAL Mesh API (#27)

* Expose PDAL Mesh API

* Make Native Autoclosable

* Add CHANGELOG.md, fix codestyle to avoid syntax deprecated in Dotty

* Update PDAL-Scala reader types

* DSL expressions rename & DSL implicits reorganization

* Update PDAL filters up to 2.0.0

Author: pomadchin

.travis.yml

@@ -1,6 +1,6 @@
 sudo: false
 
-matrix:
+jobs:
   include:
     - os: linux
       jdk: openjdk8
@@ -20,6 +20,7 @@ matrix:
 
     - os: osx
       osx_image: xcode9.3
+      language: scala
       scala:
         - 2.13.1
       compiler:
@@ -31,6 +32,7 @@ matrix:
 
     - os: osx
       osx_image: xcode9.3
+      language: scala
       scala:
         - 2.12.10
       compiler:

CHANGELOG.md

@@ -0,0 +1,94 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [2.1.2] - 2020-03-02
+### Added
+- Expose PDAL Mesh API [#27](https://github.com/PDAL/java/pull/27)
+- [CHANGELOG](/CHANGELOG.md)
+
+### Changed
+- **Breaking change** // Rename Scala DSL case classes [#28](https://github.com/PDAL/java/issues/28)
+- **Breaking change** // Renamed dispose method to close [#27](https://github.com/PDAL/java/pull/27)
+- Update Scala DSL up to PDAL 2.0 [#29](https://github.com/PDAL/java/issues/29)
+
+## [2.0.0] - 2020-01-14
+### Changed 
+- PDAL 2.0 compatible release.
+- Added Scala 2.13 support.
+
+## [1.9.0] - 2019-05-15
+### Changed 
+- Release process improvements.
+
+## [1.8.6] - 2019-04-26
+### Changed 
+- Release process improvements.
+
+## [1.8.5] - 2019-04-15
+### Changed 
+- PDAL JNI Bindings thread safety [#19](https://github.com/PDAL/java/issues/19)
+
+### Fixed
+- PythonFilter gives invalid syntax [#9](https://github.com/PDAL/java/issues/9)
+- Option in filter.Python seems to be misspelled [#8](https://github.com/PDAL/java/issues/8)
+
+## [1.8.4] - 2019-04-09
+### Changed
+- Make Exceptions Handling better [#17](https://github.com/PDAL/java/pull/17)
+
+## [1.8.3] - 2019-04-09
+### Fixed
+- CSV files read issues [#15](https://github.com/PDAL/java/issues/15)
+
+## [1.8.2] - 2019-03-27
+### Fixed
+- Fix dimName calls to be better, otherwise CSV reads wont work [#14](https://github.com/PDAL/java/pull/14)
+
+## [1.8.1] - 2019-03-25
+### Changed
+- Update JTS to make it GeoTrellis compatible [#13](https://github.com/PDAL/java/pull/13)
+
+## [1.7.0-RC4] - 2019-01-14
+### Changed
+- Scala version update up to 2.12.6.
+- Update dependencies and base PDAL version.
+- Release process improvements.
+
+### Fixed
+- Fix matlab reader ReaderType [#10](https://github.com/PDAL/java/pull/10)
+- Fix typo in GdalWrite [#11](https://github.com/PDAL/java/pull/11)
+
+## [1.7.0-RC3] - 2018-04-16
+### Changed
+- Release process improvements.
+
+## [1.7.0-RC2] - 2018-04-15
+### Added
+- An [examples](https://github.com/PDAL/java/tree/1.7.0-RC2/examples/pdal-jni) project.
+
+### Changed
+- Release process improvements.
+
+## [1.7.0-RC1] - 2018-04-15
+### Changed
+- Moved from the PDAL repo and established own lifecycle.
+
+[Unreleased]: https://github.com/PDAL/java/compare/2.1.2...HEAD
+[2.1.2]: https://github.com/PDAL/java/compare/2.0.0...2.1.2
+[2.0.0]: https://github.com/PDAL/java/compare/1.9.0...2.0.0
+[1.9.0]: https://github.com/PDAL/java/compare/1.8.6...1.9.0
+[1.8.6]: https://github.com/PDAL/java/compare/1.8.5...1.8.6
+[1.8.5]: https://github.com/PDAL/java/compare/1.8.4...1.8.5
+[1.8.4]: https://github.com/PDAL/java/compare/1.8.3...1.8.4
+[1.8.3]: https://github.com/PDAL/java/compare/1.8.2...1.8.3
+[1.8.2]: https://github.com/PDAL/java/compare/1.8.1...1.8.2
+[1.8.1]: https://github.com/PDAL/java/compare/1.7.0-RC4...1.8.1
+[1.7.0-RC4]: https://github.com/PDAL/java/compare/1.7.0-RC3...1.7.0-RC4
+[1.7.0-RC3]: https://github.com/PDAL/java/compare/1.7.0-RC2...1.7.0-RC3
+[1.7.0-RC2]: https://github.com/PDAL/java/compare/1.7.0-RC1...1.7.0-RC2
+[1.7.0-RC1]: https://github.com/PDAL/java/compare/1.7.0-RC1...1.7.0-RC1

README.md

@@ -18,16 +18,16 @@ resolvers ++= Seq(
 )
 
 libraryDependencies ++= Seq(
-  "io.pdal" %% "pdal" % "2.0.0", // core library
-  "io.pdal" %  "pdal-native" % "2.0.0" // jni bindings
+  "io.pdal" %% "pdal" % "2.1.2", // core library
+  "io.pdal" %  "pdal-native" % "2.1.2" // jni bindings
 )
 ```
 
-It's required to have native JNI binary in `java.library.path`:
+If you would like to use your own bindings, it is necessary to set `java.library.path`:
 
 ```scala
 // Mac OS X example with manual JNI installation
-// cp -f native/target/resource_managed/main/native/x86_64-darwin/libpdaljni.2.0.dylib /usr/local/lib/libpdaljni.2.0.dylib
+// cp -f native/target/resource_managed/main/native/x86_64-darwin/libpdaljni.2.1.dylib /usr/local/lib/libpdaljni.2.1.dylib
 // place built binary into /usr/local/lib, and pass java.library.path to your JVM
 javaOptions += "-Djava.library.path=/usr/local/lib"
 ```
@@ -37,18 +37,17 @@ Dependency contains bindings for `x86_64-darwin` and `x86_64-linux`, other versi
 
 ## PDAL-Scala
 
-Scala API to build pipeline expressions instead of writing a raw JSON.
+Scala API allows to build pipeline expressions instead of writing a raw JSON.
 
 ```scala
 libraryDependencies ++= Seq(
-  "io.pdal" %% "pdal-scala" % "2.0.0", // scala core library
-  "io.pdal" %  "pdal-native" % "2.0.0" // jni bindings
+  "io.pdal" %% "pdal-scala" % "2.1.2", // scala core library
+  "io.pdal" %  "pdal-native" % "2.1.2" // jni bindings
 )
 ```
 
-Scala API covers PDAL 1.8.x but is compatible with PDAL >= 1.4.x, to use any custom DSL
-that is not covered by the current Scala API you can use `RawExpr` type to build `Pipeline 
-Expression`.
+Scala API covers PDAL 2.0.x, to use any custom DSL that is not covered by the 
+current Scala API you can use `RawExpr` type to build `Pipeline Expression`.
 
 ### Code examples
 
@@ -74,11 +73,11 @@ val expected =
   """.stripMargin
 
 // The same, but using scala DSL
-val pc: PipelineConstructor = LasRead("/path/to/las") ~ CropFilter() ~ LasWrite("/path/to/new/las")
+val pc = ReadLas("/path/to/las") ~ FilterCrop() ~ WriteLas("/path/to/new/las")
 
 // The same, but using RawExpr, to support not implemented PDAL Pipeline API features
 // RawExpr accepts a circe.Json type, which can be a json object of any desired complexity
-val pcWithRawExpr = LasRead("/path/to/las") ~ RawExpr(Map("type" -> "filters.crop").asJson) ~ LasWrite("/path/to/new/las") 
+val pcWithRawExpr = ReadLas("/path/to/las") ~ RawExpr(Map("type" -> "filters.crop").asJson) ~ WriteLas("/path/to/new/las") 
 ```
 
 ### Demo project example
@@ -87,12 +86,12 @@ JNI bindings basic usage examples can be found [here](./examples).
 
 ## How to compile
 
-Development purposes (including binaries):
+Development purposes (including binaries) compilation:
   1. Install PDAL (using brew / package managers (unix) / build from sources / etc)     
   2. Build native libs `./sbt native/nativeCompile` (optionally, binaries would be built during tests run)
   3. Run `./sbt core/test` to run PDAL tests
 
-Only Java development purposes:
+Only Java development purposes compilation:
   1. Provide `$LD_LIBRARY_PATH` or `$DYLD_LIBRARY_PATH`
   2. If you don't want to provide global variable you can pass `-Djava.library.path=<path>` into sbt:
     `./sbt -Djava.library.path=<path>`

build.sbt

@@ -1,7 +1,7 @@
 name := "pdal-jni"
 
 lazy val commonSettings = Seq(
-  version := "2.0.0" + Environment.versionSuffix,
+  version := "2.1.2" + Environment.versionSuffix,
   scalaVersion := "2.13.1",
   crossScalaVersions := Seq("2.13.1", "2.12.10", "2.11.12"),
   organization := "io.pdal",
@@ -57,6 +57,8 @@ lazy val root = (project in file("."))
 
 lazy val `core-scala` = project
   .settings(commonSettings: _*)
+  .settings(Dependencies.macroSettings)
+  .settings(Dependencies.licenseSettings)
   .settings(name := "pdal-scala")
   .settings(javah / target := (native / nativeCompile / sourceDirectory).value / "include")
   .settings(libraryDependencies ++= Seq(
@@ -67,9 +69,8 @@ lazy val `core-scala` = project
     Dependencies.jtsCore,
     Dependencies.scalaTest % Test
   ))
-  .settings(headerLicense := Some(HeaderLicense.ALv2("2017", "Azavea")))
-  .settings(licenses := Seq("Apache-2.0" -> url("https://www.apache.org/licenses/LICENSE-2.0.html")))
   .dependsOn(core)
+  .dependsOn(Environment.dependOnNative(native % Runtime): _*)
 
 lazy val core = project
   .settings(commonSettings: _*)

core-scala/src/main/scala/io/pdal/pipeline/ExprType.scala

@@ -16,6 +16,10 @@
 
 package io.pdal.pipeline
 
+import io.circe.{Decoder, Encoder}
+import io.circe.syntax._
+import cats.syntax.either._
+
 import scala.util.Try
 
 trait ExprType {
@@ -26,6 +30,11 @@ trait ExprType {
 }
 
 object ExprType {
+  implicit def exprTypeEncoder[T <: ExprType]: Encoder[T] = Encoder.instance { _.toString.asJson }
+  implicit def exprTypeDecoder[T <: ExprType]: Decoder[T] = Decoder.decodeString.emap { str =>
+    Either.catchNonFatal(ExprType.fromName(str).asInstanceOf[T]).leftMap(_ => "ExprType")
+  }
+
   def fromName(name: String): ExprType =
     Try(FilterTypes.fromName(name))
       .getOrElse(Try(ReaderTypes.fromName(name))

core-scala/src/main/scala/io/pdal/pipeline/FilterTypes.scala

@@ -20,24 +20,30 @@ sealed trait FilterType extends ExprType { val `type` = "filters" }
 
 object FilterTypes {
   case object approximatecoplanar extends FilterType
+  case object assign extends FilterType
   case object chipper extends FilterType
   case object cluster extends FilterType
   case object colorinterp extends FilterType
   case object colorization extends FilterType
   case object computerange extends FilterType
+  case object covariancefeatures extends FilterType
   case object cpd extends FilterType
   case object crop extends FilterType
   case object decimation extends FilterType
+  case object dem extends FilterType
+  case object delaunay extends FilterType
   case object divider extends FilterType
   case object eigenvalues extends FilterType
   case object estimaterank extends FilterType
+  case object elm extends FilterType
   case object ferry extends FilterType
   case object greedyprojection extends FilterType
   case object gridprojection extends FilterType
   case object groupby extends FilterType
   case object hag extends FilterType
   case object head extends FilterType
   case object hexbin extends FilterType
+  case object info extends FilterType
   case object icp extends FilterType
   case object iqr extends FilterType
   case object kdistance extends FilterType
@@ -49,16 +55,21 @@ object FilterTypes {
   case object mongus extends FilterType
   case object mortonorder extends FilterType
   case object movingleastsquares extends FilterType
+  case object miniball extends FilterType
+  case object neighborclassifier extends FilterType
+  case object nndistance extends FilterType
   case object normal extends FilterType
   case object overlay extends FilterType
   case object outlier extends FilterType
   case object pclblock extends FilterType
+  case object planefit extends FilterType
   case object pmf extends FilterType
   case object poisson extends FilterType
   case object python extends FilterType
   case object radialdensity extends FilterType
   case object range extends FilterType
   case object randomize extends FilterType
+  case object reciprocity extends FilterType
   case object reprojection extends FilterType
   case object sample extends FilterType
   case object smrf extends FilterType
@@ -72,12 +83,12 @@ object FilterTypes {
   case object voxelgrid extends FilterType
 
   lazy val all = List(
-    approximatecoplanar, chipper, cluster, colorinterp, colorization, computerange, crop,
-    cpd, decimation, divider, eigenvalues, estimaterank, ferry, greedyprojection, gridprojection, groupby,
-    hag, head, hexbin, icp, iqr, kdistance, locate, lof, mad, matlab, merge, mongus, mortonorder, movingleastsquares,
-    normal, outlier, overlay, pclblock, pmf, poisson, python, radialdensity, randomize, range, reprojection,
-    sample, smrf, sort, splitter, stats, transformation, voxelcenternearestneighbor, voxelcentroidnearestneighbor,
-    voxelgrid
+    approximatecoplanar, assign, chipper, cluster, colorinterp, colorization, computerange, covariancefeatures, crop,
+    cpd, decimation, dem, delaunay, divider, eigenvalues, estimaterank, elm, ferry, greedyprojection, gridprojection, groupby,
+    hag, head, hexbin, info, icp, iqr, kdistance, locate, lof, mad, matlab, merge, mongus, mortonorder, movingleastsquares,
+    miniball, neighborclassifier, nndistance, normal, outlier, overlay, pclblock, planefit, pmf, poisson, python, radialdensity,
+    randomize, reciprocity, range, reprojection, sample, smrf, sort, splitter, stats, transformation, voxelcenternearestneighbor,
+    voxelcentroidnearestneighbor, voxelgrid
   )
 
   def fromName(name: String): FilterType =

core-scala/src/main/scala/io/pdal/pipeline/Implicits.scala

@@ -19,11 +19,10 @@ package io.pdal.pipeline
 import io.pdal.PointCloud
 import org.locationtech.jts.geom.Coordinate
 
-object Implicits extends Implicits
+object Implicits extends Implicits with Serializable
 
-trait Implicits extends Serializable {
+trait Implicits {
   implicit class withPointCloudMethods(pointCloud: PointCloud) {
-    def getCoordinate(i: Int) =
-      new Coordinate(pointCloud.getX(i), pointCloud.getY(i), pointCloud.getZ(i))
+    def getCoordinate(i: Int) = new Coordinate(pointCloud.getX(i), pointCloud.getY(i), pointCloud.getZ(i))
   }
 }

core-scala/src/main/scala/io/pdal/pipeline/PipelineConstructor.scala

@@ -0,0 +1,55 @@
+/*
+ * Copyright 2020 Azavea
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package io.pdal.pipeline
+
+import io.pdal.Pipeline
+
+import io.circe.{Decoder, Encoder, Json}
+import io.circe.syntax._
+
+case class PipelineConstructor(list: List[PipelineExpr]) {
+  def ~(e: PipelineExpr): PipelineConstructor =
+    e match {
+      case ENil => this
+      case _    => PipelineConstructor(list :+ e)
+    }
+  def ~(e: Option[PipelineExpr]): PipelineConstructor = e.fold(this)(this ~ _)
+  def map[B](f: PipelineExpr => B): List[B] = list.map(f)
+  def toPipeline: Pipeline = Pipeline(this.asJson.noSpaces)
+}
+
+object PipelineConstructor {
+  implicit val pipelineConstructorEncoder: Encoder[PipelineConstructor] = Encoder.instance { constructor =>
+    Json.obj(
+      "pipeline" -> constructor.list.flatMap {
+        _.list.flatMap {
+          case RawExpr(json) => json.asObject
+          case expr => expr.asJson.asObject
+        }.map {
+          _.remove("class_type") // remove type
+            .filter { case (_, value) => !value.isNull } // cleanup options
+        }
+      }.asJson
+    )
+  }
+  implicit val pipelineConstructorDecoder: Decoder[PipelineConstructor] = Decoder.instance {
+    _.downField("pipeline").as[PipelineConstructor]
+  }
+
+  implicit def pipelineConstructorToJson(expr: PipelineConstructor): Json = expr.asJson
+  implicit def pipelineConstructorToString(expr: PipelineConstructor): String = expr.asJson.noSpaces
+}