Skip to content

Commit 194ac3b

Browse files
huaxingaosrowen
huaxingao
authored and
srowen
committed
[SPARK-31708][ML][DOCS] Add docs and examples for ANOVASelector and FValueSelector
### What changes were proposed in this pull request? Add docs and examples for ANOVASelector and FValueSelector ### Why are the changes needed? Complete the implementation of ANOVASelector and FValueSelector ### Does this PR introduce _any_ user-facing change? Yes <img width="850" alt="Screen Shot 2020-05-13 at 5 17 44 PM" src="https://user-images.githubusercontent.com/13592258/81878703-b4f94480-953d-11ea-9166-da3c64852b90.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 05 15 PM" src="https://user-images.githubusercontent.com/13592258/81878600-6055c980-953d-11ea-8b24-09c31647139b.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 06 06 PM" src="https://user-images.githubusercontent.com/13592258/81878603-621f8d00-953d-11ea-9447-39913ccc067d.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 06 21 PM" src="https://user-images.githubusercontent.com/13592258/81878606-65b31400-953d-11ea-9d76-51859266d1a8.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 07 10 PM" src="https://user-images.githubusercontent.com/13592258/81878611-69df3180-953d-11ea-8618-23a2a6cfd730.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 07 33 PM" src="https://user-images.githubusercontent.com/13592258/81878620-6cda2200-953d-11ea-9c46-da763328364e.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 07 47 PM" src="https://user-images.githubusercontent.com/13592258/81878625-6f3c7c00-953d-11ea-9d11-2281b33a0bd8.png"> <img width="851" alt="Screen Shot 2020-05-13 at 5 19 35 PM" src="https://user-images.githubusercontent.com/13592258/81878882-13bebe00-953e-11ea-9776-288bac97d93f.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 08 42 PM" src="https://user-images.githubusercontent.com/13592258/81878637-76638a00-953d-11ea-94b0-dc9bc85ae2b7.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 09 01 PM" src="https://user-images.githubusercontent.com/13592258/81878640-79f71100-953d-11ea-9a66-b27f9482fbd3.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 09 50 PM" src="https://user-images.githubusercontent.com/13592258/81878644-7cf20180-953d-11ea-9142-9658c8e90986.png"> <img width="851" alt="Screen Shot 2020-05-13 at 5 10 06 PM" src="https://user-images.githubusercontent.com/13592258/81878653-81b6b580-953d-11ea-9dc2-8015095cf569.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 10 59 PM" src="https://user-images.githubusercontent.com/13592258/81878658-854a3c80-953d-11ea-8dc9-217aa749fd00.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 11 27 PM" src="https://user-images.githubusercontent.com/13592258/81878659-87ac9680-953d-11ea-8c6b-74ab76748e4a.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 14 54 PM" src="https://user-images.githubusercontent.com/13592258/81878664-8b401d80-953d-11ea-9ee1-05f6677e263c.png"> <img width="850" alt="Screen Shot 2020-05-13 at 5 15 17 PM" src="https://user-images.githubusercontent.com/13592258/81878669-8da27780-953d-11ea-8216-77eb8bb7e091.png"> ### How was this patch tested? Manually build and check Closes apache#28524 from huaxingao/examples. Authored-by: Huaxin Gao <[email protected]> Signed-off-by: Sean Owen <[email protected]>
1 parent 5d90886 commit 194ac3b

14 files changed

+610
-6
lines changed

Diff for: ‎docs/ml-features.md

+140
Original file line numberDiff line numberDiff line change
@@ -1793,6 +1793,146 @@ for more details on the API.
17931793
</div>
17941794
</div>
17951795

1796+
## ANOVASelector
1797+
1798+
`ANOVASelector` operates on categorical labels with continuous features. It uses the
1799+
[one-way ANOVA F-test](https://en.wikipedia.org/wiki/F-test#Multiple-comparison_ANOVA_problems) to decide which
1800+
features to choose.
1801+
It supports five selection methods: `numTopFeatures`, `percentile`, `fpr`, `fdr`, `fwe`:
1802+
* `numTopFeatures` chooses a fixed number of top features according to ANOVA F-test.
1803+
* `percentile` is similar to `numTopFeatures` but chooses a fraction of all features instead of a fixed number.
1804+
* `fpr` chooses all features whose p-values are below a threshold, thus controlling the false positive rate of selection.
1805+
* `fdr` uses the [Benjamini-Hochberg procedure](https://en.wikipedia.org/wiki/False_discovery_rate#Benjamini.E2.80.93Hochberg_procedure) to choose all features whose false discovery rate is below a threshold.
1806+
* `fwe` chooses all features whose p-values are below a threshold. The threshold is scaled by 1/numFeatures, thus controlling the family-wise error rate of selection.
1807+
By default, the selection method is `numTopFeatures`, with the default number of top features set to 50.
1808+
The user can choose a selection method using `setSelectorType`.
1809+
1810+
**Examples**
1811+
1812+
Assume that we have a DataFrame with the columns `id`, `features`, and `label`, which is used as
1813+
our target to be predicted:
1814+
1815+
~~~
1816+
id | features | label
1817+
---|--------------------------------|---------
1818+
1 | [1.7, 4.4, 7.6, 5.8, 9.6, 2.3] | 3.0
1819+
2 | [8.8, 7.3, 5.7, 7.3, 2.2, 4.1] | 2.0
1820+
3 | [1.2, 9.5, 2.5, 3.1, 8.7, 2.5] | 3.0
1821+
4 | [3.7, 9.2, 6.1, 4.1, 7.5, 3.8] | 2.0
1822+
5 | [8.9, 5.2, 7.8, 8.3, 5.2, 3.0] | 4.0
1823+
6 | [7.9, 8.5, 9.2, 4.0, 9.4, 2.1] | 4.0
1824+
~~~
1825+
1826+
If we use `ANOVASelector` with `numTopFeatures = 1`, the
1827+
last column in our `features` is chosen as the most useful feature:
1828+
1829+
~~~
1830+
id | features | label | selectedFeatures
1831+
---|--------------------------------|---------|------------------
1832+
1 | [1.7, 4.4, 7.6, 5.8, 9.6, 2.3] | 3.0 | [2.3]
1833+
2 | [8.8, 7.3, 5.7, 7.3, 2.2, 4.1] | 2.0 | [4.1]
1834+
3 | [1.2, 9.5, 2.5, 3.1, 8.7, 2.5] | 3.0 | [2.5]
1835+
4 | [3.7, 9.2, 6.1, 4.1, 7.5, 3.8] | 2.0 | [3.8]
1836+
5 | [8.9, 5.2, 7.8, 8.3, 5.2, 3.0] | 4.0 | [3.0]
1837+
6 | [7.9, 8.5, 9.2, 4.0, 9.4, 2.1] | 4.0 | [2.1]
1838+
~~~
1839+
1840+
<div class="codetabs">
1841+
<div data-lang="scala" markdown="1">
1842+
1843+
Refer to the [ANOVASelector Scala docs](api/scala/org/apache/spark/ml/feature/ANOVASelector.html)
1844+
for more details on the API.
1845+
1846+
{% include_example scala/org/apache/spark/examples/ml/ANOVASelectorExample.scala %}
1847+
</div>
1848+
1849+
<div data-lang="java" markdown="1">
1850+
1851+
Refer to the [ANOVASelector Java docs](api/java/org/apache/spark/ml/feature/ANOVASelector.html)
1852+
for more details on the API.
1853+
1854+
{% include_example java/org/apache/spark/examples/ml/JavaANOVASelectorExample.java %}
1855+
</div>
1856+
1857+
<div data-lang="python" markdown="1">
1858+
1859+
Refer to the [ANOVASelector Python docs](api/python/pyspark.ml.html#pyspark.ml.feature.ANOVASelector)
1860+
for more details on the API.
1861+
1862+
{% include_example python/ml/anova_selector_example.py %}
1863+
</div>
1864+
</div>
1865+
1866+
## FValueSelector
1867+
1868+
`FValueSelector` operates on categorical labels with continuous features. It uses the
1869+
[F-test for regression](https://en.wikipedia.org/wiki/F-test#Regression_problems) to decide which
1870+
features to choose.
1871+
It supports five selection methods: `numTopFeatures`, `percentile`, `fpr`, `fdr`, `fwe`:
1872+
* `numTopFeatures` chooses a fixed number of top features according to a F-test for regression.
1873+
* `percentile` is similar to `numTopFeatures` but chooses a fraction of all features instead of a fixed number.
1874+
* `fpr` chooses all features whose p-values are below a threshold, thus controlling the false positive rate of selection.
1875+
* `fdr` uses the [Benjamini-Hochberg procedure](https://en.wikipedia.org/wiki/False_discovery_rate#Benjamini.E2.80.93Hochberg_procedure) to choose all features whose false discovery rate is below a threshold.
1876+
* `fwe` chooses all features whose p-values are below a threshold. The threshold is scaled by 1/numFeatures, thus controlling the family-wise error rate of selection.
1877+
By default, the selection method is `numTopFeatures`, with the default number of top features set to 50.
1878+
The user can choose a selection method using `setSelectorType`.
1879+
1880+
**Examples**
1881+
1882+
Assume that we have a DataFrame with the columns `id`, `features`, and `label`, which is used as
1883+
our target to be predicted:
1884+
1885+
~~~
1886+
id | features | label
1887+
---|--------------------------------|---------
1888+
1 | [6.0, 7.0, 0.0, 7.0, 6.0, 0.0] | 4.6
1889+
2 | [0.0, 9.0, 6.0, 0.0, 5.0, 9.0] | 6.6
1890+
3 | [0.0, 9.0, 3.0, 0.0, 5.0, 5.0] | 5.1
1891+
4 | [0.0, 9.0, 8.0, 5.0, 6.0, 4.0] | 7.6
1892+
5 | [8.0, 9.0, 6.0, 5.0, 4.0, 4.0] | 9.0
1893+
6 | [8.0, 9.0, 6.0, 4.0, 0.0, 0.0] | 9.0
1894+
~~~
1895+
1896+
If we use `FValueSelector` with `numTopFeatures = 1`, the
1897+
3rd column in our `features` is chosen as the most useful feature:
1898+
1899+
~~~
1900+
id | features | label | selectedFeatures
1901+
---|--------------------------------|---------|------------------
1902+
1 | [6.0, 7.0, 0.0, 7.0, 6.0, 0.0] | 4.6 | [0.0]
1903+
2 | [0.0, 9.0, 6.0, 0.0, 5.0, 9.0] | 6.6 | [6.0]
1904+
3 | [0.0, 9.0, 3.0, 0.0, 5.0, 5.0] | 5.1 | [3.0]
1905+
4 | [0.0, 9.0, 8.0, 5.0, 6.0, 4.0] | 7.6 | [8.0]
1906+
5 | [8.0, 9.0, 6.0, 5.0, 4.0, 4.0] | 9.0 | [6.0]
1907+
6 | [8.0, 9.0, 6.0, 4.0, 0.0, 0.0] | 9.0 | [6.0]
1908+
~~~
1909+
1910+
<div class="codetabs">
1911+
<div data-lang="scala" markdown="1">
1912+
1913+
Refer to the [FValueSelector Scala docs](api/scala/org/apache/spark/ml/feature/FValueSelector.html)
1914+
for more details on the API.
1915+
1916+
{% include_example scala/org/apache/spark/examples/ml/FValueSelectorExample.scala %}
1917+
</div>
1918+
1919+
<div data-lang="java" markdown="1">
1920+
1921+
Refer to the [FValueSelector Java docs](api/java/org/apache/spark/ml/feature/FValueSelector.html)
1922+
for more details on the API.
1923+
1924+
{% include_example java/org/apache/spark/examples/ml/JavaFValueSelectorExample.java %}
1925+
</div>
1926+
1927+
<div data-lang="python" markdown="1">
1928+
1929+
Refer to the [FValueSelector Python docs](api/python/pyspark.ml.html#pyspark.ml.feature.FValueSelector)
1930+
for more details on the API.
1931+
1932+
{% include_example python/ml/anova_selector_example.py %}
1933+
</div>
1934+
</div>
1935+
17961936
## VarianceThresholdSelector
17971937

17981938
`VarianceThresholdSelector` is a selector that removes low-variance features. Features with a

Diff for: ‎docs/ml-statistics.md

+55-1
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,35 @@ The output will be a DataFrame that contains the correlation matrix of the colum
7979

8080
Hypothesis testing is a powerful tool in statistics to determine whether a result is statistically
8181
significant, whether this result occurred by chance or not. `spark.ml` currently supports Pearson's
82-
Chi-squared ( $\chi^2$) tests for independence.
82+
Chi-squared ( $\chi^2$) tests for independence, as well as ANOVA test for classification tasks and
83+
F-value test for regression tasks.
84+
85+
### ANOVATest
86+
87+
`ANOVATest` computes ANOVA F-values between labels and features for classification tasks. The labels should be categorical
88+
and features should be continuous.
89+
90+
<div class="codetabs">
91+
<div data-lang="scala" markdown="1">
92+
Refer to the [`ANOVATest` Scala docs](api/scala/org/apache/spark/ml/stat/ANOVATest$.html) for details on the API.
93+
94+
{% include_example scala/org/apache/spark/examples/ml/ANOVATestExample.scala %}
95+
</div>
96+
97+
<div data-lang="java" markdown="1">
98+
Refer to the [`ANOVATest` Java docs](api/java/org/apache/spark/ml/stat/ANOVATest.html) for details on the API.
99+
100+
{% include_example java/org/apache/spark/examples/ml/JavaANOVATestExample.java %}
101+
</div>
102+
103+
<div data-lang="python" markdown="1">
104+
Refer to the [`ANOVATest` Python docs](api/python/index.html#pyspark.ml.stat.ANOVATest$) for details on the API.
105+
106+
{% include_example python/ml/anova_test_example.py %}
107+
</div>
108+
</div>
109+
110+
### ChiSquareTest
83111

84112
`ChiSquareTest` conducts Pearson's independence test for every feature against the label.
85113
For each feature, the (feature, label) pairs are converted into a contingency matrix for which
@@ -106,6 +134,32 @@ Refer to the [`ChiSquareTest` Python docs](api/python/index.html#pyspark.ml.stat
106134

107135
</div>
108136

137+
### FValueTest
138+
139+
`FValueTest` computes F-values between labels and features for regression tasks. Both the labels
140+
and features should be continuous.
141+
142+
<div class="codetabs">
143+
<div data-lang="scala" markdown="1">
144+
Refer to the [`FValueTest` Scala docs](api/scala/org/apache/spark/ml/stat/FValueTest$.html) for details on the API.
145+
146+
{% include_example scala/org/apache/spark/examples/ml/FValueTestExample.scala %}
147+
</div>
148+
149+
<div data-lang="java" markdown="1">
150+
Refer to the [`FValueTest` Java docs](api/java/org/apache/spark/ml/stat/FValueTest.html) for details on the API.
151+
152+
{% include_example java/org/apache/spark/examples/ml/JavaFValueTestExample.java %}
153+
</div>
154+
155+
<div data-lang="python" markdown="1">
156+
Refer to the [`FValueTest` Python docs](api/python/index.html#pyspark.ml.stat.FValueTest$) for details on the API.
157+
158+
{% include_example python/ml/fvalue_test_example.py %}
159+
</div>
160+
161+
</div>
162+
109163
## Summarizer
110164

111165
We provide vector column summary statistics for `Dataframe` through `Summarizer`.

Diff for: ‎examples/src/main/java/org/apache/spark/examples/ml/JavaANOVASelectorExample.java

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.spark.examples.ml;
19+
20+
import org.apache.spark.sql.Dataset;
21+
import org.apache.spark.sql.SparkSession;
22+
23+
// $example on$
24+
import java.util.Arrays;
25+
import java.util.List;
26+
27+
import org.apache.spark.ml.feature.ANOVASelector;
28+
import org.apache.spark.ml.linalg.VectorUDT;
29+
import org.apache.spark.ml.linalg.Vectors;
30+
import org.apache.spark.sql.Row;
31+
import org.apache.spark.sql.RowFactory;
32+
import org.apache.spark.sql.types.*;
33+
// $example off$
34+
35+
/**
36+
* An example for ANOVASelector.
37+
* Run with
38+
* <pre>
39+
* bin/run-example ml.JavaANOVASelectorExample
40+
* </pre>
41+
*/
42+
public class JavaANOVASelectorExample {
43+
public static void main(String[] args) {
44+
SparkSession spark = SparkSession
45+
.builder()
46+
.appName("JavaANOVASelectorExample")
47+
.getOrCreate();
48+
49+
// $example on$
50+
List<Row> data = Arrays.asList(
51+
RowFactory.create(1, Vectors.dense(1.7, 4.4, 7.6, 5.8, 9.6, 2.3), 3.0),
52+
RowFactory.create(2, Vectors.dense(8.8, 7.3, 5.7, 7.3, 2.2, 4.1), 2.0),
53+
RowFactory.create(3, Vectors.dense(1.2, 9.5, 2.5, 3.1, 8.7, 2.5), 3.0),
54+
RowFactory.create(4, Vectors.dense(3.7, 9.2, 6.1, 4.1, 7.5, 3.8), 2.0),
55+
RowFactory.create(5, Vectors.dense(8.9, 5.2, 7.8, 8.3, 5.2, 3.0), 4.0),
56+
RowFactory.create(6, Vectors.dense(7.9, 8.5, 9.2, 4.0, 9.4, 2.1), 4.0)
57+
);
58+
StructType schema = new StructType(new StructField[]{
59+
new StructField("id", DataTypes.IntegerType, false, Metadata.empty()),
60+
new StructField("features", new VectorUDT(), false, Metadata.empty()),
61+
new StructField("label", DataTypes.DoubleType, false, Metadata.empty())
62+
});
63+
64+
Dataset<Row> df = spark.createDataFrame(data, schema);
65+
66+
ANOVASelector selector = new ANOVASelector()
67+
.setNumTopFeatures(1)
68+
.setFeaturesCol("features")
69+
.setLabelCol("label")
70+
.setOutputCol("selectedFeatures");
71+
72+
Dataset<Row> result = selector.fit(df).transform(df);
73+
74+
System.out.println("ANOVASelector output with top " + selector.getNumTopFeatures()
75+
+ " features selected");
76+
result.show();
77+
78+
// $example off$
79+
spark.stop();
80+
}
81+
}

Diff for: ‎examples/src/main/java/org/apache/spark/examples/ml/JavaANOVATestExample.java

+1-1
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ public static void main(String[] args) {
5151
List<Row> data = Arrays.asList(
5252
RowFactory.create(3.0, Vectors.dense(1.7, 4.4, 7.6, 5.8, 9.6, 2.3)),
5353
RowFactory.create(2.0, Vectors.dense(8.8, 7.3, 5.7, 7.3, 2.2, 4.1)),
54-
RowFactory.create(1.0, Vectors.dense(1.2, 9.5, 2.5, 3.1, 8.7, 2.5)),
54+
RowFactory.create(3.0, Vectors.dense(1.2, 9.5, 2.5, 3.1, 8.7, 2.5)),
5555
RowFactory.create(2.0, Vectors.dense(3.7, 9.2, 6.1, 4.1, 7.5, 3.8)),
5656
RowFactory.create(4.0, Vectors.dense(8.9, 5.2, 7.8, 8.3, 5.2, 3.0)),
5757
RowFactory.create(4.0, Vectors.dense(7.9, 8.5, 9.2, 4.0, 9.4, 2.1))

Diff for: ‎examples/src/main/java/org/apache/spark/examples/ml/JavaFValueSelectorExample.java

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.spark.examples.ml;
19+
20+
import org.apache.spark.sql.Dataset;
21+
import org.apache.spark.sql.SparkSession;
22+
23+
// $example on$
24+
import java.util.Arrays;
25+
import java.util.List;
26+
27+
import org.apache.spark.ml.feature.FValueSelector;
28+
import org.apache.spark.ml.linalg.VectorUDT;
29+
import org.apache.spark.ml.linalg.Vectors;
30+
import org.apache.spark.sql.Row;
31+
import org.apache.spark.sql.RowFactory;
32+
import org.apache.spark.sql.types.*;
33+
// $example off$
34+
35+
/**
36+
* An example demonstrating FValueSelector.
37+
* Run with
38+
* <pre>
39+
* bin/run-example ml.JavaFValueSelectorExample
40+
* </pre>
41+
*/
42+
public class JavaFValueSelectorExample {
43+
public static void main(String[] args) {
44+
SparkSession spark = SparkSession
45+
.builder()
46+
.appName("JavaFValueSelectorExample")
47+
.getOrCreate();
48+
49+
// $example on$
50+
List<Row> data = Arrays.asList(
51+
RowFactory.create(1, Vectors.dense(6.0, 7.0, 0.0, 7.0, 6.0, 0.0), 4.6),
52+
RowFactory.create(2, Vectors.dense(0.0, 9.0, 6.0, 0.0, 5.0, 9.0), 6.6),
53+
RowFactory.create(3, Vectors.dense(0.0, 9.0, 3.0, 0.0, 5.0, 5.0), 5.1),
54+
RowFactory.create(4, Vectors.dense(0.0, 9.0, 8.0, 5.0, 6.0, 4.0), 7.6),
55+
RowFactory.create(5, Vectors.dense(8.0, 9.0, 6.0, 5.0, 4.0, 4.0), 9.0),
56+
RowFactory.create(6, Vectors.dense(8.0, 9.0, 6.0, 4.0, 0.0, 0.0), 9.0)
57+
);
58+
StructType schema = new StructType(new StructField[]{
59+
new StructField("id", DataTypes.IntegerType, false, Metadata.empty()),
60+
new StructField("features", new VectorUDT(), false, Metadata.empty()),
61+
new StructField("label", DataTypes.DoubleType, false, Metadata.empty())
62+
});
63+
64+
Dataset<Row> df = spark.createDataFrame(data, schema);
65+
66+
FValueSelector selector = new FValueSelector()
67+
.setNumTopFeatures(1)
68+
.setFeaturesCol("features")
69+
.setLabelCol("label")
70+
.setOutputCol("selectedFeatures");
71+
72+
Dataset<Row> result = selector.fit(df).transform(df);
73+
74+
System.out.println("FValueSelector output with top " + selector.getNumTopFeatures()
75+
+ " features selected");
76+
result.show();
77+
78+
// $example off$
79+
spark.stop();
80+
}
81+
}

Diff for: ‎examples/src/main/java/org/apache/spark/examples/ml/JavaFValueTestExample.java

+1-1
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ public static void main(String[] args) {
6666
Row r = FValueTest.test(df, "features", "label").head();
6767
System.out.println("pValues: " + r.get(0).toString());
6868
System.out.println("degreesOfFreedom: " + r.getList(1).toString());
69-
System.out.println("fvalue: " + r.get(2).toString());
69+
System.out.println("fvalues: " + r.get(2).toString());
7070

7171
// $example off$
7272

0 commit comments

Comments
 (0)