Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of XE_NDCG_MART for the ranking task #2620

Merged
merged 13 commits into from
Jan 30, 2020
Merged

Implementation of XE_NDCG_MART for the ranking task #2620

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
95005c6
Implementation of XE_NDCG loss function for ranking.
Dec 6, 2019
4a93880
Add citation
Dec 6, 2019
7deec06
Check in example usage for xe_ndcg loss.
Dec 6, 2019
c0876fc
Seed the generator when a seed is provided in the config. Add unit-te…
Dec 7, 2019
c08921d
Update documentation
Dec 7, 2019
fe95e62
Fix indentation
Dec 7, 2019
ad176e0
Address issues raised by reviewers.
Dec 24, 2019
51fa03a
Clean up include statements.
Dec 26, 2019
f5f5d72
Fix issues raised by reviewers.
Jan 2, 2020
763a289
Regenerate parameters.rst
Jan 2, 2020
81d4b92
Add a note to explain that reproducing xe_ndcg results requires num_t…
Jan 2, 2020
a7d9fb8
Introduce objective_seed and use that in rank_xendcg instead of direc…
Jan 28, 2020
a6584fb
Change default value of objective_seed
Jan 28, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion docs/Parameters.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Core Parameters

- **Note**: can be used only in CLI version; for language-specific packages you can use the correspondent functions

- ``objective`` :raw-html:`<a id="objective" title="Permalink to this parameter" href="#objective">&#x1F517;&#xFE0E;</a>`, default = ``regression``, type = enum, options: ``regression``, ``regression_l1``, ``huber``, ``fair``, ``poisson``, ``quantile``, ``mape``, ``gamma``, ``tweedie``, ``binary``, ``multiclass``, ``multiclassova``, ``cross_entropy``, ``cross_entropy_lambda``, ``lambdarank``, aliases: ``objective_type``, ``app``, ``application``
- ``objective`` :raw-html:`<a id="objective" title="Permalink to this parameter" href="#objective">&#x1F517;&#xFE0E;</a>`, default = ``regression``, type = enum, options: ``regression``, ``regression_l1``, ``huber``, ``fair``, ``poisson``, ``quantile``, ``mape``, ``gamma``, ``tweedie``, ``binary``, ``multiclass``, ``multiclassova``, ``cross_entropy``, ``cross_entropy_lambda``, ``lambdarank``, ``rank_xendcg``, aliases: ``objective_type``, ``app``, ``application``

- regression application

Expand Down Expand Up @@ -99,6 +99,10 @@ Core Parameters

- all values in ``label`` must be smaller than number of elements in ``label_gain``

- ``rank_xendcg``, `XE_NDCG_MART <https://arxiv.org/abs/1911.09798>`__ ranking objective function, aliases: ``xendcg``, ``xe_ndcg``, ``xe_ndcg_mart``, ``xendcg_mart``

- to obtain reproducible results, you should disable parallelism by setting ``num_threads`` to 1

- ``boosting`` :raw-html:`<a id="boosting" title="Permalink to this parameter" href="#boosting">&#x1F517;&#xFE0E;</a>`, default = ``gbdt``, type = enum, options: ``gbdt``, ``rf``, ``dart``, ``goss``, aliases: ``boosting_type``, ``boost``

- ``gbdt``, traditional Gradient Boosting Decision Tree, aliases: ``gbrt``
Expand Down Expand Up @@ -838,6 +842,12 @@ Objective Parameters

- separate by ``,``

- ``objective_seed`` :raw-html:`<a id="objective_seed" title="Permalink to this parameter" href="#objective_seed">&#x1F517;&#xFE0E;</a>`, default = ``5``, type = int

- random seed for objectives

- used only in the ``rank_xendcg`` objective

Metric Parameters
-----------------

Expand Down
33 changes: 33 additions & 0 deletions examples/xendcg/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
XE_NDCG Ranking Example
=======================

Here is an example for LightGBM to train a ranking model with the [XE_NDCG loss](https://arxiv.org/abs/1911.09798).

***You must follow the [installation instructions](https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html)
for the following commands to work. The `lightgbm` binary must be built and available at the root of this project.***

Training
--------

Run the following command in this folder:

```bash
"../../lightgbm" config=train.conf
```

Prediction
----------

You should finish training first.

Run the following command in this folder:

```bash
"../../lightgbm" config=predict.conf
```

Data Format
-----------

To learn more about the query format used in this example, check out the
[query data format](https://lightgbm.readthedocs.io/en/latest/Parameters.html#query-data).
5 changes: 5 additions & 0 deletions examples/xendcg/predict.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
task = predict

data = rank.test

input_model= LightGBM_model.txt
768 changes: 768 additions & 0 deletions examples/xendcg/rank.test
View file
Open in desktop

Large diffs are not rendered by default.

50 changes: 50 additions & 0 deletions examples/xendcg/rank.test.query
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
12
19
18
10
15
15
22
23
18
16
16
11
6
13
17
21
20
16
13
16
21
15
10
19
10
13
18
17
23
24
16
13
17
24
17
10
17
15
18
16
9
9
21
14
13
13
13
10
10
6
3,005 changes: 3,005 additions & 0 deletions examples/xendcg/rank.train
View file
Open in desktop

Large diffs are not rendered by default.

201 changes: 201 additions & 0 deletions examples/xendcg/rank.train.query
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
1
13
5
8
19
12
18
5
14
13
8
9
16
11
21
14
21
9
14
11
20
18
13
20
22
22
13
17
10
13
12
13
13
23
18
13
20
12
22
14
13
23
13
14
14
5
13
15
14
14
16
16
15
21
22
10
22
18
25
16
12
12
15
15
25
13
9
12
8
16
25
19
24
12
16
10
16
9
17
15
7
9
15
14
16
17
8
17
12
18
23
10
12
12
4
14
12
15
27
16
20
13
19
13
17
17
16
12
15
14
14
19
12
23
18
16
9
23
11
15
8
10
10
16
11
15
22
16
17
23
16
22
17
14
12
14
20
15
17
15
15
22
9
21
9
17
16
15
13
13
15
14
18
21
14
17
15
14
16
12
17
19
16
11
18
11
13
14
9
16
15
16
25
9
13
22
16
18
20
14
11
9
16
19
19
11
11
13
14
14
13
16
6
21
16
12
16
11
24
12
10
Loading