Skip to content

Commit 4844e1f

Browse files
tbartley94pre-commit-ci[bot]
tbartley94
and
pre-commit-ci[bot]
authored
Hebrew itn (#366) (#367)
* Hebrew itn (#366) * rebase to main Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rebase Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rebasing Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * responding to formatting pr Signed-off-by: tbartley94 <[email protected]> * isort and moving string map to string file Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: tbartley94 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Hebrew itn (#368) * rebase to main Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rebase Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rebasing Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * responding to formatting pr Signed-off-by: tbartley94 <[email protected]> * isort and moving string map to string file Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * forgot init Signed-off-by: tbartley94 <[email protected]> --------- Signed-off-by: tbartley94 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Hebrew itn (#369) * rebase to main Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rebase Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rebasing Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * responding to formatting pr Signed-off-by: tbartley94 <[email protected]> * isort and moving string map to string file Signed-off-by: tbartley94 <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * forgot init Signed-off-by: tbartley94 <[email protected]> * more inits Signed-off-by: tbartley94 <[email protected]> --------- Signed-off-by: tbartley94 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --------- Signed-off-by: tbartley94 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent edd2288 commit 4844e1f

File tree

81 files changed

+3138
-2
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

81 files changed

+3138
-2
lines changed

‎Jenkinsfile‎

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ pipeline {
2323
SV_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/06-08-23-0'
2424
ZH_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/11-13-24-0'
2525
IT_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/08-22-24-0'
26+
HE_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/09-24-25-0'
2627
HY_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/03-12-24-0'
2728
MR_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/03-12-24-1'
2829
JA_TN_CACHE='/home/jenkins/TestData/text_norm/ci/grammars/10-17-24-1'
@@ -257,7 +258,24 @@ pipeline {
257258
}
258259
}
259260
}
260-
261+
stage('L0: Create He TN/ITN Grammars & MR') {
262+
when {
263+
anyOf {
264+
branch 'main'
265+
branch 'staging/**'
266+
branch 'staging_*'
267+
changeRequest target: 'main'
268+
}
269+
}
270+
failFast true
271+
parallel {
272+
stage('L0: HE ITN grammars') {
273+
steps {
274+
sh 'CUDA_VISIBLE_DEVICES="" python nemo_text_processing/inverse_text_normalization/inverse_normalize.py --lang=he --text="ת " --cache_dir ${HE_TN_CACHE}'
275+
}
276+
}
277+
}
278+
}
261279
stage('L0: Create HY TN/ITN Grammars & MR') {
262280
when {
263281
anyOf {
@@ -417,6 +435,11 @@ pipeline {
417435
sh 'CUDA_VISIBLE_DEVICES="" pytest tests/nemo_text_processing/hy/ -m "not pleasefixme" --cpu --tn_cache_dir ${HY_TN_CACHE}'
418436
}
419437
}
438+
stage('L1: Run all HE TN/ITN tests (restore grammars from cache)') {
439+
steps {
440+
sh 'CUDA_VISIBLE_DEVICES="" pytest tests/nemo_text_processing/he/ -m "not pleasefixme" --cpu --tn_cache_dir ${HE_TN_CACHE}'
441+
}
442+
}
420443
}
421444
}
422445

‎nemo_text_processing/inverse_text_normalization/he/__init__.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎nemo_text_processing/inverse_text_normalization/he/data/__init__.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎nemo_text_processing/inverse_text_normalization/he/data/decimals/__init__.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎nemo_text_processing/inverse_text_normalization/he/data/decimals/minutes_exception.tsv‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
חצי
2+
רבע

‎nemo_text_processing/inverse_text_normalization/he/data/measurements.tsv‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
°F פרנהייט
2+
°C צלסיוס
3+
° מעלות
4+
°F מעלות פרנהייט
5+
°C מעלות צלסיוס
6+
K קלווין
7+
% אחוז
8+
% אחוזים
9+
Hz הרץ
10+
kW קילוואט
11+
kW קילו ואט
12+
kW קילו וואט
13+
kWh קילו ואט לשעה
14+
kWh קילוואט לשעה
15+
Wh ואט לשעה
16+
W ואט
17+
ghz ג׳יגה הרץ
18+
ghz גיגה הרץ
19+
khz קילו הרץ
20+
mhz מגה הרץ
21+
v וולט
22+
nm ננומטר
23+
mA מילי אמפר
24+
tW טרה ואט
25+
mv מילי וולט
26+
mW מגה ואט
27+
μm מיקרומטר
28+
" אינץ׳
29+
cc סי סי
30+
ω אוהם
31+
db דציבל
32+
db דציבלים
33+
kb קילו ביט
34+
mb מגה ביט
35+
gb ג׳יגה ביט
36+
gb גיגה ביט
37+
tb טרה ביט
38+
pb פטה ביט
39+
mb מגה בייט
40+
kb קילו בייט
41+
gb ג׳יגה בייט
42+
gb גיגה בייט
43+
tb טרה בייט
44+
pb פטה בייט
45+
A אמפר

‎nemo_text_processing/inverse_text_normalization/he/data/months.tsv‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
ינואר
2+
פברואר
3+
מרץ
4+
מרס
5+
אפריל
6+
מאי
7+
יוני
8+
יולי
9+
אוגוסט
10+
ספטמבר
11+
אוקטובר
12+
נובמבר
13+
דצמבר

‎nemo_text_processing/inverse_text_normalization/he/data/months_name2number.tsv‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
ינואר 1
2+
פברואר 2
3+
מרץ 3
4+
אפריל 4
5+
מאי 5
6+
יוני 6
7+
יולי 7
8+
אוגוסט 8
9+
ספטמבר 9
10+
אוקטובר 10
11+
נובמבר 11
12+
דצמבר 12

‎nemo_text_processing/inverse_text_normalization/he/data/months_ordinal2number.tsv‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
ראשון 1
2+
שני 2
3+
שלישי 3
4+
רביעי 4
5+
חמישי 5
6+
שישי 6
7+
שביעי 7
8+
שמיני 8
9+
תשיעי 9
10+
עשירי 10
11+
אחת עשרה 11
12+
שתיים עשרה 12

‎nemo_text_processing/inverse_text_normalization/he/data/numbers/__init__.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

0 commit comments

Comments
 (0)