tools/StreamingTDSModelConverter.cpp?

[Adportas]

Hi,
Where is that file -> tools/StreamingTDSModelConverter.cpp
I need it to transform the model so I can transcribe ... but I can't find it in the github repository of w2l
Thank's in advance
Daniel

[abhinavkulkarni]

@Adportas: The inference code has been moved to v0.2 branch. You can find StreamingTDSModelConverter.cpp under tools directory.

[Adportas]

@Adportas: The inference code has been moved to v0.2 branch. You can find StreamingTDSModelConverter.cpp under tools directory.

Hi @abhinavkulkarni
Thank's for the reference.
Now i got v0.2 branch with the command: git clone -b v0.2 https://github.com/facebookresearch/wav2letter
And following the "General Build Instructions": https://github.com/facebookresearch/wav2letter/wiki/General-building-instructions
I set this flag W2L_BUILD_TOOLS to ON in CMakeLists.txt in order to obtain the program StreamingTDSModelConverter

The build of StreamingTDSModelConverter fail with this error:

/home/empresa/wav2letter_v0.2/tools/StreamingTDSModelConverter.cpp:24:10: fatal error: inference/module/feature/feature.h: File or directory does not exist (No existe el archivo o el directorio)
#include "inference/module/feature/feature.h"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The path is really bad because the inference folder is duplicated: /home/empresa/wav2letter_v0.2/inference/inference/module/feature/feature.h

I tried various ways to fix the path's errors, in the end the one that worked best was adding this:
include_directories(/home/empresa/wav2letter_v0.2/src /home/empresa/wav2letter_v0.2/inference/)
To that file: /home/empresa/wav2letter_v0.2/tools/CMakeLists.txt

Then I get this error from a truly non-existent file

In file included from /home/empresa/wav2letter_v0.2/inference/inference/module/nn/backend/fbgemm/fbgemm.h:11:0,
from /home/empresa/wav2letter_v0.2/inference/inference/module/nn/nn.h:23,
from /home/empresa/wav2letter_v0.2/tools/StreamingTDSModelConverter.cpp:26:
/home/empresa/wav2letter_v0.2/inference/inference/module/nn/backend/fbgemm/Conv1dFbGemm.h:15:10: fatal error: fbgemm/FbgemmFP16.h: No existe el archivo o el directorio
#include <fbgemm/FbgemmFP16.h>
^~~~~~~~~~~~~~~~~~~~~

It will be that I must use some special configuration option somewhere?

Thank's in advance
Daniel

[abhinavkulkarni]

@Adportas: The following are the CMake options for my build:

-DW2L_BUILD_TESTS=OFF -DW2L_INFERENCE_BUILD_TESTS=OFF -DW2L_BUILD_TOOLS=ON -DW2L_BUILD_INFERENCE=ON -DW2L_INFERENCE_BACKEND=fbgemm

Also, make sure you have the right version of Flashlight (when in doubt either use the Docker container or the instructions in the Dockerfile and replicate them on your machine). There's something around checking out a specific branch of Flashlight and installing contrib packages - just follow instructions in the Dockerfile, if you are having trouble.

Yes, the inference folder is duplicated and you shouldn't have to de-duplicate it.

The repository and the branch v0.2 work as is if built correctly.

[Adportas]

@Adportas: The following are the CMake options for my build:

-DW2L_BUILD_TESTS=OFF -DW2L_INFERENCE_BUILD_TESTS=OFF -DW2L_BUILD_TOOLS=ON -DW2L_BUILD_INFERENCE=ON -DW2L_INFERENCE_BACKEND=fbgemm

Also, make sure you have the right version of Flashlight (when in doubt either use the Docker container or the instructions in the Dockerfile and replicate them on your machine). There's something around checking out a specific branch of Flashlight and installing contrib packages - just follow instructions in the Dockerfile, if you are having trouble.

Yes, the inference folder is duplicated and you shouldn't have to de-duplicate it.

The repository and the branch v0.2 work as is if built correctly.

Hi @abhinavkulkarni
Thank's for the parameters and Docker reference.
Docker is new to me, it took me a bit to understand it and choose the right image of the 3 available, finally the CPU type image allow me to generate the executable, but when using it I got some errors:

I1124 13:37:44.484884 16905 StreamingTDSModelConverter.cpp:192] Number of classes (network): 45
Skipping View module: V -1 1 NFEAT 0
terminate called after throwing an instance of 'std::invalid_argument'
what(): stoi
*** Aborted at 1606225064 (unix time) try "date -d @1606225064" if you are using GNU date ***
PC: @ 0x7f1b97733e97 gsignal
*** SIGABRT (@0x4209) received by PID 16905 (TID 0x7f1ba2b68840) from PID 16905; stack trace: ***
@ 0x7f1b98a8a890 (unknown)
@ 0x7f1b97733e97 gsignal
@ 0x7f1b97735801 abort

Which seem to be referenced in that previous post with broken usefull links inside unfortunately.

streaming_tds_model_converter error #646

Summary -> "We support only TDS based modules for streaming inference at the moment"

# 8 Temporal Convolutions followed by 2 Linear layers with ReLU activations.
# First Conv layer has stride of 2. All Conv layers perform "SAME" padding.
V -1 1 NFEAT 0
C2 NFEAT 256 8 1 2 1 -1 -1
R
C2 256 256 8 1 1 1 -1 -1
R
C2 256 256 8 1 1 1 -1 -1
R
C2 256 256 8 1 1 1 -1 -1
R
C2 256 256 8 1 1 1 -1 -1
R
C2 256 256 8 1 1 1 -1 -1
R
C2 256 256 8 1 1 1 -1 -1
R
C2 256 256 8 1 1 1 -1 -1
R
RO 2 0 3 1
L 256 512
R
L 512 NLABEL

I guess it would not be tds and I have to change it to that mode, I will try with this model I hope it works

V -1 NFEAT 1 0
C2 1 10 21 1 2 1 -1 -1
R
DO 0.2
LN 3
TDS 10 21 80 0.2
TDS 10 21 80 0.2
C2 10 14 21 1 2 1 -1 -1
R
DO 0.2
LN 3
TDS 14 21 80 0.2
TDS 14 21 80 0.2
TDS 14 21 80 0.2
C2 14 18 21 1 2 1 -1 -1
R
DO 0.2
LN 3
TDS 18 21 80 0.2
TDS 18 21 80 0.2
TDS 18 21 80 0.2
TDS 18 21 80 0.2
TDS 18 21 80 0.2
TDS 18 21 80 0.2
V 0 1440 1 0
RO 1 0 3 2
L 1440 1024

Finally i will leave the steps i use to compile and try the streaming_tds_model_converter in case they help:

Install Docker
sudo docker run --rm -itd --ipc=host --name w2l wav2letter/wav2letter:cpu-latest
sudo docker exec -it w2l bash
export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.5.274/linux/mkl/lib/intel64:$LD_IBRARY_PAT
export MKLROOT=/opt/intel/mkl
export KENLM_ROOT_DIR=/root/kenlm
cmake -DW2L_BUILD_TESTS=OFF -DW2L_INFERENCE_BUILD_TESTS=OFF -DW2L_BUILD_TOOLS=ON -DW2L_BUILD_INFERENCE=ON -DW2L_INFERENCE_BACKEND=fbgemm ..
...
[100%] Building CXX object tools/CMakeFiles/streaming_tds_model_converter.dir/__/inference/inference/module/ModuleProcessingState.cpp.o
[100%] Linking CXX executable streaming_tds_model_converter
[100%] Built target streaming_tds_model_converter

./streaming_tds_model_converter -am Daniel/001_model_last.bin --outdir Daniel/Resultado

NOTE: I had to recreate some paths and copy referenced files of the model in the docker image with combination of "mkdir -p" and "sudo docker cp"

[abhinavkulkarni]

@Adportas: Not sure if you were able to run the TDS converter, but here's a sample run from my machine:

$ /home/w2luser/Projects/wav2letter/cmake-build-debug-fbgemm/tools/streaming_tds_model_converter --am /data/podcaster/model/wav2letter/am_500ms_future_context_dev_other/am_500ms_future_context_dev_other.bin --outdir /home/w2luser/models --flagsfile /home/w2luser/Projects/wav2letter/recipes/models/streaming_convnets/librispeech/train_am_500ms_future_context.cfg --logtostderr=1
I1203 21:45:45.544514 35115 StreamingTDSModelConverter.cpp:152] [Network] Reading acoustic model from /data/podcaster/model/wav2letter/am_500ms_future_context_dev_other/am_500ms_future_context_dev_other.bin
I1203 21:45:46.513530 35115 StreamingTDSModelConverter.cpp:157] [Network] Sequential [input -> (0) -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> (7) -> (8) -> (9) -> (10) -> (11) -> (12) -> (13) -> (14) -> (15) -> (16) -> (17) -> (18) -> (19) -> (20) -> (21) -> (22) -> (23) -> (24) -> (25) -> (26) -> (27) -> (28) -> (29) -> (30) -> (31) -> (32) -> (33) -> (34) -> (35) -> (36) -> (37) -> (38) -> (39) -> output]
	(0): View (-1 80 1 0)
	(1): SpecAugment ( W: 80, F: 27, mF: 2, T: 100, p: 1, mT: 2 )
	(2): Padding (0, { (5, 3), (0, 0), (0, 0), (0, 0), })
	(3): Conv2D (1->15, 10x1, 2,1, 0,0, 1, 1) (with bias)
	(4): ReLU
	(5): Dropout (0.100000)
	(6): LayerNorm ( axis : { 1 2 } , size : -1)
	(7): Time-Depth Separable Block (9, 80, 15) [1200 -> 1200 -> 1200]
	(8): Time-Depth Separable Block (9, 80, 15) [1200 -> 1200 -> 1200]
	(9): Padding (0, { (7, 1), (0, 0), (0, 0), (0, 0), })
	(10): Conv2D (15->19, 10x1, 2,1, 0,0, 1, 1) (with bias)
	(11): ReLU
	(12): Dropout (0.100000)
	(13): LayerNorm ( axis : { 1 2 } , size : -1)
	(14): Time-Depth Separable Block (9, 80, 19) [1520 -> 1520 -> 1520]
	(15): Time-Depth Separable Block (9, 80, 19) [1520 -> 1520 -> 1520]
	(16): Time-Depth Separable Block (9, 80, 19) [1520 -> 1520 -> 1520]
	(17): Padding (0, { (9, 1), (0, 0), (0, 0), (0, 0), })
	(18): Conv2D (19->23, 12x1, 2,1, 0,0, 1, 1) (with bias)
	(19): ReLU
	(20): Dropout (0.100000)
	(21): LayerNorm ( axis : { 1 2 } , size : -1)
	(22): Time-Depth Separable Block (11, 80, 23) [1840 -> 1840 -> 1840]
	(23): Time-Depth Separable Block (11, 80, 23) [1840 -> 1840 -> 1840]
	(24): Time-Depth Separable Block (11, 80, 23) [1840 -> 1840 -> 1840]
	(25): Time-Depth Separable Block (11, 80, 23) [1840 -> 1840 -> 1840]
	(26): Padding (0, { (10, 0), (0, 0), (0, 0), (0, 0), })
	(27): Conv2D (23->27, 11x1, 1,1, 0,0, 1, 1) (with bias)
	(28): ReLU
	(29): Dropout (0.100000)
	(30): LayerNorm ( axis : { 1 2 } , size : -1)
	(31): Time-Depth Separable Block (11, 80, 27) [2160 -> 2160 -> 2160]
	(32): Time-Depth Separable Block (11, 80, 27) [2160 -> 2160 -> 2160]
	(33): Time-Depth Separable Block (11, 80, 27) [2160 -> 2160 -> 2160]
	(34): Time-Depth Separable Block (11, 80, 27) [2160 -> 2160 -> 2160]
	(35): Time-Depth Separable Block (11, 80, 27) [2160 -> 2160 -> 2160]
	(36): Reorder (2,1,0,3)
	(37): View (2160 -1 1 0)
	(38): Linear (2160->9998) (with bias)
	(39): View (9998 0 -1 1)
I1203 21:45:46.513664 35115 StreamingTDSModelConverter.cpp:158] [Criterion] ConnectionistTemporalClassificationCriterion
I1203 21:45:46.513669 35115 StreamingTDSModelConverter.cpp:159] [Network] Number of params: 115111823
I1203 21:45:46.513698 35115 StreamingTDSModelConverter.cpp:165] [Network] Updating flags from config file: /data/podcaster/model/wav2letter/am_500ms_future_context_dev_other/am_500ms_future_context_dev_other.bin
I1203 21:45:46.514137 35115 StreamingTDSModelConverter.cpp:174] Gflags after parsing 
--flagfile=; --fromenv=; --tryfromenv=; --undefok=; --tab_completion_columns=80; --tab_completion_word=; --help=false; --helpfull=false; --helpmatch=; --helpon=; --helppackage=false; --helpshort=false; --helpxml=false; --version=false; --adambeta1=0.90000000000000002; --adambeta2=0.999; --am=/data/podcaster/model/wav2letter/am_500ms_future_context_dev_other/am_500ms_future_context_dev_other.bin; --am_decoder_tr_dropout=0; --am_decoder_tr_layerdrop=0; --am_decoder_tr_layers=1; --arch=am_500ms_future_context.arch; --archdir=/home/w2luser/Projects/wav2letter/recipes/models/streaming_convnets/librispeech; --attention=content; --attentionthreshold=0; --attnWindow=no; --attnconvchannel=0; --attnconvkernel=0; --attndim=0; --batchsize=8; --beamsize=2500; --beamsizetoken=250000; --beamthreshold=25; --blobdata=false; --channels=1; --criterion=ctc; --critoptim=sgd; --datadir=; --dataorder=input; --decoderattnround=1; --decoderdropout=0; --decoderrnnlayer=1; --decodertype=wrd; --devwin=0; --emission_dir=; --emission_queue_size=3000; --enable_distributed=true; --encoderdim=0; --eosscore=0; --eostoken=false; --everstoredb=false; --fftcachesize=1; --filterbanks=80; --flagsfile=/home/w2luser/Projects/wav2letter/recipes/models/streaming_convnets/librispeech/train_am_500ms_future_context.cfg; --framesizems=25; --framestridems=10; --gamma=1; --gumbeltemperature=1; --input=flac; --inputbinsize=100; --inputfeeding=false; --isbeamdump=false; --iter=1000000; --itersave=false; --labelsmooth=0; --leftWindowSize=50; --lexicon=/home/w2luser/w2l/am/librispeech-train+dev-unigram-10000-nbest10.lexicon; --linlr=-1; --linlrcrit=-1; --linseg=0; --lm=; --lm_memory=5000; --lm_vocab=; --lmtype=kenlm; --lmweight=0; --localnrmlleftctx=300; --localnrmlrightctx=0; --logadd=false; --lr=0.40000000000000002; --lr_decay=9223372036854775807; --lr_decay_step=9223372036854775807; --lrcosine=false; --lrcrit=0; --max_devices_per_node=8; --maxdecoderoutputlen=200; --maxgradnorm=0.5; --maxisz=33000; --maxload=-1; --maxrate=10; --maxsil=50; --maxtsz=9223372036854775807; --maxword=-1; --melfloor=1; --memstepsize=10485760; --mfcc=false; --mfcccoeffs=13; --mfsc=true; --minisz=200; --minrate=3; --minsil=0; --mintsz=2; --momentum=0; --netoptim=sgd; --noresample=false; --nthread=6; --nthread_decoder=1; --nthread_decoder_am_forward=1; --numattnhead=8; --onorm=target; --optimepsilon=1e-08; --optimrho=0.90000000000000002; --outputbinsize=5; --pctteacherforcing=100; --pcttraineval=1; --pow=false; --pretrainWindow=0; --replabel=0; --reportiters=1000; --rightWindowSize=50; --rndv_filepath=/checkpoint/vineelkpratap/experiments/speech/inference_tds//inference_paper_500ms_do0.1_lr0.4_G32_archtds_k10s_d8_p100m_do0.1_saug_mln_500ms.arch_bch8/rndvz.21621542; --rundir=[...]; --runname=inference_2019; --samplerate=16000; --sampletarget=0; --samplingstrategy=rand; --saug_fmaskf=27; --saug_fmaskn=2; --saug_start_update=-1; --saug_tmaskn=2; --saug_tmaskp=1; --saug_tmaskt=100; --sclite=; --seed=0; --show=false; --showletters=false; --silscore=0; --smearing=none; --smoothingtemperature=1; --softwoffset=10; --softwrate=5; --softwstd=5; --sqnorm=true; --stepsize=1000000; --surround=; --tag=; --target=tkn; --test=; --tokens=librispeech-train-all-unigram-10000.tokens; --tokensdir=/home/w2luser/w2l/am; --train=/home/w2luser/w2l/lists/train-clean-100.lst,/home/w2luser/w2l/lists/train-clean-360.lst,/home/w2luser/w2l/lists/train-other-500.lst,[DATA_DST_librilight]/lists/librilight.lst; --trainWithWindow=false; --transdiag=0; --unkscore=-inf; --use_memcache=false; --uselexicon=true; --usewordpiece=true; --valid=dev-clean:/home/w2luser/w2l/lists/dev-clean.lst,dev-other:/home/w2luser/w2l/lists/dev-other.lst; --validbatchsize=-1; --warmup=1; --weightdecay=0; --wordscore=0; --wordseparator=_; --world_rank=0; --world_size=32; --outdir=/home/w2luser/models; --alsologtoemail=; --alsologtostderr=false; --colorlogtostderr=false; --drop_log_memory=true; --log_backtrace_at=; --log_dir=; --log_link=; --log_prefix=true; --logbuflevel=0; --logbufsecs=30; --logemaillevel=999; --logfile_mode=436; --logmailer=/bin/mail; --logtostderr=true; --max_log_size=1800; --minloglevel=0; --stderrthreshold=2; --stop_logging_if_full_disk=false; --symbolize_stacktrace=true; --v=0; --vmodule=; 
I1203 21:45:46.533406 35115 StreamingTDSModelConverter.cpp:192] Number of classes (network): 9998
Skipping View module: V -1 NFEAT 1 0
Skipping SpecAugment module: SAUG 80 27 2 100 1.0 2
Skipping Dropout module: DO 0.1
Skipping Dropout module: DO 0.1
Skipping Dropout module: DO 0.1
Skipping Dropout module: DO 0.1
Skipping Reorder module: RO 2 1 0 3
Skipping View module: V 2160 -1 1 0
Skipping View module: V NLABEL 0 -1 1
I1203 21:45:54.853623 35115 StreamingTDSModelConverter.cpp:289] Serializing acoustic model to '/home/w2luser/models/acoustic_model.bin'
I1203 21:46:00.994504 35115 StreamingTDSModelConverter.cpp:301] Writing tokens file to '/home/w2luser/models/tokens.txt'
I1203 21:46:01.005759 35115 StreamingTDSModelConverter.cpp:328] Serializing feature extraction model to '/home/w2luser/models/feature_extractor.bin'
I1203 21:46:01.710419 35115 StreamingTDSModelConverter.cpp:344] verifying serialization ...
I1203 21:46:03.194089 35115 StreamingTDSModelConverter.cpp:372] Done !

Process finished with exit code 0

[xiaosdawn]

Hi, I have tried streaming_tds_model_converter.
Here are my steps:

1. use the Docker container to build flashlight

   docker build -f .docker/Dockerfile-CUDA -t fl:81c4d8d .
   nvidia-docker run -t -i -v /data/w2l:/app/w2l-share --name=fl-81c4d8d fl:81c4d8d /bin/bash
   

2. clone wav2letter in the container above
3. edit /root/wav2letter/recipes/CMakeLists.txt, add line:

   add_subdirectory(${CMAKE_CURRENT_LIST_DIR}/streaming_convnets/tools)
   

   edit /root/wav2letter/recipes/streaming_convnets/tools/CMakeLists.txt, add line:

   include_directories(/root/wav2letter/recipes/streaming_convnets/inference/)
   

4. cd wav2letter && mkdir build && cd build && cmake .. -DW2L_BUILD_TOOLS=ON
   But, I got errors the same as @Adportas :

In file included from /root/wav2letter/recipes/streaming_convnets/inference/inference/module/nn/backend/fbgemm/fbgemm.h:10:0,
                 from /root/wav2letter/recipes/streaming_convnets/inference/inference/module/nn/nn.h:22,
                 from /root/wav2letter/recipes/streaming_convnets/tools/StreamingTDSModelConverter.cpp:29:
/root/wav2letter/recipes/streaming_convnets/inference/inference/module/nn/backend/fbgemm/Conv1dFbGemm.h:14:10: fatal error: fbgemm/FbgemmFP16.h: No such file or directory
 #include <fbgemm/FbgemmFP16.h>
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
recipes/streaming_convnets/tools/CMakeFiles/streaming_tds_model_converter.dir/build.make:62: recipe for target 'recipes/streaming_convnets/tools/CMakeFiles/streaming_tds_model_converter.dir/StreamingTDSModelConverter.cpp.o' failed
make[2]: *** [recipes/streaming_convnets/tools/CMakeFiles/streaming_tds_model_converter.dir/StreamingTDSModelConverter.cpp.o] Error 1
CMakeFiles/Makefile2:219: recipe for target 'recipes/streaming_convnets/tools/CMakeFiles/streaming_tds_model_converter.dir/all' failed
make[1]: *** [recipes/streaming_convnets/tools/CMakeFiles/streaming_tds_model_converter.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Then, I change the cmake parameters @abhinavkulkarni suggested.

cmake .. -DW2L_BUILD_TESTS=OFF -DW2L_INFERENCE_BUILD_TESTS=OFF -DW2L_BUILD_TOOLS=ON -DW2L_BUILD_INFERENCE=ON -DW2L_INFERENCE_BACKEND=fbgemm

But, the same error comes up again.

In addtion, I tried wav2letter v0.2

1. clone in the container from step1
2. cd wav2letter-v0.2 && mkdir build && cd build && cmake .. -DW2L_BUILD_TOOLS=ON
   I got errors as:

-- Found SndFile: /usr/local/include
CMake Error at /usr/local/share/flashlight/cmake/FindSndFile.cmake:65 (get_target_property):
  get_target_property() called with non-existent target "Vorbis::vorbis".
Call Stack (most recent call first):
  /usr/local/share/cmake-3.10/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
  /usr/local/share/flashlight/cmake/flashlightConfig.cmake:57 (find_dependency)
  CMakeLists.txt:99 (find_package)


CMake Error at /usr/local/share/flashlight/cmake/FindSndFile.cmake:66 (get_target_property):
  get_target_property() called with non-existent target "Vorbis::vorbisenc".
Call Stack (most recent call first):
  /usr/local/share/cmake-3.10/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
  /usr/local/share/flashlight/cmake/flashlightConfig.cmake:57 (find_dependency)
  CMakeLists.txt:99 (find_package)


CMake Error at /usr/local/share/flashlight/cmake/FindSndFile.cmake:68 (get_target_property):
  get_target_property() called with non-existent target "Ogg::ogg".
Call Stack (most recent call first):
  /usr/local/share/cmake-3.10/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
  /usr/local/share/flashlight/cmake/flashlightConfig.cmake:57 (find_dependency)
  CMakeLists.txt:99 (find_package)


-- Found libsndfile: (lib: /usr/local/lib/libsndfile.so include: /usr/local/include
-- The CUDA compiler identification is NVIDIA 10.0.130
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- flashlight found (include:  lib: flashlight::flashlight )
CMake Error at CMakeLists.txt:103 (message):
  flashlight must be build in distributed mode for wav2letter++


-- Configuring incomplete, errors occurred!
See also "/root/wav2letter-0.2/build/CMakeFiles/CMakeOutput.log".

Maybe I have missed something. Any help will be appreciate. Thanks.

[abhinavkulkarni]

@xiaosdawn: Have you checked if any of the pre-built Docker images streaming_tds_model_converter binary built? The CPU/CUDA ones for training should have that pre-built (or you may be able to build the binary by switching on a particular CMake option).

[xiaosdawn]

I didn't find any pre-built Docker images streaming_tds_model_converter binary built? Could you give me some advice or the link for getting the specific image? Thanks very much.

[abhinavkulkarni]

@xiaosdawn: You may want to check pre-built Docker images detailed here: https://github.com/facebookresearch/wav2letter/wiki/Building-Running-with-Docker

The ones used for training should have the binary built.

[xiaosdawn]

Ok, thanks.

[xiaosdawn]

Thank you @abhinavkulkarni .
I use wav2letter/wav2letter:cuda-latest image to build a container.
Then

cd /root/wav2letter/build
cmake .. -DW2L_BUILD_TOOLS=ON && make -j8

So the streaming_tds_model_converter binary is built at build/tools/streaming_tds_model_converter.

Thanks again.

[xiaosdawn]

Hi, sorry to have bothered you again.
After I built the streaming_tds_model_converter binary, I start to convert my well-trained ConvNet model. But I got errors:

E0115 07:58:02.565189  4068 Serial.h:77] Error while loading "/app/w2l-share/AM_rundir/am_convnet_inference_2019/001_model_dev-clean.bin": Trying to load an unregistered polymorphic type (fl::SpecAugment).
Make sure your type is registered with CEREAL_REGISTER_TYPE and that the archive you are using was included (and registered with CEREAL_REGISTER_ARCHIVE) prior to calling CEREAL_REGISTER_TYPE.
If your type is already registered and you still see this error, you may need to use CEREAL_REGISTER_DYNAMIC_INIT.
terminate called after throwing an instance of 'cereal::Exception'
  what():  Trying to load an unregistered polymorphic type (fl::SpecAugment).
Make sure your type is registered with CEREAL_REGISTER_TYPE and that the archive you are using was included (and registered with CEREAL_REGISTER_ARCHIVE) prior to calling CEREAL_REGISTER_TYPE.
If your type is already registered and you still see this error, you may need to use CEREAL_REGISTER_DYNAMIC_INIT.
*** Aborted at 1610697482 (unix time) try "date -d @1610697482" if you are using GNU date ***
PC: @     0x7fa535b3be97 gsignal
*** SIGABRT (@0xfe4) received by PID 4068 (TID 0x7fa57b00b3c0) from PID 4068; stack trace: ***
    @     0x7fa53d839890 (unknown)
    @     0x7fa535b3be97 gsignal
    @     0x7fa535b3d801 abort
    @     0x7fa536530957 (unknown)
    @     0x7fa536536ab6 (unknown)
    @     0x7fa536536af1 std::terminate()
    @     0x7fa536536d79 __cxa_rethrow
    @     0x55ca685f92bb main
    @     0x7fa535b1eb97 __libc_start_main
    @     0x55ca686678ca _start
Aborted (core dumped)

Hope you show me some advice continuely. Any suggestion will be appreciate.

Here is my train.cfg details:

--runname=am_convnet_inference_2019
--rundir=/app/w2l-share/AM_rundir
--archdir=/app/w2l-share/configs
--arch=am_arch/am_500ms_future_context.arch
--tokensdir=/app/w2l-share/Lexicon-and-Tokens/Download
--tokens=librispeech-train-all-unigram-10000.tokens
--lexicon=/app/w2l-share/Lexicon-and-Tokens/Download/librispeech-train+dev-unigram-10000-nbest10.lexicon
--train=/app/w2l-share/lists/train-clean-100.lst,/app/w2l-share/lists/train-clean-360.lst,/app/w2l-share/lists/train-other-500.lst
--valid=dev-clean:/app/w2l-share/lists/dev-clean.lst,dev-other:/app/w2l-share/lists/dev-other.lst
--criterion=ctc
--batchsize=8
--lr=0.4
--momentum=0.0
--maxgradnorm=0.5
--reportiters=1000
--nthread=6
--mfsc=true
--usewordpiece=true
--wordseparator=_
--filterbanks=80
--minisz=200
--mintsz=2
--maxisz=33000
--enable_distributed=true
--pcttraineval=1
--minloglevel=0
--logtostderr
--onorm=target
--sqnorm
--localnrmlleftctx=300
--lr_decay=10000

and the arch details as:

V -1 NFEAT 1 0
SAUG 80 27 2 100 1.0 2
PD 0 5 3
C2 1 15 10 1 2 1 0 0
R
DO 0.1
LN 1 2
TDS 15 9 80 0.1 0 1 0
TDS 15 9 80 0.1 0 1 0
PD 0 7 1
C2 15 19 10 1 2 1 0 0
R
DO 0.1
LN 1 2
TDS 19 9 80 0.1 0 1 0
TDS 19 9 80 0.1 0 1 0
TDS 19 9 80 0.1 0 1 0
PD 0 9 1
C2 19 23 12 1 2 1 0 0
R
DO 0.1
LN 1 2
TDS 23 11 80 0.1 0 1 0
TDS 23 11 80 0.1 0 1 0
TDS 23 11 80 0.1 0 1 0
TDS 23 11 80 0.1 0 0 0
PD 0 10 0
C2 23 27 11 1 1 1 0 0
R
DO 0.1
LN 1 2
TDS 27 11 80 0.1 0 0 0
TDS 27 11 80 0.1 0 0 0
TDS 27 11 80 0.1 0 0 0
TDS 27 11 80 0.1 0 0 0
TDS 27 11 80 0.1 0 0 0
RO 2 1 0 3
V 2160 -1 1 0
L 2160 NLABEL
V NLABEL 0 -1 1

[tlikhomanenko]

cc @vineelpratap could you navigate here with latest wav2letter/flashlight?

Otherwise @xiaosdawn you needd to use docker built at time of branch v0.2 before we did refactor of the whole code base and merged flashlight and wav2letter.