juanjucm/whisper-large-v3-turbo-OpenSLR-GL-EN

whisper-large-v3-turbo-gl-en

This model is a fine-tuned version of openai/whisper-large-v3-turbo trained on juanjucm/OpenSLR-SpeechT-GL-EN for Galician-to-English Text to Speech Translation task. It takes galician speech audios as input and generates the correspondant translated transcription in English.

The motivation behind this work is to increase the visibility of the Galician language, making it more accessible for non-Galician speakers to understand and engage with Galician audio content.

This model was developed during a 3-week Speech Translation workshop organised by Yasmin Moslem.

Performance and training details

Baseline model achieved a BLEU score of 3.38 on the evaluation dataset.

After fine-tuning, it achieves the following results on the evaluation set:

• Loss: 0.9360
• BLEU: 55.6535

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 32
• total_eval_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 3500
• mixed_precision_training: Native AMP

Training results

We used BLEU Score as our reference translation metric for selecting the best checkpoint after training.

Training Loss	Epoch	Step	Validation Loss	Bleu
0.2758	1.6667	250	0.7646	50.6055
0.0592	3.3333	500	0.7730	53.1258
0.0406	5.0	750	0.7860	53.3406
0.0173	6.6667	1000	0.8358	51.9789
0.0091	8.3333	1250	0.8909	54.4806
0.0071	10.0	1500	0.8862	54.2655
0.0039	11.6667	1750	0.9216	52.5119
0.0014	13.3333	2000	0.9281	54.5752
0.0013	15.0	2250	0.9471	54.5791
0.0009	16.6667	2500	0.9541	54.8725
0.0006	18.3333	2750	0.9614	53.1879
0.0006	20.0	3000	0.9701	54.6499
0.0006	21.6667	3250	0.9739	54.4341
0.0006	23.3333	3500	0.9747	54.5311

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0