hw4_tokenizer_1k / README.md
alexgichamba's picture
Update README.md
6fc3e8f verified
metadata
library_name: transformers
tags: []

Tokenizer

A tokenizer with a vocab size of 1k for Intro to Deep Learning Homework 4 on Language Modelling and Automatic Speech Recognition.

The tokenizer was trained on LibriSpeech LM text