Benchmarks

Gigatoken is a new open-source tokenizer that is ~1000x faster than HuggingFace's tokenizers and ~100x faster than OpenAI's tiktoken for most tokenizer definitions on most machines.

Apple's new SpeechAnalyzer API has been benchmarked against Whisper and its predecessor, showing significant improvements in accuracy and speed. The results have major implications for developers and users relying on speech-to-text transcription.