AI

Getting OpenAI Whisper Running: A Setup and Transcription Walkthrough.

AI Note-Taker

— 11 min read

Unlocking OpenAI's Whisper: A Complete User Manual

By Alex Chen | Updated: 2024-11-15 | 6 mins read

Contents

From groundbreaking models like GPT to visual creators like DALL-E, and now Whisper, OpenAI has consistently pushed the boundaries of artificial intelligence, delivering some of the most impressive and useful tools available. Their audio processing model, Whisper, stands out as a transcription tool that challenges competitors in terms of speed, cost-effectiveness, and precision.

Despite its acclaimed capabilities, a common hurdle remains: many potential users are unsure how to get started. The absence of a straightforward download process, typical of most software, can be a significant deterrent. Through research, common feedback includes sentiments like, "It seems overly technical!" and "Navigating through extensive developer documentation is daunting!" If you've faced these challenges, this guide provides a clear, step-by-step approach to using OpenAI's Whisper.

Contents

What is OpenAI's Whisper?

Whisper is an automated speech recognition (ASR) system developed by OpenAI, the same organization behind ChatGPT and DALL-E. This project is open-source, which means it is freely available for use, distribution, and modification.

Unlike conventional speech-to-text software, Whisper doesn't offer a direct download link or installer. Its source files are hosted in a GitHub repository. To install and utilize it, you'll need to download specific developer tools and execute code commands within your system's terminal.

Who is OpenAI Whisper For?

Anyone requiring the conversion of spoken language into written text can benefit from Whisper AI. For instance:

  • A student needing to transcribe lecture recordings.
  • A project manager aiming to extract key points from a recorded online meeting.
  • A podcaster intending to repurpose audio content into different textual formats.
  • A video content creator looking to generate subtitles for their videos, and many others.

While Whisper excels at transcription, users often require further functionalities like AI-generated summaries or visual mind maps, which can be found in comprehensive platforms such as DeepVo.ai.

How to Download and Set Up Whisper

Firstly, it's crucial to recognize that Whisper operates differently from typical transcription and translation tools. There isn't a dedicated website with a ready-to-download installer. To set it up, a foundational knowledge of the command line interface (CLI) for Windows, Linux, or macOS is necessary, depending on your operating system.

This tutorial outlines a step-by-step method for installing Whisper on a Windows system for offline operation. To begin, certain prerequisite software must be present on your computer to guarantee a seamless download and installation process.

Prerequisites:

  • Python
  • Git
  • Rust
  • NVIDIA CUDA (Recommended, but optional)
  • Pip (Typically included with recent Python versions)
  • PyTorch
  • FFmpeg

Python

For this setup, we'll utilize Python version 3.9.9, though Whisper is compatible with versions ranging from 3.7 to 3.11. Navigate to the official Python website and select your desired Python version based on its release. For this example, Python 3.9.9 is chosen. Click on it, scroll to the files section, and download the installer appropriate for your system architecture. The download should commence immediately. After downloading, run the installer. If installing Python for the first time, ensure you check the "Add Python to PATH" option on the installer's initial page. This enables running Python from any terminal window. Overlooking this step can lead to failure during the Whisper installation.

Git

Since OpenAI Whisper's files are located in a GitHub repository, you must download, configure, and install Git on your system to access these files. Go to the Git for Windows website and pick an installer that matches your device.

Rust

Installing Rust can help prevent errors during the build process for tokenizers, a specific dependency when installing Python-based projects. There are two primary methods to install Rust:

  1. Visit Rust's official website and download the installer suitable for your system.
  2. Alternatively, open your command prompt (CMD) and execute the command: pip install setuptools-rust. (To open CMD, press Windows Key + R, type "cmd", and click "Run".)

NVIDIA CUDA (Optional)

If you've worked with AI tools previously, you're likely aware of the substantial computational power they demand. Consequently, running AI applications on systems equipped with NVIDIA GPUs and having NVIDIA CUDA installed is highly advantageous. CUDA enhances the GPU's processing capabilities, making them more efficient for data processing than standard GPUs. CUDA can only be installed on systems with NVIDIA GPUs. However, this doesn't preclude using Whisper on CPU-only devices. As detailed later, Whisper offers various models (tiny, base, small, medium, large), with higher models requiring more computational power. Thus, users with both CPU and GPU setups can utilize Whisper. If your device supports NVIDIA CUDA, visit the NVIDIA website and download the latest CUDA version compatible with PyTorch. At the time of writing, PyTorch supports CUDA 11.7 and 11.8.

Pip

Pip is Python's package installer and management tool, essential for managing PyPI installations via the command line. Modern Python versions usually come with Pip pre-installed. However, if you're using an older version, you'll need to install it. To verify Pip's presence, open your command prompt and run: pip help. A response indicates Pip is installed. If you receive an error, you must install it. Visit https://pip.pypa.io/en/stable/installation/ for a detailed guide.

PyTorch

PyTorch is a deep learning library frequently used for applications leveraging GPUs and CPUs, favored by developers for its speed and implementation flexibility. To install it, go to the PyTorch website and select your installation preferences based on your system (OS, Package, Language, Compute Platform). Once configured, a command will be provided. Copy and execute this command in your CMD interface to download PyTorch. Note: If using a GPU, select CUDA 11.7 or 11.8. Choose CPU if your device lacks an NVIDIA graphics card.

FFmpeg

FFmpeg is a vital tool in this setup, as it facilitates converting audio into a format that Whisper can process. To download it:

  1. Go to the FFmpeg official website to obtain the authentic files.
  2. Scroll to the Windows icon and click it. Select one of the provided builds (e.g., "Windows builds by BtbN").
  3. This will redirect you to a page with various FFmpeg assets. Scroll and choose the one matching your system (e.g., the larger Win64gpl build). Click to download the ZIP archive.
  4. Extract the files to a folder. Inside the 'bin' subfolder, you'll find three application files.
  5. To make these accessible system-wide, create a new folder in your Local Disk C: (e.g., named "Path"). Copy the three FFmpeg applications into this "C:\Path" folder.
  6. Click at the top of the File Explorer window showing "C:\Path" to copy this file path.
  7. Next, click the Start button, search for "Edit environment variables for your account," and open it.
  8. In the Environment Variables window, under "User variables," select "Path" and click "Edit...".
  9. Click "New," paste the copied file path (C:\Path), and click "OK" on all open dialog boxes.
  10. To confirm successful installation, open a new command prompt window and type ffmpeg. If the installation was successful, you'll see version information and other details.

Install Whisper

With all prerequisites in place, you can now install Whisper:

  1. Open your command console (CMD) and run the following command:pip install git+https://github.com/openai/whisper.git
  2. Two outcomes are possible:
    • The installation completes successfully.
    • You might encounter an error like "cannot find command 'git'". This signifies that pip cannot locate Git on your system and thus cannot connect to the Whisper repository. To resolve this, ensure Git is installed (e.g., from Git for Windows) and that its installation directory is added to your system's PATH. During Git installation, there's usually an option to update the PATH automatically. After fixing, re-run the pip install command.
  3. Once installation is complete, you can run Whisper from any command interface. To see its options, type:whisperThis will display available languages and other parameters like model selection and output format. For more detailed help on commands:whisper -h

Note: If you encounter an error stating "whisper is not recognized as an internal or external command," you may need to add Python's script directory to your system's PATH. This directory is typically located within your Python installation folder (e.g., C:\Users\YourUser\AppData\Local\Programs\Python\Python39\Scripts).

How to Record Audio on Your Computer

The installation was the most challenging part. From here on, the process is much simpler. To record your voice on Mac or Windows, you can use free software like Audacity or built-in tools such as Voice Memos on Mac or Voice Recorder on Windows. For web-based recording, various online tools are also available.

For optimal recording quality, ensure you:

  • Use a decent microphone.
  • Record in a quiet environment with minimal background noise.

When using a tool like Audacity:

  1. Download the software from its official website.
  2. Open Audacity and connect your microphone.
  3. In "Audio Setup," select your microphone as the recording device for clearer audio.
  4. Click the Record icon (red circle) to begin recording. Once finished, click the Stop button.
  5. Navigate to File > Export and save your recording in a common format like MP3, WAV, or OGG.

Once you have your audio file, you can proceed to transcribe it with Whisper. For users seeking an integrated experience beyond transcription, including features like automated summaries or structured mind maps from their audio, platforms like DeepVo.ai can directly process these audio files and offer these advanced functionalities.

How to Convert Speech to Text Using Whisper

Now that you have an audio file, you can transcribe it using Whisper.

  1. Save the audio file you wish to transcribe into a dedicated folder. For this example, let's name the folder "MyAudioTranscriptions".
  2. Open a new command prompt window directly from this folder. An easy way to do this in Windows is to navigate to the folder in File Explorer, then type "cmd" in the address bar and press Enter.
  3. In the command prompt window, type whisper followed by the name of your audio file. If the filename contains spaces, enclose it in quotation marks. For example:whisper "my meeting audio.mp3"Or, to specify a model (e.g., 'base'):whisper "my lecture.wav" --model base

The transcription process will commence. The time required for completion will vary based on:

  • The length and size of your audio file.
  • The processing speed of your GPU or CPU.

Whisper's Performance Metrics

OpenAI's Whisper is recognized as one of the most accurate speech recognition models available. Its accuracy can be assessed in a couple of ways:

Analyzing Transcription Quality

Whisper's developers state that the model was trained on 680,000 hours of diverse, multilingual data. This extensive training contributes to its high accuracy in both transcription and translation into English. This rigorous training has enhanced Whisper's robustness and its capability to handle various accents, as well as filter out background noise and technical interferences.

A comparative analysis of Word Error Rate (WER) from research papers, which measures the differences between Whisper and other leading speech recognition models, indicates that Whisper generally outperforms other open-source models like NVIDIA's STT across multiple datasets. As evident from such comparisons, Whisper often leads in terms of accuracy among many language models.

However, it's important to note that only a few languages achieve a Word Error Rate below 5%, while over 25 languages have a WER of 50% or higher. Despite this, Whisper frequently makes significantly fewer errors than many other models. Note: AI speech technology is continually evolving, and Whisper AI, while advanced, is not flawless. Some areas where it might show limitations include:

  • Occasional omission of punctuation.
  • Incorrect transcription of certain words or failure to transcribe some segments.
  • Lack of speaker diarization (distinguishing between different speakers).
  • No native real-time transcription capability for the local installation; it primarily focuses on zero-shot asynchronous transcription. To use OpenAI Whisper online with real-time features, one would typically use the Whisper API (which may have associated costs).

While Whisper excels in performance, accuracy remains a consideration for all language models, especially when processing non-English languages or highly specialized terminologies.

Whisper Speech Recognition Languages

Whisper can transcribe audio in up to 99 languages and translate any of them into English. According to OpenAI's research, languages like Spanish, Italian, English, and Portuguese are among the most accurately transcribed, often achieving a word error rate below 5%.

Here is a general distribution of how languages compare by their word error rates, based on OpenAI's findings:

Number of Languages Word Error Rate
4 < 5%
9 5% - 10%
19 10% - 20%
11 20% - 30%
4 30% - 40%
6 40% - 50%
11 50% - 90%
18 90% - 200%

Cost to Run Whisper

The primary advantage of using Whisper locally is that it's free! You can run Whisper on your own machine without any registration or subscription fees. However, there's an implicit cost: it requires your time and system resources to install and operate the software. Given that OpenAI doesn't offer direct ongoing support or integration assistance for the open-source version, encountering technical issues could lead to operational delays.

Furthermore, to achieve optimal performance from Whisper, using a device with a capable GPU is recommended. Whisper offers five different models for transcription: Tiny, Base, Small, Medium, and Large. Each model has varying VRAM (Video RAM) requirements for efficient operation: Tiny and Base models need approximately 1 GB of VRAM each, Small requires about 2GB, Medium around 5GB, and the Large model demands about 10 GB. Greater processing power generally leads to faster results. Ideally, an Nvidia GPU (such as a GTX 970 or any more recent version) will serve well. It's important to distinguish speed from accuracy; while larger models process audio faster and utilize more GPU resources, they are not inherently more accurate than smaller models on clear audio, though they tend to be more robust to noise.

A User-Friendly Alternative to Whisper - DeepVo.ai

As demonstrated, Whisper AI is a formidable tool in terms of transcription accuracy. However, its command-line nature, setup complexity, limited built-in features beyond raw transcription, and potential for encountering unassisted errors can be drawbacks for some users. Additionally, users with CPU-only systems might not experience its full speed potential.

For individuals and teams seeking a more accessible, feature-rich, and user-friendly solution that still delivers high precision, DeepVo.ai emerges as a compelling alternative. DeepVo.ai is designed to simplify the entire workflow from audio/video to actionable insights.

Key features that make DeepVo.ai stand out include:

  • High-Precision Transcription: DeepVo.ai boasts impressive accuracy, often reaching up to 99.5%, and supports transcription for over 100 languages.
  • AI-Powered Summaries: Beyond just text, DeepVo.ai can generate intelligent summaries of your transcribed content in mere seconds. It often offers customizable templates to tailor summaries for different needs (e.g., meeting notes, research highlights).
  • Intelligent Mind Maps: A unique offering is the automatic creation of smart mind maps from your transcriptions. This feature provides a structured, visual representation of key topics and their relationships, aiding comprehension and recall. These mind maps can typically be exported as images.
  • User-Friendly Interface: Unlike Whisper's command-line operation, DeepVo.ai usually provides an intuitive web-based platform, eliminating complex setup procedures.
  • Fast Turnaround: Transcription and subsequent processing like summaries and mind maps are delivered rapidly.
  • Accessibility and Security: DeepVo.ai often offers a free tier for users to try its services and emphasizes security with features like end-to-end encryption for your data.

To use DeepVo.ai, you generally:

  1. Visit the DeepVo.ai website and sign up or log in.
  2. Upload your audio or video file.
  3. The platform will process your file, providing the transcription, and often options for generating summaries and mind maps.
  4. You can then edit, export, or share your results as needed.

Final Thoughts

At first glance, OpenAI's Whisper AI might appear to be a tool exclusively for technically proficient individuals, but it is, in reality, quite manageable once set up. The primary challenge typically lies in the initial installation process. While the steps can seem technical, by carefully following this guide, you should be able to navigate the setup successfully.

Keep in mind that a local Whisper AI installation is tied to the specific device on which you install it. If you require a tool that offers cross-device compatibility, a more user-friendly interface, and integrated features like AI summaries and mind mapping, while still maintaining high accuracy comparable to advanced models, consider exploring DeepVo.ai. It offers a streamlined and powerful solution for a wide range of transcription and content analysis needs.

Read more