Meta LLaMA Inference Crash ValueError Unsupported Audio Sample Rate

Q: Root Cause

Meta LLaMA’s audio branch (e.g., LLaMAForConditionalGeneration with a Whisper‑style front‑end) is hard‑coded to accept a single sample rate of 16 kHz and mono channel audio. The underlying transformers.audio_utils module validates the incoming waveform against these constraints before feature extraction. When an audio file is recorded at 44.1 kHz or contains two channels, the validation logic throws the ValueError shown above.

Q: Option 1 – On‑the‑fly resampling in the preprocessing pipeline

Wrap the audio loading step with torchaudio or librosa to enforce the correct sample rate and channel count.

Q: Option 2 – Pre‑process files on disk

Use ffmpeg to create a canonical copy of each audio asset.

Q: Why the fix works

Both methods guarantee that the waveform passed to processor matches the sampling_rate and num_channels expected by the feature extractor. The validation step succeeds, allowing the model to compute mel‑spectrograms and continue to token generation.

In this article

Table of Contents

Problem

During local development of a Meta LLaMA inference pipeline that incorporates audio inputs, the process aborts with a ValueError: unsupported audio sample rate. The exception is raised when the transformers audio preprocessing utilities attempt to load WAV or MP3 files whose sample rate or channel layout does not match the model’s expectations.

Typical traceback observed on Ubuntu 22.04 (Python 3.11, PyTorch 2.2, Transformers 4.41.0):

Traceback (most recent call last):
  File "run_llama.py", line 78, in <module>
    inputs = processor(audio_path, return_tensors="pt")
  File ".../transformers/models/auto/processing_auto.py", line 215, in __call__
    return self.feature_extractor.__call__(*args, **kwargs)
  File ".../transformers/models/auto/feature_extraction_auto.py", line 162, in __call__
    return self._process_one(audio, **kwargs)
  File ".../transformers/models/auto/feature_extraction_auto.py", line 210, in _process_one
    raise ValueError(f"Unsupported audio sample rate: {sample_rate}")
ValueError: Unsupported audio sample rate: 44100

The crash prevents any downstream token generation and halts the development workflow.

Root Cause

Meta LLaMA’s audio branch (e.g., LLaMAForConditionalGeneration with a Whisper‑style front‑end) is hard‑coded to accept a single sample rate of 16 kHz and mono channel audio. The underlying transformers.audio_utils module validates the incoming waveform against these constraints before feature extraction. When an audio file is recorded at 44.1 kHz or contains two channels, the validation logic throws the ValueError shown above.

Why the validation exists:

Model weights were trained on 16 kHz, 1‑channel PCM; mismatched sampling introduces spectral aliasing that degrades transcription quality.
Feature extractor (e.g., Wav2Vec2FeatureExtractor) expects a fixed frame size; varying sample rates would require dynamic padding and reshape logic that is not implemented.

Debug

Step‑by‑step investigation that isolates the failure:

Confirm audio metadata using ffprobe or soxi:

ffprobe -v error -show_entries stream=sample_rate,channels \
  -of default=noprint_wrappers=1:nokey=1 sample.wav
44100
2

Inspect the processor configuration in Python:

from transformers import AutoProcessor
processor = AutoProcessor.from_pretrained("meta-llama/AudioLLaMA")
print(processor.feature_extractor.sampling_rate)
print(processor.feature_extractor.num_mel_bins)
# Output
# 16000
# 80

The processor reports a required sampling rate of 16000 Hz.

Reproduce the error with a minimal script:

import torch
from transformers import AutoProcessor

processor = AutoProcessor.from_pretrained("meta-llama/AudioLLaMA")
audio_path = "sample.wav"
try:
    inputs = processor(audio_path, return_tensors="pt")
except ValueError as e:
    print("Caught:", e)

Running the script prints the same Unsupported audio sample rate: 44100 message.

Solution

The fix consists of normalizing audio files to the required 16 kHz mono format before they reach the processor. Two approaches are common:

Option 1 – On‑the‑fly resampling in the preprocessing pipeline

Wrap the audio loading step with torchaudio or librosa to enforce the correct sample rate and channel count.

# before (failing)
inputs = processor(audio_path, return_tensors="pt")

# after (fixed)
import torchaudio

def load_and_normalize(path, target_sr=16000):
    waveform, sr = torchaudio.load(path)          # waveform: Tensor [C, T]
    if sr != target_sr:
        waveform = torchaudio.transforms.Resample(sr, target_sr)(waveform)
    # Convert to mono if needed
    if waveform.shape[0] > 1:
        waveform = waveform.mean(dim=0, keepdim=True)
    return waveform.squeeze(0).numpy()            # return 1‑D numpy array

waveform = load_and_normalize(audio_path)
inputs = processor(waveform, sampling_rate=16000, return_tensors="pt")

Option 2 – Pre‑process files on disk

Use ffmpeg to create a canonical copy of each audio asset.

# before (original file)
sample.wav  # 44.1kHz, stereo

# after (converted file)
ffmpeg -i sample.wav -ar 16000 -ac 1 sample_16k_mono.wav

Then point the inference script at the converted file.

Why the fix works

Both methods guarantee that the waveform passed to processor matches the sampling_rate and num_channels expected by the feature extractor. The validation step succeeds, allowing the model to compute mel‑spectrograms and continue to token generation.

Verify

After applying the fix, run the same minimal script. Expected output:

inputs = processor(waveform, sampling_rate=16000, return_tensors="pt")
print(inputs.input_features.shape)
# torch.Size([1, 80, 3000])   # example dimensions

Additional verification steps:

Check the model’s logits are produced without exception.
Run an end‑to‑end inference test and compare the transcription against a known reference.
Inspect the logs for the absence of ValueError: Unsupported audio sample rate.

Prevent

Implement safeguards to avoid recurrence:

Input validation wrapper that asserts sampling_rate == 16000 and channels == 1 before invoking the processor.
Automated preprocessing CI job that scans the repository for audio assets and runs ffmpeg -ar 16000 -ac 1 on any non‑conforming file.
Monitoring: Add a Prometheus metric audio_preprocess_resample_total that increments whenever a resample occurs, alerting if the rate spikes unexpectedly.
Documentation: Clearly state the required audio format in the project README and in any API contract.

Related Topic Hub: LLM Systems Troubleshooting Hub

FAQ

Why does the error appear only for some audio files?
Because only those files have a sample rate or channel count that differs from the model’s 16 kHz mono requirement. Files already recorded at 16 kHz mono bypass the validation.
Can I change the model to accept 44.1 kHz audio?
In theory, you could fine‑tune the feature extractor and retrain the encoder on 44.1 kHz data, but this requires substantial compute and is not supported out‑of‑the‑box.
Is torchaudio the only library for resampling?
No. Alternatives include librosa.resample, sox via command line, or ffmpeg. Choose the one that fits your deployment constraints.
Do I need to set sampling_rate explicitly when calling the processor?
If you pass a raw waveform (numpy array or torch tensor), you must provide sampling_rate=16000. When passing a file path, the processor extracts the rate automatically and validates it.
How can I batch multiple audio files efficiently?
Load each file with the normalization helper, stack the resulting tensors, and call the processor once with batch=True. Ensure all waveforms are padded to the same length before stacking.