Falcon Speaker Diarization: Usage Guide

What Falcon speaker diarization answers

Falcon speaker diarization is the process of taking an audio recording and answering one practical question: who spoke when? The system does not know the speakers in advance, and it does not identify them by name. It separates the speech into speaker turns, groups segments that appear to belong to the same voice, and assigns each group a temporary label such as Speaker A or Speaker B. The output is a timeline of anonymous speaker labels aligned with the transcript or the audio timestamps.

This task is often confused with related speech technologies. Speaker diarization is not speech recognition: transcription converts speech into words, while diarization answers which speaker produced each segment. Speaker diarization is also not speaker identification or verification. Identification compares a voice against a known enrolled identity; verification accepts or rejects a claimed identity. Diarization stops at grouping similar voices and does not equate an anonymous label with a verified personal identity unless an external identification step supplies that evidence.

How diarization fits into the Falcon pipeline

A typical Falcon speech pipeline separates the jobs before combining them into a useful result. Audio is decoded into words and timing information. In parallel, the diarization component produces speaker boundaries and speaker groups. The final assembly maps the anonymous speaker labels onto the transcript segments, producing a speaker-attributed transcript.

Input audio
    |
    +-- Speech recognition ---> words with timestamps
    |
    +-- Speaker diarization ---> speaker segments and clusters
                                    |
                                    v
                      Speaker-attributed transcript

Core stages: segmentation and clustering

Diarization is usually implemented as two connected stages. Segmentation splits the audio into short sections and detects where the speaker changes. Clustering takes those sections, represents each one as a voice embedding, and merges sections whose embeddings are close enough to be treated as the same speaker.

A voice embedding is a fixed-length vector that summarizes the acoustic characteristics of a short speech segment. Segments from the same person should produce similar embeddings, while different voices should occupy different regions of the embedding space. The clustering step decides how to convert those distances into a small number of speaker groups.

Reading and using speaker-attributed output

The final speaker-attributed transcript typically contains a start time, an end time, the recognized text, and an anonymous speaker label. This is enough to reconstruct the conversation flow and to search for what each speaker said.

start  end    speaker  text
00:00  02:14  Speaker A  Let's review the release plan.
02:14  04:02  Speaker B  I can share the current status.
04:02  05:30  Speaker A  Good, start with the open issues.

When consuming this output, treat Speaker A and Speaker B as labels that are consistent within one diarization run, not as portable identities. The same person may be assigned a different label in another file or another run. If the application needs named participants, add a separate identification step that maps labels to verified identities using evidence outside the diarization output.

Choosing configuration settings

The main opening decision is whether the number of speakers is known in advance. Supplying an expected speaker count can help clustering choose the correct number of groups in recordings with clearly separated turns. In open conversations where participants enter or leave, or where the count is uncertain, an unsupervised clustering mode lets the system infer an appropriate number of groups.

Audio conditions also affect performance. Clean recordings with one active speaker at a time and limited background noise are usually easier to segment. Overlapping speech, music, low-bandwidth telephony, and very short turns can make speaker-change detection less reliable. Where the audio is known to be difficult, first improving source quality or separating channels is often more effective than tuning clustering thresholds alone.

Evaluating diarization quality

Diarization is evaluated with the diarization error rate, or DER. DER combines three kinds of mistakes: missed speech, false alarm speech, and speaker confusion. Speaker confusion counts how much speech is assigned to the wrong speaker label. Lower DER means better agreement with the reference speaker labels.

A common companion metric is speaker-attributed word error rate, which measures errors in both the recognized words and their speaker assignment. This is useful when the end product is a speaker-attributed transcript rather than a standalone diarization timeline.

Any numerical performance claim must be tied to its dataset, metric, and evaluation conditions. A DER measured on clean meeting audio with a known speaker count does not predict the same score on noisy telephone calls or broadcast material. Benchmark results should be read as evidence about a specific setup, not as universal expectations for every deployment.

Troubleshooting common diarization failures

• Split speakers: one person is labeled as two speakers, often because of changing microphone distance, emotionally varied speech, or short separated turns. Supplying an expected speaker count or tuning clustering can help.
• Merged speakers: two people are combined under one label, often when their voices are similar or turns are short. Stricter clustering or better source separation can reduce this.
• Boundary errors: the speaker change is detected early or late, usually during fast turn-taking or overlapping speech. Checking the audio for overlap and using a segmentation model suited to the domain can improve the boundaries.
• Too many or too few clusters: the inferred speaker count is unstable. If the number of participants is known, pass it to the clustering stage; otherwise validate the cluster count against a held-out sample.

When a transcript appears correct but the diarization is unstable, inspect the audio conditions before changing model parameters. Many failures trace back to recording quality, channel differences, or conversation style rather than to a diarization setting.

Data and identity limitations

Falcon diarization produces anonymous speaker separation, not verified identification. Do not present Speaker A as a named individual on the basis of diarization alone. If downstream systems attach names, keep that mapping separate and clearly evidence-based.

The guide does not include benchmark figures here because no dataset, metric, and evaluation-condition details are supplied in this task. Without those conditions, a performance number could mislead. Use the Falcon documentation, release notes, or reproducible evaluation outputs for the specific model version and audio domain you plan to deploy.

Start with a known use case: choose whether the speaker count is known, run diarization on representative audio, inspect the timeline for split or merged speakers, and evaluate with DER or speaker-attributed word error rate against a labeled reference. That loop gives a practical answer to whether Falcon diarization is suitable for the target audio.