Sample-Based Quality Assurance
Quality assurance is often sample-based, meaning only a partial picture of the entire system can be obtained.
The Juniperus project was realized with the support of the University Research Scholarship Programme (EKÖP) of the University of Pécs, within the framework of the 2025/2026 research period.
Professional support and methodological contribution for the use of audio materials and the audit framework were provided by the Gábor Aurél Research Group of the National Ambulance Service.
Juniperus
Juniperus is a research and development project focused on the AI-based, evidence-driven processing and quality auditing of emergency medical dispatch calls. The goal is not to replace human decision-making, but to create a reproducible analytical framework that enables the structured, searchable, and professionally interpretable examination of large volumes of calls.
Focus
Emergency medical dispatch, audit, clinical communication
Approach
LLM + rule-based structuring
Target Audience
Healthcare professionals and researchers
Format
Summary publication overview
1. Background and Motivation
Scalability Limitations
Manual evaluation is difficult to scale, especially with high case volumes or when multiple audit dimensions must be assessed simultaneously.
Retrieval and Structuring
Structured retrieval of communication and professional elements is often challenging even retrospectively.
Potential of AI-Assisted Evaluation
An AI-assisted system could help make evaluation more transparent, faster, and more consistent.
Why is this problem important?
The professional quality of emergency medical dispatch calls can directly influence the early stages of patient care. At the same time, retrospective auditing of calls is time-consuming, dependent on human resources, and often limited to only a small portion of the total call volume.
2. Methodology
Preparation and Segmentation
In the first step, raw audio is converted into a standardized processable format, then speech activity detection is used to isolate relevant speech segments. This reduces noise and provides a more suitable input for subsequent transcription and interpretation stages.
Diarization and Transcript
The system aims not only to determine what was said, but also who said it. Speaker separation is especially important in emergency dispatch environments, where instructions, clarifying questions, and caller responses carry different professional meanings.
Normalization and Structuring
The transcribed text is then transformed into a controlled structured format. The goal is to create a consistent, line-by-line searchable representation that preserves chronology, speaker attribution, and the auditability of textual evidence.
Evidence-Based Evaluation
In the final stage, the structured text is analyzed according to a predefined audit framework. The system may only evaluate elements positively if direct evidence exists within the call transcript, thereby reducing the risks of hallucination and retrospective overinterpretation.
A multi-step processing pipeline
Juniperus is not a black-box system built around a single model, but a chain of multiple interconnected processing stages. The emphasis is on gradual, verifiable structuring derived from raw audio material.
3. Evaluation Criteria
Itemized Evaluation
Each evaluated element appears as an independent category, such as scene safety, consciousness, breathing, communication, or advice provided.
Evidence Linking
Positive evaluation can only be assigned if a concrete quotation or clearly identifiable text segment from the call supports the given criterion.
Handling Non-Relevant Criteria
If an evaluation criterion is not applicable within the given clinical context, it must not be treated as an error, but instead categorized separately as non-relevant.
What do the evaluation criteria represent in the project?
One of the central elements of Juniperus is a structured evaluation framework that examines different dimensions of emergency dispatch communication and professional data collection through separate criteria. This is not a general impression-based scoring system, but a detailed, evidence-linked evaluation process.
4. Scientific and Practical Significance
Reproducible Analysis
Structured processing makes it possible to apply the same audit logic consistently across large numbers of calls, enabling research comparability.
Scalable Quality Assurance
The method could help extend the review of communication processes that were previously examined only through sampling.
Decision Support, Not Replacement
The system may make call materials more interpretable, but human review and clinical responsibility remain professionally central.
Why is this interesting from a research perspective?
The project exists at the intersection of emergency communication, healthcare quality assurance, and applied artificial intelligence. Its potential value is not only technological, but also methodological.
5. Limitations and Considerations
Input Quality Limitations
Audio quality, background noise, and speech intelligibility may influence the quality of the entire processing pipeline.
Interpretation Boundaries
Good structuring is not equivalent to complete professional truth; the system can only work from what was actually said.
Sensitive Data Handling
Anonymization and data handling are especially sensitive areas in emergency call processing.
Scientific Validation
Validation of the methodology requires comparison with human auditors and prospective studies.
What should not be overstated?
Juniperus is not a clinical decision-making system and does not replace professional auditors. Model-based processing remains sensitive to input quality, context, and the accuracy of the governing rule system. The core principle of the project is that the system must not improve missing professional content through assumptions, but instead separate what is provable from what cannot be verified.
6. Next Steps
Human–AI Audit Comparison
Comparison of human and AI-based audits according to agreement, discrepancy, and uncertainty.
Large-Scale Methodological Evaluation
Methodological evaluation on larger and more heterogeneous datasets.
Communication Metrics and Feedback
Development of more detailed communication metrics and feedback modules.
Research Dissemination
Transparent and publishable presentation of research-oriented results within healthcare and AI professional communities.
How could Juniperus evolve further?
The current form represents a methodological foundation and the mindset of a working prototype. The next steps point less toward technical fine-tuning and more toward validation, comparability, and professional integration.
