[Live session] Choosing safer LLMs: From LLM benchmarks to your production agents 🚀

July 21, 2026 | 5PM CEST

Giskard Vision: Enhance Computer Vision models for image classification, object and landmark detection

Giskard Vision is a new module in our open-source library designed to assess and improve computer vision models. It offers automated detection of performance issues, biases, and ethical concerns in image classification, object detection, and landmark detection tasks. The article provides a step-by-step guide on how to integrate Giskard Vision into existing workflows, enabling data scientists to enhance the reliability and fairness of their computer vision systems.

.png)

We’re pleased to announce the release of giskard-vision, the latest module in our open-source AI Testing library, designed specifically for computer vision tasks. The Giskard Open-Source library is designed to assess the reliability and safety of machine learning models, and identify critical issues such as biases, hallucinations, toxicity, robustness, and misinformation across different AI model types.

🎯 Giskard-Vision: Key goals and principles for Computer Vision AI

Why Giskard-Vision?

Even the most advanced vision models can have hidden flaws that impact their performance and fairness in the real world. These flaws can take the form of performance issues, biases, inconsistencies, or ethical issues that are tough to catch with standard metrics.

For example, a vision model might struggle with images exhibiting specific attributes like contrast, color, or brightness. It may also underperform on sensitive subsets of data, such as roads from specific regions in autonomous driving or medical images of particular groups of people, like the elderly or young individuals in healthcare applications.

Identifying these weaknesses is crucial for taking countermeasures to ensure the model's predictions remain safe, accurate, and reliable in all circumstances. giskard-vision offers an automated and systematic way to spot these problems early on, helping you fine-tune your models and avoid costly challenges in production.

Key features: Enhancing Computer Vision models' reliability

giskard-vision is designed to automatically detect and report:

• Performance Degradation: Identifies specific conditions or subsets of data where your model's performance may drop, allowing targeted improvements. This could take the form of bad performance under certain weather conditions.
• Fairness and Bias Detection: Scans your model for biases linked to sensitive attributes, ensuring that your model’s decisions are fair across all groups, and highlights potential ethical issues that could arise from model predictions, guiding you toward responsible AI practices. For example, this could be bad performance on elderly people.

How Giskard-Vision works

Giskard-Vision’s functionality consists of a three-step process that seamlessly integrates with your existing ML pipeline:

1. Wrap Your Dataset: To begin, wrap your dataset using custom data iterators. This allows giskard-vision to access your images, labels, and any associated metadata.
2. Wrap Your Model: Next, wrap your model to enable giskard-vision to perform scans using its predictions. This involves defining a simple interface that the library can use to interpret your model’s outputs.
3. Scan Your Model: Finally, run the automated scan, which analyzes your model’s behavior across various data points. The scan produces a comprehensive report that highlights vulnerabilities, biases, and performance issues, offering you a clear path toward improving your model.

Supported Computer Vision tasks: From object recognition to healthcare applications

giskard-vision is designed to support a wide range of vision tasks, making it a versatile tool for any computer vision model. Whether you are working with image classification, object detection, or landmark detection, giskard-vision provides tailored scanning capabilities to ensure your model meets the highest standards of performance, fairness, and ethical responsibility.

• Image Classification: For models that classify images into predefined categories, such as identifying types of animals, detecting medical conditions from scans, or sorting products in an inventory. giskard-vision can scan your model for biases that affect specific classes, identify performance issues across different image conditions, and provide insights into potential ethical concerns tied to model predictions.

• Object Detection: Used for tasks where models must identify and locate objects within an image, such as detecting cars and pedestrians in autonomous driving, defects on a production line, or faces on camera images. giskard-vision ensures your object detection models perform reliably across diverse scenes, detects biases that might cause uneven detection rates, and flags areas where model predictions could raise ethical questions.

• Landmark Detection: Ideal for applications requiring precise localization of specific points in images, such as facial landmark detection for AR applications, keypoint detection for pose estimation, or identifying specific anatomical points in medical imaging.

giskard-vision offers the flexibility to adapt its scanning procedures to each of these tasks, providing insights that are crucial for fine-tuning and improving your vision models.

Real-world Computer Vision examples

📚 Tutorial: Applying Giskard-Vision to your Computer Vision models

Ensure that you have both the base and vision libraries of Giskard installed:

pip install giskard giskard-vision

Step 1: Wrapping your Computer Vision datasets

To scan your model, the first step is to wrap your dataset using Giskard's DataIteratorBase class. This will allow you to define how data is loaded and labeled, setting up the structure needed for Giskard to understand your dataset.

from giskard_vision.core.dataloaders.base import DataIteratorBase
import numpy as np
import cv2
from typing import Optional

class DataLoaderClassification(DataIteratorBase):
    @property
    def idx_sampler(self) -> np.ndarray:
        return list(range(len(self.image_paths)))

@classmethod
    def get_image(self, idx: int) -> np.ndarray:
        return cv2.imread(str(self.image_paths[idx]))
    
    @classmethod
    def get_label(self, idx: int) -> Optional[np.ndarray]:
        return 'label'
    
    @classmethod
    def get_meta(self, idx: int) -> Optional[MetaData]:
        default_meta = super().get_meta()  # To load default metadata
        return MetaData(
            data={
                **default_meta.data,
                'meta1': 'value1',
                'meta2': 'value2',
                'categorical_meta1': 'cat_value1',
                'categorical_meta2': 'cat_value2'
            },
            categories=default_meta.categories + ['categorical_meta1', 'categorical_meta2'],
            issue_groups={
                **default_meta.issue_groups,
                'meta1': PerformanceIssueMeta,
                'meta2': EthicalIssueMeta,
                'categorical_meta1': PerformanceIssueMeta,
                'categorical_meta2': EthicalIssueMeta,
            }
        )

giskard_dataset = DataLoaderClassification()

Step 2: Wrapping your model

Once your dataset is wrapped, the next step is to wrap your model using Giskard's ModelBase class. This wrapper defines how your model interacts with the dataset during the scan.

from giskard_vision.core.models.base import ModelBase
import numpy as np

class ModelMyTask(ModelBase):
    def __init__(self, model):
        super().__init__()
        self.model = model

def predict_rgb_image(self, image: np.ndarray) -> np.ndarray:
        return self.model.predict_rgb_image(image)

def predict_gray_image(self, image: np.ndarray) -> np.ndarray:
        return self.model.predict_gray_image(image)

mymodel = ...  # Replace with your model
giskard_model = ModelMyTask(model=mymodel)

Step 3: Scanning your Computer Vision system

With your dataset and model wrapped, you can now perform a scan to identify vulnerabilities. Below is an example using a demo dataloader and an OpenCV model. Substitute these with your custom dataloader and model wrapper as needed.

from giskard_vision.image_classification.models.wrappers import SkinCancerHFModel
from giskard_vision.image_classification.dataloaders.loaders import DataLoaderSkinCancer
from giskard_vision.core.scanner import scan

# Example dataset and model
dataset = DataLoaderSkinCancer()
model = SkinCancerHFModel()

# Scan the model with the dataset
scan_results = scan(model, dataset, num_images=5)

# Display scan results (useful in notebooks)
display(scan_results)

After completing these steps, you'll receive an HTML report detailing various types of issues based on the dataset's metadata. In this example, the report highlights:

• Performance issues: where the model underperforms on non-sensitive metadata subgroups.
• Ethical issues: related to sensitive subgroups where the model shows poor performance.
• Robustness issues: where the model struggles when images are degraded (e.g., blur, noise).
• Attributes issues: where the model performs poorly on images with specific physical traits, such as contrast, brightness, or color.

This report helps you identify your model's flaws and weaknesses, enabling you to fine-tune it or address specific subgroups differently to boost overall performance and make it reliable on all your use cases.

Future perspectives

In this article we explored how to use the new giskard-vision scan to detect flaws in computer vision models. Now, you’ll be able to wrap a vision model, a dataset, and run the scan on tasks like image classification, object detection, and landmark detection to ensure your model is reliable, fair, and robust.

In the future, we plan to integrate tools for automatic slice detection based solely on images, eliminating the need to manually input metadata, and expanding the potential applications of the scan. We'll also be expanding its capabilities to include monitoring of image generation models and multimodal models.

To learn more about giskard-vision, visit our quickstart guide and our GitHub repo. Don’t hesitate to give us feedback!

Continuously secure LLM agents, preventing hallucinations and security issues.

L'Oréal leverages Giskard for advanced Facial Landmark Detection

L'Oréal has partnered with Giskard to enhance its AI models for Facial Landmark Detection. The collaboration focuses on evaluating and comparing various AI models using metrics such as Normalized Mean Error, prediction time, and robustness against image perturbations.