Computer Vision (CV) - Notes¶

Quick Reference¶

Definition: Computer vision is a field of artificial intelligence that enables computers to interpret and make decisions based on visual data (images, videos).
Key Use Cases: Object detection, facial recognition, autonomous driving, medical image analysis, and visual search.
Prerequisites: Basic understanding of programming (e.g., Python), familiarity with linear algebra and basic probability is helpful but not required.

Table of Contents¶

Introduction
Key Concepts
Applications
Computer Vision Problem & Challenges
Computer Vision Pipeline Architecture
Computer Vision Sub-fields/domains
CV Algorithms & Applications
Computer Vision & Deep Learning
Computer Vision & Generative AI
Computer Vision & Multimodality
CV Datasets
Hello World!
Lab: Zero to Hero Projects
Computer Vision Projects in Industry
Top Computer Vision Conferences
Computer Vision Roadmap
Computer Vision Researcher - Starting Resources
References

Introduction¶

Computer Vision is a field of artificial intelligence that enables computers to interpret and make decisions based on visual data from the real-world.

Key Concepts¶

Image Processing: Manipulating raw images to enhance or extract meaningful information.
Feature Extraction: Identifying essential patterns or characteristics from visual data (e.g., edges, textures, shapes).
Object Detection: Locating and identifying objects within an image or video.
Convolutional Neural Networks (CNNs): A type of neural network specifically designed to process and analyze visual data.
Common Misconception: Many believe that computer vision only involves recognizing objects in images, but it also encompasses tasks like image generation, segmentation, and video analysis.

Applications¶

Self-driving cars use computer vision for detecting pedestrians, traffic signals, and other vehicles.
Healthcare applies computer vision for analyzing medical images (e.g., MRI, X-rays) to diagnose diseases.
Retail benefits from facial recognition and in-store customer behavior analysis.
Manufacturing employs vision systems for quality control and defect detection.
Robotics relies on vision for navigation and manipulation in dynamic environments.
Augmented Reality (AR) uses computer vision for overlaying digital information on the real world.
Remote Sensing involves analyzing data from sensors to understand the Earth's surface and atmosphere.
And more ...

Computer Vision Problem & Challenges: Why Computer Vision is a Hard problem in AI?¶

Challenge Category	Description
Ambiguity and Interpretation	- Contextual understanding: Objects require context for interpretation. - Occlusions: Objects can be partially hidden. - Illusions: Visual illusions can mislead algorithms.
Variability and Diversity	- Object variations: Objects can appear differently due to lighting, pose, scale, or deformation. - Background clutter: Images often have distracting backgrounds. - Real-world complexity: The real world is diverse and complex.
Data Challenges	- Data acquisition: Collecting large, diverse, and high-quality datasets is difficult. - Annotation: Labeling images accurately is time-consuming and error-prone. - Data bias: Datasets can be biased, leading to biased models.
Computational Complexity	- Computational resources: Computer vision algorithms can be computationally demanding. - Real-time processing: Many applications require real-time performance.
Generalization	- Domain adaptation: Models may not generalize well to new, unseen data. - Out-of-distribution data: Models can struggle with data that is significantly different from their training data.

Computer Vision Pipeline Architecture¶

graph LR
    A[Image Acquisition fa:fa-camera-retro] --> B[Preprocessing fa:fa-cogs  ]
    B --> C[Feature Extraction]
    C --> D[Model Training / Classification]
    D --> E[Post-processing / Interpretation]

    style A font-weight:bold, fill:#CCC,stroke:#333,stroke-width:2px
    style B font-weight:bold, fill:#F88,stroke:#333,stroke-width:2px
    style C font-weight:bold, fill:#4F4,stroke:#333,stroke-width:2px
    style D font-weight:bold, fill:#99F,stroke:#333,stroke-width:2px
    style E font-weight:bold, fill:#FFF,stroke:#333,stroke-width:2px

- Image Acquisition: Collecting raw images. - Preprocessing: Preparing data by resizing, noise reduction, etc. - Feature Extraction: Identifying key visual elements. - Model Training: Teaching models to classify or recognize features. - Post-processing: Interpreting results and making decisions.

How Computer Vision Works¶

Step 1: Images or videos are captured from sensors (camera, etc.).
Step 2: Data is preprocessed—resized, normalized, or enhanced to make it suitable for analysis.
Step 3: Key features are extracted through techniques like edge detection, filtering, or using CNNs.
Step 4: Models, often neural networks, are trained on labeled datasets to classify, detect, or segment objects.
Step 5: Results are interpreted, visualized, or fed into decision-making systems.

Methods, Types & Variations¶

Image Classification: Categorizing images into predefined labels.
Object Detection: Locating objects in images and bounding them with boxes.
Semantic Segmentation: Assigning a label to every pixel in an image.
Instance Segmentation: Identifying distinct objects and outlining each separately.
Contrasting Example: While classification gives a global label (e.g., cat/dog), segmentation offers pixel-level understanding.

Tools & Frameworks¶

Category	Tools	Description
Data	- Roboflow - LandingAI - Encord - Albumentations - Kangas	Tools for data preparation, augmentation, and management in computer vision projects.
Build/Evaluate	- OpenCV - Scikit-Image - Pillow (PIL Fork) - TorchVision - MMCV - TensorFlow - Keras - MATLAB - PyTorch - Hugging Face - Caffe - Detectron2 - Anomalib - Ultralytics - Pythae - SimpleCV - BoofCV - CUDA - NVIDIA CUDA-X - NVIDIA Performance Primitives	Libraries and frameworks for building, training, and evaluating computer vision models.
Deploy & Monitoring	- OpenVINO - Comet - Viso Suite - DeepFace	Tools for deploying computer vision models and monitoring their performance in production environments.

Pros & Cons: OpenCV is fast but limited for deep learning; TensorFlow/PyTorch offer flexibility but require more setup.

Computer Vision Sub-fields/domains¶

cv-diagram

Src: Rui Nobrega

Sub-domains of computer vision include :
- scene reconstruction, - object detection, - event detection, - activity recognition, - video tracking, - object recognition, - 3D pose estimation, - learning, indexing, - motion estimation, - visual servoing, - 3D scene modeling, - image restoration - ...

Computer Vision Algorithms & Applications¶

Structure from motion algorithms - Can reconstruct a sparse 3D point model of a large complex scene from hundreds of partially overlapping photographs

Stereo matching algorithms - Can build a detailed 3D model of a building façade from hundreds of differently exposed photographs taken from the Internet

Person tracking algorithms - Can track a person walking in front of a cluttered background

Face detection algorithms - Coupled with color-based clothing and hair detection algorithms, can locate and recognize the individuals in this image

Computer Vision & Deep Learning¶

Non-exhautive list of Conv Nets (CNN) Archictectures

AlexNet, BN-AlexNet, BN-INI, ENet, GoogLeNet (Inception-v1), ResNet-18, ResNet-34, ResNet-50, ResNet-101, ResNet-152 , Inception-v3 , Inception-v4, VGG-16, VGG-19, ViT, ...

(From: Benchmark Analysis of Representative Deep Neural Network Architectures - Luigi Celona (@CeLuigi) - 2018)

Deep Learning Research Notes

Deep Learning Neural Nets
Deep Learning Neural Nets Architectures
Convolutional Neural Network (CNN) - Notebook
The State of Computer Vision at Hugging Face
Computer Vision & Keras resources

Computer Vision & Generative AI (VLMs, LVMs, ...)¶

Vision GenAI - Generative Adversarial Network (GAN) - OpenAI DALL-E - Midjourney - Stable Diffusion - Blog articles: - GenAI in Computer Vision: Transforming business with visual intelligence - crayon.com

Large Vision Models (LVMs)

Vision-LLMs (LLMs handling Visual tasks): Text-To-Image, ... - LLM-in-Vision - Yong-Lu Li(@DirtyHarryLYL) - Computer vision and Large Language Models (LLM) - How do Large Language Models (LLMs) Integrate with Computer Vision? - How do LLMs work with Vision AI? | OCR, Image & Video Analysis - It's not just words: LLMs in Computer Vision - Microsoft Reactor

Computer Vision & Multimodality¶

Vision-Language Models (VLMs): Image-To-Text, ... - A Dive into Vision-Language Models - HF - Vision Language Models Explained - HF

Computer Vision Datasets (Non-exhautive list)¶

List of datasets in computer vision and image processing - Wikipedia

SOTA Computer Vision Dataset - @paperwithcode

Top 10 Open-Source Datasets for Computer Vision in 2023 - analyticsinsight.net

ImageNet
COCO (Common Objects in Context)
Open Images
Pascal VOC (Visual Object Classes)
Cityscapes
SUN (Scene Understanding)
LFW (Labeled Faces in the Wild)
CelebA
CIFAR-10 and CIFAR-100
MNIST

Automotive datasets:

Healthcare

Hello World!¶

The example below shows how to read an image, convert it to grayscale, and display it using OpenCV.

import cv2
img = cv2.imread('image.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.imshow('Gray Image', gray)
cv2.waitKey(0)
cv2.destroyAllWindows()

Want to level up your Computer Vision skills?

Check out the Computer Vision Challenge (CVC) and dive into exciting projects that will help you master the fundamentals.

Lab: Zero to Hero Projects¶

Classify images from CIFAR-10 or MNIST datasets using a simple CNN.
Build an object detection model using YOLO (You Only Look Once) to detect objects in real-time videos.
Apply edge detection filters (e.g., Sobel, Canny) on images to enhance boundaries.
Segment a medical image (e.g., brain scan) using a basic segmentation model.
Train a GAN to generate synthetic images based on real-world datasets.

Please refer to the lab repository.