📘 Conference Paper (To Appear) • 📝 arXiv Paper • 📦 Download Datasets • 📄 LVVO Dataset arXiv Paper

This repository contains the source code for the research paper:
"Visual Content Detection in Educational Videos with Transfer Learning and Dataset Enrichment"
by Dipayan Biswas, Shishir Shah, and Jaspal Subhlok (University of Houston)

This work presents a deep learning framework for detecting visual objects—such as tables, charts, images, and illustrations—in lecture video frames using transfer learning and dataset enrichment. Evaluated on three datasets (LDD, LPM, and the newly introduced LVVO), six object detection models were fine-tuned, with YOLOv11 achieving the best performance. The model was further optimized through cross-dataset training and a semi-supervised auto-labeling pipeline, demonstrating that transfer learning and data enrichment significantly improve detection accuracy under limited annotation.

• Introduced the LVVO dataset comprising 4,000 annotated lecture video frames.
• Benchmarked six state-of-the-art object detection models across LVVO, LDD, and LPM datasets.
• Addressed the challenge of generalization through cross-dataset training and analysis on diverse educational video sources.
• Boosted model accuracy with limited labeled data using a semi-supervised auto-labeling pipeline.

We utilize three annotated datasets: LVVO (4,000 frames, introduced in this work), LDD, and LPM.
🔗 See the LVVO Dataset Repository for details and downloads.

git clone https://github.com/dipayan1109033/edu-video-visual-detection.git
cd edu-video-visual-detection

cd src/utils
git clone https://github.com/dipayan1109033/calculate_ODmetrics.git

Set up a virtual environment and install dependencies from requirements.txt.

Download the dataset from the LVVO Dataset Repository, place the zip files in data/processed/, and unzip them:

cd data/processed
unzip dataset_name.zip

This project supports training both YOLOv11 and torchvision-based models (e.g., Faster R-CNN, RetinaNet, FCOS) using either manual split ratios or predefined dataset splits.
Predefined splits can be generated using the following script:

python src/prepare/setup_experiment.py

✅ YOLOv11 Training with split ratios

python src/main.py model.identifier="yolo" model.pretrained_model="yolo11m.pt" exp.mode="train" exp.name="train_yolo_LVVO1k" data.folder="LVVO_1k" data.split_ratios="[0.8,0.2,0.0]" train.lr=0.001 train.epoch=30

✅ YOLOv11 Training with split code

python src/main.py model.identifier="yolo" model.pretrained_model="yolo11m.pt" exp.mode="train" exp.name="train_yolo_csplitLVVO4k" data.split_code="LVVO_4k_val200_seed42" train.lr=0.001 train.epoch=30

✅ Torchvision Model Cross-validation (e.g., Faster R-CNN)

python src/main.py model.identifier="rcnn" model.code=33 exp.mode="crossval" exp.name="crossval_rcnn_LVVO1k" data.folder="LVVO_1k" data.num_folds=5 train.lr=0.001 train.epoch=30

✅ YOLOv11 Cross-validation with split code

python src/main.py model.identifier="yolo" model.pretrained_model="yolo11m.pt" exp.mode="crossval" exp.name="crossval_yolo_csplitLVVO4k" data.split_code="LVVO_4k_val200_cv5_seed42" train.lr=0.001 train.epoch=30

Note: Table 1 reports the numerical results visualized in Figure 2 of the paper.

Note: Table 2 reports the numerical results visualized in Figure 3 of the paper.

Note: Table 3 reports the numerical results visualized in Figure 4 of the paper.

Note: Table 4 provides detailed results corresponding to Table II in the paper.

If you use this code or dataset, please cite:

@inproceedings{biswas2025visualcontent,
  title     = {Visual Content Detection in Educational Videos with Transfer Learning and Dataset Enrichment},
  author    = {Biswas, Dipayan and Shah, Shishir and Subhlok, Jaspal},
  booktitle = {Proceedings of the IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR)},
  year      = {2025},
  note      = {To appear}
}

This project is licensed under the MIT License.
See the LICENSE file for details.