Add Image and Segmentation Extraction Scripts for Waymo Dataset

[TaylorBrown96]

Overview

This initial commit introduces a set of core scripts for extracting and processing images and segmentation masks from the Waymo Open Dataset. The scripts ensure efficient data handling, conversion, and validation to support machine learning workflows.

This framework not only facilitates segmentation tasks but is also adaptable for various other applications, such as object detection and other computer vision models. By enabling the conversion of parquet files into easily usable file formats, it provides a solid foundation for training a wider range of machine learning models.

---

Added Files

1. entry.py

   • Orchestrates the entire extraction and conversion process.
   • Checks for the existence of required files and prompts for downloads if missing.
   • Converts parquet files to CSV for easier inspection.
   • Extracts images and segmentation masks, and reports the number of matching pairs.

2. extract_images.py

   • Extracts images from parquet files where corresponding segmentation keys exist.
   • Ensures efficient storage by avoiding duplicate extractions.

3. extract_segmentations.py

   • Extracts segmentation masks only if a corresponding image exists to ensure data consistency.
   • Utilizes parquet key matching for accurate alignment.

4. parquetToCSV.py

   • Converts parquet datasets into CSV format for human-readable inspection and validation.

5. schemaExtraction.py

   • Removes metadata from parquet schemas and prints them in an indented, readable format.
   • Provides row counts for both image and segmentation datasets.

6. README.md

   • Provides an overview of the project, setup instructions, and usage guidelines.
   • Includes explanations for each script and how they interact within the data processing pipeline.
   • Outlines requirements and expected input/output formats.

---

Testing & Validation

• Verified the extraction pipeline using sample Waymo parquet files.
• Ensured that only matching image-segmentation pairs are processed.
• Confirmed the correctness of the CSV outputs.
• Validated schema extraction for clarity and accuracy.
• Reviewed the README.md for completeness and clarity.