Autonomous Target Tracking (ATT) systems, especially ATT drones, are widely used in applications such as surveillance, border control, and law enforcement, while also being misused in stalking and destructive actions. Thus, the security of ATT is highly critical for real-world applications. Under the scope, we present a new type of attack: distance-pulling attacks (DPA) and a systematic study of it, which exploits vulnerabilities in ATT systems to dangerously reduce tracking distances, leading to drone capturing, increased susceptibility to sensor attacks, or even physical collisions. To achieve these goals, we present FlyTrap, a novel physical-world attack framework that employs an adversarial umbrella as a deployable and domain-specific attack vector. In this artifact, we provide the instructions to reproduce the main results in the paper to support our main claim and main contribution.

• Installation
• Reproduction
  • Attack Evaluation
    • FlyTrap Evaluation
    • TGT Evaluation
  • Defense Evaluation
    • PercepGuard Evaluation
    • VOGUES Evaluation
• Acknowledgments

Please refer to the INSTALL.md for the installation instructions.

All the time estimation is based on a single NVIDIA RTX 3090 GPU. You can run each of the following experiments in parallel.

We provide three options to evaluate the FlyTrap attack based on the available resources and time requirements.

• Option 1: Full evaluation (~5 hours):

bash scripts/eval_flytrap.sh

• Option 2: Partial evaluation (~1 hour):

You can select the following config files inside config/final:

python tools/main.py <config_file>

• Option 3: Pre-computed results (~10 minutes):

Download the pre-computed results from the above link. You can also download using the terminal command:

bash download/download_flytrap_results.sh

Finally, run the following command to compute the detailed metric:

• Option 1 Full Evaluation:

If you run the full evaluation above (option 1) or if you downloaded the pre-computed results (option 3), you can run the following command to compute the detailed metric.

bash scripts/metric_summary.sh

• Option 2 Partial Evaluation:

If you run the partial evaluation above (option 2), you can run the following command to compute the detailed metric. Please replace <config> with the config file name you evaluated above.

python analysis/analyze_result_metric.py --file work_dirs/<config>/json_files/results_epoch-1.json

We also provide three options to evaluate the TGT attack based on the available resources and time requirements.

• Option 1: Full Evaluation (~40 hours):

Run the following command to evaluate the effectiveness and universality of TGT Images; all the TGT baseline attack patches are located in tgt:

bash scripts/eval_tgt.sh

• Option 2: Partial Evaluation (~10 hours):

You can specify the config file from the following list, depending on the victim model you want to evaluate.

• config/final/mixformer_tgt.py
• config/final/siamrpn_alex_tgt.py
• config/final/siamrpn_mob_tgt.py
• config/final/siamrpn_resnet_tgt.py

bash scripts/eval_tgt_partial.sh <config_file>

• Option 3: Pre-computed results (~10 minutes):

Download the pre-computed results from the above link. You can also download using the terminal command:

bash download/download_tgt_results.sh

If you run the full evaluation above (option 1) or if you downloaded the pre-computed results (option 3), you can run the following command to compute the detailed metric. If you run the partial evaluation above (option 2), please use the specific <config_file> you evaluated above.

python analysis/analyze_tgt_metric.py --input_dir work_dirs/mixformer_tgt/json_files
python analysis/analyze_tgt_metric.py --input_dir work_dirs/siamrpn_alex_tgt/json_files
python analysis/analyze_tgt_metric.py --input_dir work_dirs/siamrpn_mob_tgt/json_files
python analysis/analyze_tgt_metric.py --input_dir work_dirs/siamrpn_resnet_tgt/json_files

We provide the code to evaluate the defense methods: Security'23 PercepGuard and Security'24 VOGUES.

Run the following command one by one to evaluate the defense of FlyTrap_ATG and vanilla FlyTrap. The results will be saved in work_dirs/percepguard_results: before means the false alarm rate and after means the true alarm rate after the attack.

# Evaluation MixFormer
bash scripts/eval_percepguard.sh config/final/mixformer_percepguard.py patches/mixformer_flytrap_atg_percepguard.png
bash scripts/eval_percepguard.sh config/final/mixformer_percepguard.py patches/mixformer_flytrap_pdp.png

# Evaluation SiamRPN-ResNet
bash scripts/eval_percepguard.sh config/final/siamrpn_resnet_percepguard.py patches/siamrpn_resnet_flytrap_atg_percepguard.png
bash scripts/eval_percepguard.sh config/final/siamrpn_resnet_percepguard.py patches/siamrpn_resnet_flytrap_pdp.png

# Evaluation SiamRPN-Mobile
bash scripts/eval_percepguard.sh config/final/siamrpn_mob_percepguard.py patches/siamrpn_mobile_flytrap_atg_percepguard.png
bash scripts/eval_percepguard.sh config/final/siamrpn_mob_percepguard.py patches/siamrpn_mobile_flytrap_pdp.png

# Evaluation SiamRPN-Alex
bash scripts/eval_percepguard.sh config/final/siamrpn_alex_percepguard.py patches/siamrpn_alex_flytrap_atg_percepguard.png
bash scripts/eval_percepguard.sh config/final/siamrpn_alex_percepguard.py patches/siamrpn_alex_flytrap_pdp.png

Run the following command one by one to evaluate FlyTrap_ATG and vanilla FlyTrap. The results will be saved in work_dirs/vogues_results

• In attack case: before means the false alarm rate before the attack and after means the true alarm rate after the attack.

• In benign case: before means the false alarm rate without the umbrella and after means the false alarm rate with the umbrella.

# Evaluation MixFormer
bash scripts/eval_vogues.sh config/final/mixformer_vogues.py patches/mixformer_flytrap_atg_vogues.png
bash scripts/eval_vogues.sh config/final/mixformer_vogues.py patches/mixformer_flytrap_pdp.png

# Evaluation SiamRPN-ResNet
bash scripts/eval_vogues.sh config/final/siamrpn_resnet_vogues.py patches/siamrpn_resnet_flytrap_atg_vogues.png
bash scripts/eval_vogues.sh config/final/siamrpn_resnet_vogues.py patches/siamrpn_resnet_flytrap_pdp.png

# Evaluation SiamRPN-Mobile
bash scripts/eval_vogues.sh config/final/siamrpn_mob_vogues.py patches/siamrpn_mobile_flytrap_atg_vogues.png
bash scripts/eval_vogues.sh config/final/siamrpn_mob_vogues.py patches/siamrpn_mobile_flytrap_pdp.png

# Evaluation SiamRPN-Alex
bash scripts/eval_vogues.sh config/final/siamrpn_alex_vogues.py patches/siamrpn_alex_flytrap_atg_vogues.png
bash scripts/eval_vogues.sh config/final/siamrpn_alex_vogues.py patches/siamrpn_alex_flytrap_pdp.png

We thank the following projects for their contributions:

• AlphaPose
• PySOT
• MixFormer
• PercepGuard