[BFCL] Package the Codebase (#565)

This PR aims to improve the organization and distribution of the
codebase by packaging the BFCL codebase. This PR is part of a series of
changes that break down the tasks outlined in #510.

---------

Co-authored-by: Huanzhi Mao <[email protected]>

Author: devanshamin

berkeley-function-call-leaderboard/README.md

@@ -12,15 +12,34 @@ We introduce the Berkeley Function Leaderboard (BFCL), the **first comprehensive
 Read more about the technical details and interesting insights in our [blog post](https://gorilla.cs.berkeley.edu/blogs/8_berkeley_function_calling_leaderboard.html)!
 
 ![image](./architecture_diagram.png)
+
+
 ### Install Dependencies
 
 ```bash
+# Create a new Conda environment with Python 3.10
 conda create -n BFCL python=3.10
+
+# Activate the new environment
 conda activate BFCL
-pip install -r requirements.txt # Inside gorilla/berkeley-function-call-leaderboard
-pip install vllm==0.5.0 # If you have vLLM supported GPU(s) and want to run our evaluation data against self-hosted OSS models.
+
+# Clone the Gorilla repository
+git clone https://github.com/ShishirPatil/gorilla.git
+
+# Change directory to the berkeley-function-call-leaderboard
+cd gorilla/berkeley-function-call-leaderboard
+
+# Install the package in editable mode
+pip install -e .
 ```
 
+### Installing Extra Dependencies for Self-Hosted Open Source Models
+
+To do LLM generation on self-hosted open source models, you need to run the following command to install the extra dependencies.
+```bash
+pip install -e .[oss_eval]
+```
+Note that this requires GPU supported by vLLM and it can only be installed on Linux and Windows (not Mac).
 
 ## Execution Evaluation Data Post-processing (Can be Skipped: Necesary for Executable Test Categories)
 Add your keys into `function_credential_config.json`, so that the original placeholder values in questions, params, and answers will be reset.
@@ -189,7 +208,7 @@ In the following two sections, the optional `--test-category` parameter can be u
 
 ### Running the Checker
 
-Navigate to the `gorilla/berkeley-function-call-leaderboard/eval_checker` directory and run the `eval_runner.py` script with the desired parameters. The basic syntax is as follows:
+Navigate to the `gorilla/berkeley-function-call-leaderboard/bfcl/eval_checker` directory and run the `eval_runner.py` script with the desired parameters. The basic syntax is as follows:
 
 ```bash
 python eval_runner.py --model MODEL_NAME --test-category TEST_CATEGORY

berkeley-function-call-leaderboard/apply_function_credential_config.py

@@ -2,7 +2,7 @@
 import json
 import argparse
 import os
-from eval_checker import custom_exception
+from bfcl.eval_checker import custom_exception
 
 parser = argparse.ArgumentParser(description="Replace placeholders in the function credential config file.")
 parser.add_argument("--input-path", help="Path to the function credential config file. Can be a file or a directory.")

berkeley-function-call-leaderboard/bfcl/__init__.py

@@ -1,4 +1,4 @@
-from model_handler.constant import (
+from bfcl.model_handler.constant import (
     UNDERSCORE_TO_DOT,
     JAVA_TYPE_CONVERSION,
     JS_TYPE_CONVERSION,

berkeley-function-call-leaderboard/bfcl/eval_checker/__init__.py

@@ -1,7 +1,3 @@
-import sys
-
-sys.path.append("../")
-
 from checker import ast_checker, exec_checker, executable_checker_rest
 from custom_exception import BadAPIStatusError
 from eval_runner_helper import *
@@ -440,10 +436,10 @@ def runner(model_names, test_categories, api_sanity_check):
     )
 
 
-INPUT_PATH = "../result/"
-PROMPT_PATH = "../data/"
-POSSIBLE_ANSWER_PATH = "../data/possible_answer/"
-OUTPUT_PATH = "../score/"
+INPUT_PATH = "../../result/"
+PROMPT_PATH = "../../data/"
+POSSIBLE_ANSWER_PATH = "../../data/possible_answer/"
+OUTPUT_PATH = "../../score/"
 
 # A dictionary to store the results
 # Key is model name, value is a dictionary with keys as test category and values as a dictionary with accuracy and total count