|
1 | 1 | # Rearrange_v2 HITL application
|
2 | 2 |
|
3 |
| -SIRo's 2024H1 data-collection app. Work in progress. |
| 3 | +Experimental application for evaluating rearrangement experiments using [partnr-planner](https://github.com/facebookresearch/partnr-planner). |
| 4 | +Supports single-user, multi-user and user-agent settings. |
4 | 5 |
|
5 |
| -## Installation |
6 |
| -See [habitat-hitl/README.md](../../../habitat-hitl/README.md). |
7 |
| - |
8 |
| -In addition to the core HITL data above, you need the private [fphab](https://huggingface.co/datasets/fpss/fphab) repo. Reach out to the SIRo team for access. |
9 |
| -``` |
10 |
| -cd path/to/habitat-lab/data |
11 |
| -git clone --branch articulated-scenes --single-branch --depth 1 https://huggingface.co/datasets/fpss/fphab |
12 |
| -mv fphab fpss |
13 |
| -``` |
14 |
| - |
15 |
| -To test the Habitat-LLM episodes in `rearrange_v2` you'll need to download and unzip the following [episode dataset](https://drive.google.com/file/d/1zFCBiWE_XFY0Ry9CZOV_NF_rfxBw1y-F/view?usp=sharing) in Habitat-Lab root directory. In addition, you'll need YCB, GSO, AI2THOR, and ABO object assets. To download these assets use the following commands: |
16 |
| - |
17 |
| -``` |
18 |
| -cd data |
19 |
| -git clone https://huggingface.co/datasets/ai-habitat/OVMM_objects objects --recursive |
20 |
| -cd objects |
21 |
| -git checkout 3893a735352b92d46505f35d759553f5fc82a39b |
22 |
| -``` |
23 |
| - |
24 |
| -## Data directory |
25 |
| - |
26 |
| -Run `rearrange_v2` from the Habitat-lab root directory. It will expect `data/` for Habitat-lab data, and it will also look for `examples/hitl/rearrange_v2/app_data/demo.json.gz` (included alongside source files in our git repo). |
27 |
| - |
28 |
| -## Example launch commands |
29 |
| - |
30 |
| -Local testing (local display and keyboard control): |
31 |
| -```bash |
32 |
| -python examples/hitl/rearrange_v2/rearrange_v2.py |
33 |
| -``` |
34 |
| - |
35 |
| -Headed server (includes local display and keyboard control): |
36 |
| -```bash |
37 |
| -python examples/hitl/rearrange_v2/rearrange_v2.py habitat_hitl.networking.enable=True |
38 |
| -``` |
39 |
| - |
40 |
| -Headless server: |
41 |
| -```bash |
42 |
| -python examples/hitl/rearrange_v2/rearrange_v2.py +experiment=headless_server |
43 |
| -``` |
44 |
| - |
45 |
| -To test Habitat-LLM episodes using a user-controlled humanoid use: |
46 |
| -```bash |
47 |
| -python examples/hitl/rearrange_v2/rearrange_v2.py --config-name lang_rearrange_humanoid_only |
48 |
| -``` |
49 |
| - |
50 |
| -## Controls |
51 |
| -See on-screen help text. In addition, press `1` or `2` to select an episode. |
52 |
| - |
53 |
| -## Configuration |
54 |
| -See `config/rearrange_v2.yaml` and `config/experiment/headless_server.yaml`. |
55 |
| - |
56 |
| -## Browser client |
57 |
| - |
58 |
| -`rearrange_v2` has additional requirements for the [Unity VR client](../pick_throw_vr/README.md#vr). |
59 |
| -* Beware these instructions are a work in progress! |
60 |
| -* It is designed for use with a desktop browser-based (not VR) version of the Unity client. |
61 |
| -* Use the [`webgl-demo`](https://github.com/eundersander/siro_hitl_unity_client/tree/webgl-demo) branch. |
62 |
| -* Download Unity data folder (`data.zip`) from: https://drive.google.com/drive/folders/12VLJGf5_ntr5nztZn1KjzyzBG_UKDPme |
63 |
| -* Extract `data.zip` to Unity client's `Assets/Resources/data`. |
64 |
| -* Open and run the `PlayerMouseKeyboard` scene. |
| 6 | +Work-in-progress. Please check back later for updates. |
0 commit comments