update reproducibility and add model configurations for derpp_star, er_ace_star, er_star, and xder_rpc_star

Author: loribonna

REPRODUCIBILITY.md

@@ -23,7 +23,7 @@ The following table contains a list of models implemented in Mammoth and a check
 | ER-ACE LiDER              | `er_ace_aer_abs`         | V         |       |
 | ER-ACE with AER and ABS   | `er_ace_lider`           | V         |       |
 | ER-ACE with Tricks        | `er_ace_tricks`          | N/A       | This method does not come from a paper. |
-| ER-ACE & STAR             | `er_ace_star`            | X         |       |
+| ER-ACE & STAR             | `er_ace_star`            | V         |       |
 | EwC Online                | `ewc_on`                 | X         |       |
 | ER                        | `er`                     | N/A       | This method predates modern benchmarks (R. Ratcliff. "Connectionist models of recognition memory: constraints imposed by learning and forgetting functions", 1990 -- A. Robins, "Catastrophic forgetting, rehearsal and pseudorehearsal", 1995) |
 | ER & STAR                 | `er_star`                | V         |       |
@@ -62,4 +62,4 @@ The following table contains a list of models implemented in Mammoth and a check
 | X-DER                     | `xder`                   | V         |       |
 | X-DER with CE             | `xder_ce`                | V         |       |
 | X-DER with RPC            | `xder_rpc`               | V         |       |
-| X-DER with RPC & STAR     | `xder_rpc_star`          | X         |       |
+| X-DER with RPC & STAR     | `xder_rpc_star`          | V         |       |

models/config/derpp_star.yaml

@@ -0,0 +1,8 @@
+seq-cifar10:
+  100:
+    alpha: 0.15
+    beta: 0.15
+    lr: 0.1
+    model: derpp_star
+    p_gamma: 0.05
+    p_lam: 0.05

models/config/er_ace_star.yaml

@@ -0,0 +1,7 @@
+seq-cifar10:
+  100:
+    lr: 0.03
+    model: er_ace_star
+    p_gamma: 0.01
+    p_lam: 0.1
+    seed: 4

models/config/er_star.yaml

@@ -0,0 +1,7 @@
+seq-cifar10:
+  100:
+    lr: 0.1
+    model: er_star
+    p_gamma: 0.01
+    p_lam: 0.1
+    seed: 4

models/config/xder_rpc_star.yaml

@@ -0,0 +1,10 @@
+seq-cifar10:
+  100:
+    alpha: 0.1
+    beta: 0.5
+    lr: 0.03
+    model: xder_rpc_star
+    p_gamma: 0.001
+    p_lam: 0.01
+    eta: 0.001
+    seed: 4

models/xder_rpc_star.py

@@ -67,7 +67,7 @@ def get_parser(parser) -> ArgumentParser:
         parser.add_argument('--beta', type=float, required=True, help='Penalty weight.')
         add_perturb_args(parser)
         parser.add_argument('--gamma', type=float, default=0.85)
-        parser.add_argument('--eta', type=float, default=0.1)
+        parser.add_argument('--eta', type=float, default=0.001)
         parser.add_argument('--m', type=float, default=0.3)
 
         parser.add_argument('--clip_grad', type=none_or_float, default=None, metavar='NORM', help='Clip gradient norm (default: None, no clipping)')

scripts/reproduce.json

@@ -323,27 +323,27 @@
     "setting": "class-il",
     "result": 61.76,
     "notes": "",
-    "cmd": "--alpha=0.15 --beta=0.15 --buffer_size=100 --dataset=seq-cifar10 --lr=0.1 --model=derpp_star --p-gamma=0.05 --p-lam=0.05"
+    "cmd": "--buffer_size=100 --dataset=seq-cifar10 --model=derpp_star --model_config=best"
   },
   {
     "model": "er_ace_star",
     "setting": "class-il",
     "result": 60.69,
     "notes": "",
-    "cmd": "--buffer_size=100 --dataset=seq-cifar10 --lr=0.03 --model=er_ace_star --p-gamma=0.01 --p-lam=0.1"
+    "cmd": "--buffer_size=100 --dataset=seq-cifar10 --model=er_ace_star --model_config=best"
   },
   {
     "model": "er_star",
     "setting": "class-il",
     "result": 51.5,
     "notes": "",
-    "cmd": "--buffer_size=100 --dataset=seq-cifar10 --lr=0.1 --model=er_star --p-gamma=0.01 --p-lam=0.1"
+    "cmd": "--buffer_size=100 --dataset=seq-cifar10 --model=er_star --model_config=best"
   },
   {
     "model": "xder_rpc_star",
     "setting": "class-il",
     "result": 58.85,
     "notes": "",
-    "cmd": "--alpha=0.1 --beta=0.5 --buffer_size=100 --dataset=seq-cifar10 --lr=0.03 --model=xder_rpc_star --p-gamma=0.001 --p-lam=0.01"
+    "cmd": "--buffer_size 100 --dataset=seq-cifar10 --model=xder_rpc_star --model_config=best"
   }
 ]