Feature/v2 improvements

.aiignore

@@ -0,0 +1 @@
+artifacts

.env.sample

@@ -1 +1,60 @@
+# devnet name, optional (default `my-devnet`)
+DEVNET_NAME=
+
+# k8s global
+
+## (optional) ssh tunnel helpers (see `ssh tunnel` command)
+SSH_HOST=
+# ssh user (optional)
+SSH_USER=
+# ssh private key (optional)
+SSH_PRIVATE_KEY=
+SSH_TUNNEL_REMOTE_ADDRESS=127.0.0.1:16443
+SSH_TUNNEL_LOCAL_PORT=16444
+
+## cluster name from kubeconfig file
+K8S_KUBECTL_CLUSTER_NAME=
+
+## cluster context name from kubeconfig file
+K8S_KUBECTL_DEFAULT_CONTEXT=
+## global ingress hostname prefix for all ingresses for devnet, for keeping domains secret (example: `somesecret`)
+GLOBAL_INGRESS_HOST_PREFIX=
+
+# nodes
+ETH_NODES_INGRESS_HOSTNAME=$(DEVNET_NAME).devnet.local
+
+# dora ingress hostname (without http://)
+DORA_INGRESS_HOSTNAME=dora.$(DEVNET_NAME).devnet.local
+
+# blockscout ingress hostnames (without http://)
+BLOCKSCOUT_BACKEND_INGRESS_HOSTNAME=blockscout-backend.$(DEVNET_NAME).devnet.local
+BLOCKSCOUT_FRONTEND_INGRESS_HOSTNAME=blockscout.$(DEVNET_NAME).devnet.local
+
+# docker registry
+## registry type (local | external)
+DOCKER_REGISTRY_TYPE=external
+
+## registry ingress hostnames (without http://)
+DOCKER_REGISTRY_LOCAL_INGRESS_HOSTNAME=container-registry-ui.devnet.local
+DOCKER_REGISTRY_LOCAL_INGRESS_UI_HOSTNAME=container-registry-ui.devnet.local
+
+DOCKER_REGISTRY_EXTERNAL_HOSTNAME=container-registry-ui.devnet.local
+DOCKER_REGISTRY_EXTERNAL_UI_HOSTNAME=container-registry-ui.devnet.local
+DOCKER_REGISTRY_USERNAME=
+DOCKER_REGISTRY_PASSWORD=
+
+# kapi ingress hostname (without http://)
+KAPI_INGRESS_HOSTNAME=keys-api.$(DEVNET_NAME).devnet.local
+
+# ipfs kubo
+KUBO_INGRESS_HOSTNAME=ipfs.$(DEVNET_NAME).devnet.local
+
+# no widget backend
+NO_WIDGET_BACKEND_INGRESS_HOSTNAME=no-widget-backend.$(DEVNET_NAME).devnet.local
+
+# no widget backend
+NO_WIDGET_INGRESS_HOSTNAME=operators-widget.$(DEVNET_NAME).devnet.local
+
+# oracles (optional)
 PINATA_JWT=
+CSM_ORACLE_PINATA_JWT=

IMPROVEMENTS.md

@@ -0,0 +1,21 @@
+
+# List of features to improve
+* Make the lido-local-devnet easily extensible
+* Idempotence for all commands
+* Merge services, state, artifacts into a new entity called "project"
+  * the devnet is a root project
+  * kapi, lido-core, lido-cli are subprojects
+  * each subproject has
+    * commands
+    * templates
+    * services
+      * artifacts
+      * has optional git repo
+      * workspace
+        * patches
+    * constants
+* Add the ability to patch docker images with files, having special `pathes` folder
+* Add the ability to patch git repos with files, having special `pathes` folder
+* Improve artifact store to use NFS or S3 or Minio
+* Improve artifact store to upload it to the k8s storage with password-credentials
+* Look at cdk8s

README.md

@@ -23,44 +23,97 @@ Lido Local DevNet is a powerful tool for deploying and testing the Lido protocol
 - **npm** ([Install npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm))
 - **Docker** 27+ ([Install Docker](https://www.docker.com/))  
 - **Docker Compose** V2 ([Install Docker Compose](https://docs.docker.com/compose/))  
-- **Kurtosis** ([Install Kurtosis](https://www.kurtosistech.com/))  
+- **Kurtosis 1.11.1+** ([Install Kurtosis](https://www.kurtosistech.com/))  
 - **Foundry tools** ([Install Foundry](https://book.getfoundry.sh/getting-started/installation))  
 - **Just** ([Install Just](https://github.com/casey/just))  
+- **Make** 4+  (Install Make - Linux: `sudo apt-get install build-essential`)
+- **Kubectl** v1.30.+ (for k8s deployments) ([Install Kubectl](https://kubernetes.io/docs/tasks/tools/))`
+- **Helm** 3.12+ (for k8s deployments) ([Install Helm](https://helm.sh/docs/intro/install/))`
 
 ---
 
-## Getting Started
+## Getting Started (with k8s integration)
 
-Follow these steps to set up the DevNet:
+Original docs are located in `https://docs.kurtosis.com/k8s/`
 
-### 1. Start Kurtosis
-Kurtosis is required to launch Ethereum nodes:
+### 1. Install dependencies
 ```sh
-kurtosis engine start
+yarn && yarn build:all
 ```
 
-### 2. Install dependencies
+### 2. Create `.env` file and fill it with the required values
 ```sh
-yarn && yarn build:all
+cp .env.sample .env
+```
+
+### 3. (Optional) Turn on SSH Tunnel to the machine with k8s cluster
+```sh
+./bin/run.js ssh tunnel
+```
+
+### 4. Set the current context to the k8s cluster (if you have multiple clusters)
+
+Contexts can be found by running: `kubectl config get-contexts`
+
+```sh
+kubectl config use-context <cluster context> # or whatever your k8s context is
 ```
 
-### 3. Launch the environment and deploy Lido smart contracts
-Below is an example for launching the `pectra` test stand. If you need a different setup, refer to the [test stands documentation](./docs/commands/stands.md).
+### 5. Ensure that you are connected to the k8s cluster
+
+The cluster can be accessible via SSH Tunnel.
 
 ```sh
-./bin/run.js stands pectra --full
+kubectl cluster-info
+```
+
+### 6. Change kurtosis config to work with the k8s cluster
+
+Update once your kurtosis config at `echo $(kurtosis config path)` location
+
+```yaml
+config-version: 6
+should-send-metrics: false
+kurtosis-clusters:
+  docker:
+    type: "docker"
+  cloud:
+    type: "kubernetes"
+    config:
+      kubernetes-cluster-name: "<cluster name from kubectl>" # change the cluster name if needed
+      storage-class: "ssd-hostpath"
+      enclave-size-in-megabytes: 256
+```
+
+### 7. Point kurtosis to the cluster
+```sh
+# tell kurtosis to work with k8s cluster
+kurtosis cluster set cloud # or whatever your kurtosis cluster is
+```
+
+### 8. Start Kurtosis
+Kurtosis is required to launch Ethereum nodes
+```sh
+kurtosis engine start
+```
+
+### 9. Launch the environment and deploy Lido smart contracts
+Below is an example for launching the `fusaka` test stand. 
+If you need a different setup, refer to the [test stands documentation](./docs/commands/stands.md).
+
+```sh
+./bin/run.js stands <stand-name>  # (fusaka) or any other test stand name
 ```
 For contract verification, use the `--verify` flag:
 ```sh
-./bin/run.js stands pectra --full --verify
+./bin/run.js stands <stand-name> --verify
 ```
 For a full DSM infrastructure deployment, add the `--dsm` flag:
 ```sh
-./bin/run.js stands pectra --full --verify --dsm
+./bin/run.js stands <stand-name> --verify --dsm
 ```
 
-### 4. Interaction with Voting scripts
-
+### 10. (Optional) Interaction with Voting scripts 
 
 Since voting scripts require Python and Brownie, install the necessary dependencies:
 ```sh
@@ -85,7 +138,7 @@ After adding an account, proceed with the voting process. See the [voting docume
 ./bin/run.js voting enact-after-pectra
 ```
 
-### 5. Done!
+### 11. Done!
 The network, infrastructure, and protocol have been successfully launched.
 
 ---
@@ -100,13 +153,15 @@ To stop the DevNet and remove all services, run:
 
 ## Running Multiple Environments
 
-To run multiple environments on a single machine, use the `--network <custom-network-name>` flag:
+To run multiple devnets on a single cluster, change the `DEVNET_NAME=<another_devnet>` variable in `.env` file
+All the commands will be executed in the context of the current devnet.
+---
+
+## DevNet info
+To get the latest information on available services, run:
 ```sh
-./bin/run.js stands pectra --full --network test-pectra1
+./bin/run.js chain info
 ```
-> **Note:** The `--network test-pectra1` flag must be used with all subsequent commands to interact with the specified environment.
-
----
 
 ## Available Services
 To get the latest information on available services, run:

bin/dev.js

@@ -1,6 +1,8 @@
 #!/usr/bin/env -S node --loader ts-node/esm --disable-warning=ExperimentalWarning
 
 // eslint-disable-next-line n/shebang
-import {execute} from '@oclif/core'
+import {execute} from '@oclif/core';
+import * as dotenv from 'dotenv';
+dotenv.config({ path: '.env' });
 
 await execute({development: true, dir: import.meta.url})

bin/run.js

@@ -1,5 +1,7 @@
 #!/usr/bin/env node
 
-import {execute} from '@oclif/core'
+import {execute} from '@oclif/core';
+import * as dotenv from 'dotenv';
+dotenv.config({ path: '.env' });
 
 await execute({dir: import.meta.url})