- OpenSSL
- Java 11 (
JAVA_HOME
andupdate-alternatives --config java
both must be set up) - Maven
- Node.js
- RocksDB required libraries (refer INSTALL.md)
- grpc (Disaggregated setups only)
- HDFS (Disaggregated setups only)
-
The code for SHIELD is integrated into RocksDB.
-
Before building RocksDB, we need to build iotauth. We have forked iotauth from its official repository and made modifications to the codebase to work best with our deployment:
pushd ./iotauth/examples/ ./cleanAll.sh ./generateAll.sh -g ./configs/rocksdb.graph cd ../auth/auth-server mvn clean install java -jar target/auth-server-jar-with-dependencies.jar -p ../properties/exampleAuth101.properties popd
-
Change the configurations
3.1 Search for "/path/to/db/" and replace with location to save the data encryption keys.
3.2 Go to ./examples/c_client.config and update the paths for the public and private keys (they are located in iotauth/entity/auth_certs (and) iotauth/entity/credentials/certs/net1)
3.3 Go to options.cc file (./options/options.cc), search for "/path/to/c_client.config" and update the path to point to the c_clients/config above.
-
Build using CMake:
mkdir build cd build cmake -DROCKSDB_BUILD_SHARED=OFF -DWITH_TESTS=OFF -DWITH_HDFS=OFF -DWITH_EXAMPLES=OFF -DWITH_SSTLIB=ON -DWITH_CSA=OFF -DCMAKE_BUILD_TYPE=Release .. make -j64
-
The code for EncFS is located at: RocksDB EncFS plugin. Please follow the steps at the link to build EncFS.
-
The following will build both SHIELD and EncFS if you have completed the above SHIELD setup. Build using CMake:
mkdir build cd build cmake -DROCKSDB_BUILD_SHARED=OFF -DWITH_TESTS=OFF -DWITH_HDFS=OFF -DWITH_EXAMPLES=OFF -DWITH_SSTLIB=ON -DWITH_CSA=OFF -DCMAKE_BUILD_TYPE=Release -DROCKSDB_PLUGINS=encfs .. make -j64
#Unencrypted RocksDB
./db_bench --benchmarks=fillrandom --compression_type=none
#SHIELD
./db_bench --benchmarks=fillrandom --compression_type=encrypt --wal_compression=encrypt
#EncFS
./db_bench --benchmarks=fillrandom --fs_uri="provider={id=AES;hex_instance_key=0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF;method=AES256CTR};id=EncryptedFileSystem" --compression_type=none
Please make sure you have a second server setup and have HDFS deployed on that server. Make sure you install the RocksDB HDFS plugin.
- Build using CMake:
mkdir build cd build cmake -DROCKSDB_BUILD_SHARED=OFF -DWITH_TESTS=OFF -DWITH_HDFS=ON -DWITH_EXAMPLES=OFF -DWITH_SSTLIB=ON -DWITH_CSA=OFF -DCMAKE_BUILD_TYPE=Release -DROCKSDB_PLUGINS=hdfs .. make -j64
#Unencrypted RocksDB
./db_bench --benchmarks=fillrandom --compression_type=none --env_uri="path/to/hdfs"
#SHIELD
./db_bench --benchmarks=fillrandom --compression_type=encrypt --wal_compression=encrypt --env_uri="path/to/hdfs"
Please follow the the following commands on both servers you have.
-
Update the hdfs (hdfs_address) and the offloaded compaction path (csa_address) in the options.h file (options/options.h)
-
Search for the phrase: 'Please uncomment these lines when on the offloaded compaction server' and uncomment the 2 lines that follow that.
-
Build using CMake:
mkdir build cd build cmake -DROCKSDB_BUILD_SHARED=OFF -DWITH_TESTS=OFF -DWITH_HDFS=ON -DWITH_EXAMPLES=OFF -DWITH_SSTLIB=ON -DWITH_CSA=ON -DCMAKE_BUILD_TYPE=Release .. make -j64
On Server 2 you need to update your c_client.config file to have the correct IP address of the server where you have deployed iotauth.
On server 2, run the command ./csa_server
On server 1, run the following, commands:
#Unencrypted RocksDB
./db_bench --benchmarks=fillrandom --compression_type=none --allow_remote_compaction
#SHIELD
./db_bench --benchmarks=fillrandom --compression_type=encrypt --wal_compression=encrypt --allow_remote_compaction