O5 Improve telemetry and visualization

[tremblerz]

Currently we do not have a good way to record logs and visualize the network data. Therefore, we need to improve our log_utils library.

1. As a starting step, we need to document what all is being logged.
2. We need to have a network visualization library that shows how often and when nodes talk to each other.

[nikitaparate]

Summary of Changes:

1. Documentation:

   • Created docs/Logging.md file to document all logging activities.

2. TensorBoard Logging:

   • Modified the code to log client metrics (loss and accuracy) to TensorBoard.
   • Logs are stored in the following directories:
     • ./expt_dump/<experiment_name>/logs/client_<client_index>/
     • ./expt_dump/<experiment_name>/logs/server/

3. Summary Logging:

   • Added a summary log method in src/utils/log_utils.py to log client and server details.
   • Summary logs are saved in:
     • ./expt_dump/<experiment_name>/logs/client_<client_index>/summary.txt
     • ./expt_dump/<experiment_name>/logs/server/summary.txt

• Client metrics are now visualised on TensorBoard, allowing for real-time monitoring of training progress.
• Both clients and server details are recorded in respective summary.txt files to provide a comprehensive view of the activities.

Please review the changes and let me know if I can create a pull request for these improvements?

[tremblerz]

This sounds like a great start! Please go ahead with the PR

[nikitaparate]

Also, when an experiment results are already present, the ./expt_dump/<experiment_name>/logs/client_<client_index>/ folders are created before the user input('r' or 'e'), resulting in deletion of the ./expt_dump/<experiment_name>/logs/client_<client_index>/ folders after the input from the user.
To solve this, i added signal indicating the client code should run only after the user input 'r' (if existing experiment results are present), otherwise, the folders are created without user input for the first time.
Can i add this change along with the previous improvements in one PR.?