Skip to content

Commit f050b1e

Browse files
authored
Merge pull request #3036 from finos/system-info-ii
Upgrade `Client::system_info`
2 parents 4546433 + 9afb939 commit f050b1e

File tree

64 files changed

+1380
-432
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+1380
-432
lines changed

Cargo.lock

Lines changed: 10 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

README.md

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -41,15 +41,28 @@ applications.
4141

4242
- [Project Site](https://perspective.finos.org/)
4343
- [User Guide](https://perspective.finos.org/guide/)
44-
- [`@finos/perspective`, JavaScript Client API](https://docs.rs/perspective-js/latest/perspective_js/)
45-
- [`@finos/perspective-viewer`, JavaScript UI API](https://docs.rs/perspective-viewer/latest/perspective_viewer/)
46-
- [`perspective-python`, Python API](https://docs.rs/perspective-python/latest/perspective_python/)
47-
- [`perspective`, Rust API](https://docs.rs/perspective/latest/perspective/)
44+
- Python API
45+
- [`perspective`](https://perspective.finos.org/python/)
46+
- [`perspective.widget`](https://perspective.finos.org/python/perspective/widget.html)
47+
- [`perspective.handlers.aiohttp`](https://perspective.finos.org/python/perspective/handlers/aiohttp.htm)
48+
- [`perspective.handlers.starlette`](https://perspective.finos.org/python/perspective/handlers/starlett.htm)
49+
- [`perspective.handlers.tornado`](https://perspective.finos.org/python/perspective/handlers/tornado.htm)
50+
- JavaScript API
51+
- [`@finos/perspective` Browser](https://perspective.finos.org/browser/modules/src_ts_perspective.browser.ts.html)
52+
- [`@finos/perspective` Node.js](https://perspective.finos.org/node/modules/src_ts_perspective.node.ts.html)
53+
- [`@finos/perspective-viewer`](https://perspective.finos.org/viewer/modules/perspective-viewer.html)
54+
- Rust API
55+
- [`perspective`](https://docs.rs/perspective/latest/perspective/)
56+
- [`perspective-client`](https://docs.rs/perspective-client/latest/perspective_client/)
57+
- [`perspective-server`](https://docs.rs/perspective-server/latest/perspective_server/)
58+
- [`perspective-python`](https://docs.rs/perspective-python/latest/perspective_python/)
59+
- [`perspective-js`](https://docs.rs/perspective-js/latest/perspective_js/)
60+
- [`perspective-viewer`](https://docs.rs/perspective-viewer/latest/perspective_viewer/)
4861

4962
### Examples
5063

5164
<!-- Examples -->
52-
<table><tbody><tr><td>editable</td><td>file</td><td>fractal</td></tr><tr><td><a href="https://perspective.finos.org/block?example=editable"><img height="125" src="https://perspective.finos.org/blocks/editable/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=file"><img height="125" src="https://perspective.finos.org/blocks/file/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=fractal"><img height="125" src="https://perspective.finos.org/blocks/fractal/preview.png?"></img></a></td></tr><tr><td>market</td><td>raycasting</td><td>evictions</td></tr><tr><td><a href="https://perspective.finos.org/block?example=market"><img height="125" src="https://perspective.finos.org/blocks/market/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=raycasting"><img height="125" src="https://perspective.finos.org/blocks/raycasting/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=evictions"><img height="125" src="https://perspective.finos.org/blocks/evictions/preview.png?"></img></a></td></tr><tr><td>nypd</td><td>streaming</td><td>covid</td></tr><tr><td><a href="https://perspective.finos.org/block?example=nypd"><img height="125" src="https://perspective.finos.org/blocks/nypd/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=streaming"><img height="125" src="https://perspective.finos.org/blocks/streaming/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=covid"><img height="125" src="https://perspective.finos.org/blocks/covid/preview.png?"></img></a></td></tr><tr><td>webcam</td><td>movies</td><td>superstore</td></tr><tr><td><a href="https://perspective.finos.org/block?example=webcam"><img height="125" src="https://perspective.finos.org/blocks/webcam/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=movies"><img height="125" src="https://perspective.finos.org/blocks/movies/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=superstore"><img height="125" src="https://perspective.finos.org/blocks/superstore/preview.png?"></img></a></td></tr><tr><td>citibike</td><td>olympics</td></tr><tr><td><a href="https://perspective.finos.org/block?example=citibike"><img height="125" src="https://perspective.finos.org/blocks/citibike/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=olympics"><img height="125" src="https://perspective.finos.org/blocks/olympics/preview.png?"></img></a></td></tr></tbody></table>
65+
<table><tbody><tr><td>editable</td><td>file</td><td>fractal</td></tr><tr><td><a href="https://perspective.finos.org/block?example=editable"><img height="125" src="https://perspective.finos.org/blocks/editable/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=file"><img height="125" src="https://perspective.finos.org/blocks/file/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=fractal"><img height="125" src="https://perspective.finos.org/blocks/fractal/preview.png?"></img></a></td></tr><tr><td>market</td><td>raycasting</td><td>evictions</td></tr><tr><td><a href="https://perspective.finos.org/block?example=market"><img height="125" src="https://perspective.finos.org/blocks/market/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=raycasting"><img height="125" src="https://perspective.finos.org/blocks/raycasting/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=evictions"><img height="125" src="https://perspective.finos.org/blocks/evictions/preview.png?"></img></a></td></tr><tr><td>nypd</td><td>streaming</td><td>covid</td></tr><tr><td><a href="https://perspective.finos.org/block?example=nypd"><img height="125" src="https://perspective.finos.org/blocks/nypd/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=streaming"><img height="125" src="https://perspective.finos.org/blocks/streaming/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=covid"><img height="125" src="https://perspective.finos.org/blocks/covid/preview.png?"></img></a></td></tr><tr><td>webcam</td><td>movies</td><td>superstore</td></tr><tr><td><a href="https://perspective.finos.org/block?example=webcam"><img height="125" src="https://perspective.finos.org/blocks/webcam/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=movies"><img height="125" src="https://perspective.finos.org/blocks/movies/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=superstore"><img height="125" src="https://perspective.finos.org/blocks/superstore/preview.png?"></img></a></td></tr><tr><td>citibike</td><td>olympics</td><td>dataset</td></tr><tr><td><a href="https://perspective.finos.org/block?example=citibike"><img height="125" src="https://perspective.finos.org/blocks/citibike/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=olympics"><img height="125" src="https://perspective.finos.org/blocks/olympics/preview.png?"></img></a></td><td><a href="https://perspective.finos.org/block?example=dataset"><img height="125" src="https://perspective.finos.org/blocks/dataset/preview.png?"></img></a></td></tr></tbody></table>
5366
<!-- Examples -->
5467

5568
### Media

cpp/perspective/src/cpp/computed_function.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2205,7 +2205,7 @@ index::operator()(t_parameter_list parameters) {
22052205
t_tscalar rval;
22062206
rval.clear();
22072207

2208-
auto col = m_source_table->get_const_column("psp_pkey");
2208+
auto col = m_source_table->get_const_column("psp_okey");
22092209
auto res = col->get_scalar(m_row_idx);
22102210
rval.set(res);
22112211

cpp/perspective/src/cpp/server.cpp

Lines changed: 44 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,10 @@
3838
#include <vector>
3939
#include <ctime>
4040

41+
#if defined(PSP_ENABLE_WASM) || defined(__linux__)
42+
#include <malloc.h>
43+
#endif
44+
4145
#if !defined(WIN32) && !defined(PSP_ENABLE_WASM)
4246
#include <sys/resource.h>
4347
#endif
@@ -764,18 +768,28 @@ ServerResources::get_table_deleted_client(const t_id& table_id) {
764768

765769
std::uint32_t
766770
ProtoServer::new_session() {
771+
if (m_cpu_time_start.load().time_since_epoch().count() == 0) {
772+
m_cpu_time_start = std::chrono::high_resolution_clock::now();
773+
}
774+
767775
return m_client_id++;
768776
}
769777

770778
void
771779
ProtoServer::close_session(const std::uint32_t client_id) {
780+
const auto start = std::chrono::high_resolution_clock::now();
772781
m_resources.drop_client(client_id);
782+
const auto end = std::chrono::high_resolution_clock::now();
783+
m_cpu_time +=
784+
std::chrono::duration_cast<std::chrono::milliseconds>(end - start)
785+
.count();
773786
}
774787

775788
std::vector<ProtoServerResp<std::string>>
776789
ProtoServer::handle_request(
777790
std::uint32_t client_id, const std::string_view& data
778791
) {
792+
const auto start = std::chrono::high_resolution_clock::now();
779793
proto::Request req_env;
780794
req_env.ParseFromString(data);
781795
std::vector<ProtoServerResp<std::string>> serialized_responses;
@@ -828,11 +842,17 @@ ProtoServer::handle_request(
828842
serialized_responses.emplace_back(std::move(str_resp));
829843
}
830844

845+
const auto end = std::chrono::high_resolution_clock::now();
846+
m_cpu_time +=
847+
std::chrono::duration_cast<std::chrono::milliseconds>(end - start)
848+
.count();
849+
831850
return serialized_responses;
832851
}
833852

834853
std::vector<ProtoServerResp<std::string>>
835854
ProtoServer::poll() {
855+
const auto start = std::chrono::high_resolution_clock::now();
836856
std::vector<ProtoServerResp<std::string>> out;
837857
try {
838858
const auto& responses = _poll();
@@ -880,6 +900,11 @@ ProtoServer::poll() {
880900
out.emplace_back(std::move(str_resp));
881901
}
882902

903+
const auto end = std::chrono::high_resolution_clock::now();
904+
m_cpu_time +=
905+
std::chrono::duration_cast<std::chrono::milliseconds>(end - start)
906+
.count();
907+
883908
return out;
884909
}
885910

@@ -2664,13 +2689,30 @@ ProtoServer::_handle_request(std::uint32_t client_id, Request&& req) {
26642689
proto::Response resp;
26652690
auto* sys_info = resp.mutable_server_system_info_resp();
26662691
#if defined(PSP_ENABLE_WASM) && !defined(PSP_ENABLE_PYTHON)
2667-
auto heap_size = psp_heap_size();
2692+
const auto heap_size = psp_heap_size();
26682693
sys_info->set_heap_size(heap_size);
2669-
#elif !defined(WIN32) && !defined(PSP_ENABLE_WASM)
2694+
const auto res = mallinfo();
2695+
sys_info->set_used_size(res.uordblks);
2696+
#elif defined(__linux__) && !defined(PSP_ENABLE_WASM)
2697+
auto res = mallinfo();
2698+
sys_info->set_heap_size(res.usmblks);
2699+
sys_info->set_used_size(res.uordblks);
2700+
#elif defined(__APPLE__) && !defined(PSP_ENABLE_WASM)
26702701
rusage out;
26712702
getrusage(RUSAGE_SELF, &out);
26722703
sys_info->set_heap_size(out.ru_maxrss);
26732704
#endif
2705+
sys_info->set_cpu_time(m_cpu_time);
2706+
sys_info->set_cpu_time_epoch(
2707+
std::chrono::duration_cast<std::chrono::milliseconds>(
2708+
std::chrono::high_resolution_clock::now()
2709+
- m_cpu_time_start.load()
2710+
)
2711+
.count()
2712+
);
2713+
2714+
m_cpu_time_start = std::chrono::high_resolution_clock::now();
2715+
m_cpu_time = 0;
26742716
push_resp(std::move(resp));
26752717
break;
26762718
}

cpp/perspective/src/include/perspective/server.h

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -662,6 +662,9 @@ namespace server {
662662

663663
static std::uint32_t m_client_id;
664664
bool m_realtime_mode;
665+
std::atomic<std::chrono::high_resolution_clock::time_point>
666+
m_cpu_time_start;
667+
std::atomic<long long> m_cpu_time;
665668
ServerResources m_resources;
666669
t_computed_expression_parser m_computed_expression_parser;
667670
};

cpp/protos/perspective.proto

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -489,7 +489,10 @@ message ViewSetDepthResp {}
489489

490490
message ServerSystemInfoReq {}
491491
message ServerSystemInfoResp {
492-
double heap_size = 1;
492+
uint64 heap_size = 1;
493+
uint64 used_size = 2;
494+
uint32 cpu_time = 3;
495+
uint32 cpu_time_epoch = 4;
493496
}
494497

495498

examples/blocks/examples.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ const LOCAL_EXAMPLES = [
2525
"superstore",
2626
"citibike",
2727
"olympics",
28+
"dataset",
2829
];
2930

3031
exports.LOCAL_EXAMPLES = LOCAL_EXAMPLES;
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
A random dataset generator and system monitor based on the
2+
<a href="https://docs.rs/perspective-client/latest/perspective_client/struct.Client.html#method.system_info">perspective_client::Client::system_info</a>
3+
method.

0 commit comments

Comments
 (0)