Skip to content

Conversation

wallrj-cyberark
Copy link
Member

@wallrj-cyberark wallrj-cyberark commented Aug 8, 2025

Add CyberArk snapshot conversion

  • Introduced convertDataReadings to process DataReading objects into snapshots.
  • Added support for extracting Kubernetes server version and dynamic resources.
  • Updated CyberArkClient to use the new data conversion logic.
  • Refactored DiscoveryData and DynamicData structures for better type safety.
  • Replaced unstructured.Unstructured with runtime.Object in Snapshot fields.
  • Enhanced DataGathererDiscovery and DataGathererDynamic to return strongly typed data.
  • Added unit tests for new data extraction and conversion functions.

Part of: https://venafi.atlassian.net/browse/VC-43753

Followup PRs

  1. [VC-43753] CyberArk(client): extract cluster UID from data readings #687
  2. [VC-43403] CyberArk(helm): add Disco Agent Helm chart #678

Testing

 go run . agent --one-shot --machine-hub --agent-config-file ./examples/machinehub.yaml
I0901 08:10:33.203046 1119769 run.go:58] "Starting" logger="Run" version="development" commit=""
I0901 08:10:33.205033 1119769 run.go:116] "Healthz endpoints enabled" logger="Run.APIServer" addr=":8081" path="/healthz"
I0901 08:10:33.205147 1119769 run.go:120] "Readyz endpoints enabled" logger="Run.APIServer" addr=":8081" path="/readyz"
I0901 08:10:33.205193 1119769 run.go:269] "Pod event recorder disabled" logger="Run" reason="The agent does not appear to be running in a Kubernetes cluster." detail="When running in a Kubernetes cluster the following environment variables must be set: POD_NAME, POD_NODE, POD_UID, POD_NAMESPACE"
I0901 08:10:35.055576 1119769 identity.go:419] "successfully completed AdvanceAuthentication request to CyberArk Identity; login complete" logger="Run.gatherAndOutputData.postData" username="<REDACTED>"
I0901 08:10:36.875315 1119769 run.go:417] "Data sent successfully" logger="Run.gatherAndOutputData.postData"
richard@localhost:~/projects/jetstack/jetstack-secure$
image
$ tail -n1  request.txt | jq 
{
  "agent_version": "development",
  "cluster_id": "success-cluster-id",
  "k8s_version": "v1.33.1",
  "secrets": [
    {
      "apiVersion": "v1",
      "kind": "Secret",
      "metadata": {
        "creationTimestamp": "2025-08-29T04:07:15Z",
        "labels": {
          "cert-manager.io/next-private-key": "true",
          "controller.cert-manager.io/fao": "true"
        },
        "name": "samplewebhook-example-webhook-ca-zx9qh",
        "namespace": "samplewebhook",
        "ownerReferences": [
          {

@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-inventory-api-2 branch 2 times, most recently from 9606752 to 4d8e691 Compare August 13, 2025 16:21
@wallrj-cyberark wallrj-cyberark changed the base branch from master to VC-43403-inventory-api-sha3 August 13, 2025 16:21
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-inventory-api-2 branch 2 times, most recently from 88c1cc7 to 8b9a233 Compare August 13, 2025 21:34
@wallrj-cyberark wallrj-cyberark marked this pull request as draft August 14, 2025 08:23
Base automatically changed from VC-43403-inventory-api-sha3 to master August 14, 2025 09:16
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-inventory-api-2 branch 4 times, most recently from 2d44e46 to 34d67da Compare August 21, 2025 08:50
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-inventory-api-2 branch 2 times, most recently from caadaf0 to e8f50da Compare August 22, 2025 17:01
@wallrj-cyberark wallrj-cyberark changed the base branch from master to VC-43403-client August 22, 2025 19:48
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-client branch 6 times, most recently from 78700c9 to b7adba8 Compare August 27, 2025 17:02
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-client branch 4 times, most recently from a331e26 to 0bfb107 Compare August 28, 2025 15:38
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-client branch 5 times, most recently from d82113d to 8842333 Compare August 28, 2025 16:58
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-inventory-api-2 branch from e8f50da to 7686607 Compare August 29, 2025 03:38
@wallrj-cyberark wallrj-cyberark changed the title [VC-43753] CyberArk Discovery and Context: Upload data in the JSON format required by the API [VC-43753] CyberArk(client): add CyberArk snapshot conversion and test utilities Aug 29, 2025
Base automatically changed from VC-43403-client to master August 29, 2025 09:37
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-inventory-api-2 branch 3 times, most recently from ee3d84d to a04cdad Compare August 31, 2025 06:52
@wallrj-cyberark wallrj-cyberark changed the title [VC-43753] CyberArk(client): add CyberArk snapshot conversion and test utilities [VC-43753] CyberArk(client): add CyberArk snapshot conversion Aug 31, 2025
@wallrj-cyberark wallrj-cyberark marked this pull request as ready for review August 31, 2025 07:02
- Introduced `convertDataReadings` to process `DataReading` objects into snapshots.
- Added support for extracting Kubernetes server version and dynamic resources.
- Updated `CyberArkClient` to use the new data conversion logic.
- Refactored `DiscoveryData` and `DynamicData` structures for better type safety.
- Replaced `unstructured.Unstructured` with `runtime.Object` in `Snapshot` fields.
- Enhanced `DataGathererDiscovery` and `DataGathererDynamic` to return strongly typed data.
- Added unit tests for new data extraction and conversion functions.

Signed-off-by: Richard Wall <[email protected]>
@wallrj-cyberark wallrj-cyberark force-pushed the VC-43403-inventory-api-2 branch from a04cdad to 00e4e91 Compare September 1, 2025 07:10
@wallrj wallrj requested review from Copilot and removed request for wallrj September 1, 2025 08:18
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds CyberArk snapshot conversion functionality to process DataReading objects into the snapshot format expected by the CyberArk API.

  • Introduced data conversion logic with convertDataReadings and extractor functions for processing different resource types
  • Updated type definitions to use runtime.Object instead of unstructured.Unstructured for better type safety
  • Enhanced data gatherers to return strongly typed data structures (DynamicData and DiscoveryData)

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
pkg/client/client_cyberark.go Added conversion logic and extractor functions for transforming DataReadings to CyberArk snapshots
pkg/client/client_cyberark_convertdatareadings_test.go Added comprehensive test coverage for data conversion functions
pkg/client/client_cyberark_test.go Added helper function for generating test data and updated test cases
pkg/internal/cyberark/dataupload/dataupload.go Changed Snapshot field types from unstructured.Unstructured to runtime.Object
pkg/internal/cyberark/dataupload/dataupload_test.go Updated test to use version constant instead of hardcoded value
pkg/internal/cyberark/dataupload/mock.go Added assertions for cluster ID and agent version validation
pkg/datagatherer/k8s/discovery.go Refactored to return strongly typed DiscoveryData instead of generic map
pkg/datagatherer/k8s/dynamic.go Refactored to return strongly typed DynamicData instead of generic map
pkg/datagatherer/k8s/dynamic_test.go Updated tests to work with new strongly typed return values
api/datareading.go Added DynamicData and DiscoveryData type definitions
examples/machinehub.yaml Updated configuration to include comprehensive resource gathering definitions

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines +168 to +171
func convertDataReadings[T any](
extractorFunctions map[string]func(*api.DataReading, *T) error,
readings []*api.DataReading,
target *T,
Copy link
Preview

Copilot AI Sep 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The generic type parameter T any is too permissive. Consider constraining it to *dataupload.Snapshot or create an interface that defines the expected behavior, as this function appears to be specifically designed for snapshot conversion.

Suggested change
func convertDataReadings[T any](
extractorFunctions map[string]func(*api.DataReading, *T) error,
readings []*api.DataReading,
target *T,
func convertDataReadings(
extractorFunctions map[string]func(*api.DataReading, *dataupload.Snapshot) error,
readings []*api.DataReading,
target *dataupload.Snapshot,

Copilot uses AI. Check for mistakes.

Comment on lines +52 to +54
// Temporary hard coded cluster ID.
// TODO(wallrj): The clusterID will eventually be extracted from the supplied readings.
snapshot.ClusterID = "success-cluster-id"
Copy link
Preview

Copilot AI Sep 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded cluster ID 'success-cluster-id' should be replaced with a more appropriate default or configuration-based value for production use. Consider using a UUID or extracting it from cluster metadata.

Copilot uses AI. Check for mistakes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants