feat: add support for test-export (#133)

* refactor: updates the mock schema for tes-export

Signed-off-by: re-Tick <[email protected]>

* refactor: removes constants for mock-lib from keploy package

moves constants for mock-lib from go-sdk/keploy package to keploy/models

Signed-off-by: re-Tick <[email protected]>

* refactor: updates the type models.mock to proto.mock in keploy context

Signed-off-by: re-Tick <[email protected]>

* chore: improve log message for new context

Signed-off-by: re-Tick <[email protected]>

* chore: use different emoji to represent mocks

Signed-off-by: slayerjain <[email protected]>
Signed-off-by: re-Tick <[email protected]>

* fix: adds a grpc call to avoid pushing mocks in already recorded yaml

Signed-off-by: re-Tick <[email protected]>

* fix: removes replace statement from go.mod

Signed-off-by: re-Tick <[email protected]>

* feat: exports testcases to yaml files

Signed-off-by: re-Tick <[email protected]>

* fix(gin): adds mocks in context during test for gin router

Signed-off-by: re-Tick <[email protected]>

* fix: removes duplicate functions

adds struct for checking unique mock name of mock library

Signed-off-by: re-Tick <[email protected]>

* docs: adds documentation for test-export and mock-library

adds example and configurations of mock-library for unit-tests

Signed-off-by: re-Tick <[email protected]>

Signed-off-by: re-Tick <[email protected]>
Signed-off-by: slayerjain <[email protected]>
Signed-off-by: Ritik Jain <[email protected]>
Co-authored-by: slayerjain <[email protected]>

Author: re-Tick

README.md

@@ -1,18 +1,26 @@
 # Keploy Go-SDK
 [![PkgGoDev](https://pkg.go.dev/badge/github.com/keploy/go-sdk#readme-contents)](https://pkg.go.dev/github.com/keploy/go-sdk#readme-contents)
 
-This is the client SDK for Keploy API testing platform. There are 2 modes:
-1. **Record mode**
-    1. Record requests, response and all external calls and sends to Keploy server.
-    2. After keploy server removes duplicates, it then runs the request on the API again to identify noisy fields.
-    3. Sends the noisy fields to the keploy server to be saved along with the testcase. 
-2. **Test mode**
-    1. Fetches testcases for the app from keploy server. 
-    2. Calls the API with same request payload in testcase.
-    3. Mocks external calls based on data stored in the testcase. 
-    4. Validates the respones and uploads results to the keploy server 
-
-
+This is the client SDK for Keploy API testing platform. 
+## Features
+* **API testing**: Keploy can record API calls along with their external dependency calls and replay them in testing environment.
+	### Mechanism 
+		There are 2 modes:
+			1. Record mode
+				1. Record requests, response and all external calls and sends to Keploy server.
+				2. After removing duplicates, it runs the request on the API again to identify noisy fields.
+				3. Sends the noisy fields to the keploy server to be saved along with the testcase. 
+			2. Test mode
+				1. Fetches testcases for the app from keploy server. 
+				2. Calls the API with same request payload in testcase.
+				3. Mocks external calls based on data stored in the testcase. 
+				4. Validates the respones and uploads results to the keploy server.
+* **Mock library**: It removes the necessity to write mocks manually in your unit-tests.
+	### Mechanism
+		1. During Record, it captures the outputs of integerated dependencies and stores them in yaml file locally.
+		2. During Test, fetches and returns the stored outputs of dependency calls to respective unit-tests.
+
+Just by recording the external calls from unit tests and returning the mocked outputs during testing. 
 ## Contents
 
 1. [Installation](#installation)
@@ -28,7 +36,7 @@ This is the client SDK for Keploy API testing platform. There are 2 modes:
 go get -u github.com/keploy/go-sdk
 ```
 ## Usage
-
+### API testing
 ```go
 import(
     "github.com/keploy/go-sdk/keploy"
@@ -40,15 +48,24 @@ Create your app instance
 ```go
 k := keploy.New(keploy.Config{
      App: keploy.AppConfig{
-         Name: "<app_name>",
-         Port: "<app_port>",
+		Name   "<app_name>",         // required field. app_id for grouping testcases.
+		Host    "<api_server_host>", // optional. `default:"0.0.0.0"`
+		Port    "<api_server_port>", // required.
+		Delay   <delay_testing_for>, // optional. `default:"5s"`
+		Timeout <context_deadline_for_simulate>, // optional. `default:"60s"`
+		Filter  keploy.Filter{
+			UrlRegex: "<regular_expression>", // api routes to be tested and recorded
+		}, // optional.
+		TestPath "", // `optional. default: <absolute-path>/keploy-tests`
+		MockPath "", // `optional. default: <absolute-path>/keploy-tests/mocks`
      },
      Server: keploy.ServerConfig{
-         URL: "<keploy_host>", 
-         LicenseKey: "<license_key>", //optional for managed services
+         URL: "<keploy_host>",        // optional. `default:"https://api.keploy.io"`
+         LicenseKey: "<license_key>", // optional. for managed services
      },
     })
 ```
+**Note**: Testcases can be stored on either mongoDB or in yaml files locally. By default, testcases are generated in yaml files locally.
 For example: 
 ```go
 port := "8080"
@@ -63,6 +80,24 @@ port := "8080"
  })
 
 ```
+
+### Mock Library
+```go
+import(
+    "github.com/keploy/go-sdk/keploy"
+    "github.com/keploy/go-sdk/mock"
+)
+```
+Create your context instance containing keployContext and pass it to dependency calls. 
+```go
+ctx := mock.NewContext(mock.Config{
+	Name: "<name_for_mocks>", // unique for every mock 
+	Mode: keploy.MODE_RECORD, // Or keploy.MODE_TEST, Or keploy.MODE_OFF
+	Path: "<local_path_for_yaml>", // relative(./internals) or absolute(/users/xyz/...)
+})
+```
+Integerate the dependency according to the following docs and pass the ctx context to them for mocking in unit-tests.
+
 ## Configure
 ```
 export KEPLOY_MODE=keploy.MODE_TEST

go.mod

@@ -20,72 +20,80 @@ require (
 require (
 	github.com/benbjohnson/clock v1.1.0
 	github.com/go-chi/chi v1.5.4
-	go.keploy.io/server v0.5.2
 )
 
 require (
-	github.com/andybalholm/brotli v1.0.4 // indirect
-	github.com/cpuguy83/go-md2man/v2 v2.0.1 // indirect
-	github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.0-20210816181553-5444fa50b93d // indirect
-	github.com/goccy/go-json v0.9.7 // indirect
-	github.com/gorilla/websocket v1.4.2 // indirect
-	github.com/hashicorp/golang-lru v0.5.4 // indirect
-	github.com/lestrrat-go/backoff/v2 v2.0.8 // indirect
-	github.com/lestrrat-go/blackmagic v1.0.0 // indirect
-	github.com/lestrrat-go/httpcc v1.0.0 // indirect
-	github.com/lestrrat-go/iter v1.0.1 // indirect
-	github.com/lestrrat-go/option v1.0.0 // indirect
-	github.com/matryer/moq v0.2.5 // indirect
-	github.com/mitchellh/mapstructure v1.4.3 // indirect
-	github.com/pelletier/go-toml/v2 v2.0.1 // indirect
-	github.com/russross/blackfriday/v2 v2.1.0 // indirect
-	github.com/urfave/cli/v2 v2.3.0 // indirect
-	golang.org/x/mod v0.5.1 // indirect
-	golang.org/x/tools v0.1.9 // indirect
-	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
-	gopkg.in/yaml.v3 v3.0.1 // indirect
+	github.com/gorilla/mux v1.8.0
+	github.com/lestrrat-go/jwx v1.2.20
+	github.com/valyala/fasthttp v1.35.0
+)
+
+require (
+	github.com/k0kubun/pp/v3 v3.1.0 // indirect
+	github.com/tidwall/gjson v1.14.0 // indirect
+	github.com/tidwall/match v1.1.1 // indirect
+	github.com/tidwall/pretty v1.2.0 // indirect
+	github.com/wI2L/jsondiff v0.2.0 // indirect
 )
 
 require (
 	github.com/99designs/gqlgen v0.15.1 // indirect
 	github.com/agnivade/levenshtein v1.1.1 // indirect
+	github.com/andybalholm/brotli v1.0.4 // indirect
+	github.com/cpuguy83/go-md2man/v2 v2.0.1 // indirect
+	github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.0-20210816181553-5444fa50b93d // indirect
 	github.com/gin-contrib/sse v0.1.0 // indirect
 	github.com/go-chi/render v1.0.1 // indirect
 	github.com/go-playground/locales v0.14.0 // indirect
 	github.com/go-playground/universal-translator v0.18.0 // indirect
 	github.com/go-stack/stack v1.8.1 // indirect
+	github.com/goccy/go-json v0.9.7 // indirect
 	github.com/golang/protobuf v1.5.2 // indirect
 	github.com/golang/snappy v0.0.4 // indirect
 	github.com/google/uuid v1.3.0 // indirect
-	github.com/gorilla/mux v1.8.0
+	github.com/gorilla/websocket v1.4.2 // indirect
+	github.com/hashicorp/golang-lru v0.5.4 // indirect
 	github.com/jmespath/go-jmespath v0.4.0 // indirect
 	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/klauspost/compress v1.15.1 // indirect
 	github.com/labstack/gommon v0.3.0 // indirect
 	github.com/leodido/go-urn v1.2.1 // indirect
-	github.com/lestrrat-go/jwx v1.2.20
-	github.com/mattn/go-colorable v0.1.8 // indirect
+	github.com/lestrrat-go/backoff/v2 v2.0.8 // indirect
+	github.com/lestrrat-go/blackmagic v1.0.0 // indirect
+	github.com/lestrrat-go/httpcc v1.0.0 // indirect
+	github.com/lestrrat-go/iter v1.0.1 // indirect
+	github.com/lestrrat-go/option v1.0.0 // indirect
+	github.com/matryer/moq v0.2.5 // indirect
+	github.com/mattn/go-colorable v0.1.12 // indirect
 	github.com/mattn/go-isatty v0.0.14 // indirect
+	github.com/mitchellh/mapstructure v1.4.3 // indirect
 	github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/pelletier/go-toml/v2 v2.0.1 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
+	github.com/russross/blackfriday/v2 v2.1.0 // indirect
 	github.com/ugorji/go/codec v1.2.7 // indirect
+	github.com/urfave/cli/v2 v2.3.0 // indirect
 	github.com/valyala/bytebufferpool v1.0.0 // indirect
-	github.com/valyala/fasthttp v1.35.0
 	github.com/valyala/fasttemplate v1.2.1 // indirect
 	github.com/vektah/gqlparser/v2 v2.2.0 // indirect
 	github.com/xdg-go/pbkdf2 v1.0.0 // indirect
 	github.com/xdg-go/scram v1.1.0 // indirect
 	github.com/xdg-go/stringprep v1.0.2 // indirect
 	github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a // indirect
+	go.keploy.io/server v0.5.6
 	go.uber.org/atomic v1.9.0 // indirect
 	go.uber.org/multierr v1.7.0 // indirect
 	golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d // indirect
+	golang.org/x/mod v0.5.1 // indirect
 	golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
 	golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect
 	golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9 // indirect
 	golang.org/x/text v0.3.7 // indirect
+	golang.org/x/tools v0.1.9 // indirect
+	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
 	google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013 // indirect
 	google.golang.org/protobuf v1.28.0 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
 )

go.sum

@@ -124,6 +124,8 @@ github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGw
 github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
 github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
 github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
+github.com/k0kubun/pp/v3 v3.1.0 h1:ifxtqJkRZhw3h554/z/8zm6AAbyO4LLKDlA5eV+9O8Q=
+github.com/k0kubun/pp/v3 v3.1.0/go.mod h1:vIrP5CF0n78pKHm2Ku6GVerpZBJvscg48WepUYEk2gw=
 github.com/kelseyhightower/envconfig v1.4.0/go.mod h1:cccZRl6mQpaq41TPp5QxidR+Sa3axMbJDNb//FQX6Gg=
 github.com/keploy/go-sdk v0.5.3/go.mod h1:FrKEdAbypVbeHdboueHZ3icD24pPRSn/jsZ5KLzzjJQ=
 github.com/kevinmbeaulieu/eq-go v1.0.0/go.mod h1:G3S8ajA56gKBZm4UB9AOyoOS37JO3roToPzKNM8dtdM=
@@ -163,8 +165,9 @@ github.com/matryer/moq v0.2.5 h1:BGQISyhl7Gc9W/gMYmAJONh9mT6AYeyeTjNupNPknMs=
 github.com/matryer/moq v0.2.5/go.mod h1:9RtPYjTnH1bSBIkpvtHkFN7nbWAnO7oRpdJkEIn6UtE=
 github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
 github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
-github.com/mattn/go-colorable v0.1.8 h1:c1ghPdyEDarC70ftn0y+A/Ee++9zz8ljHG1b13eJ0s8=
 github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
+github.com/mattn/go-colorable v0.1.12 h1:jF+Du6AlPIjs2BiUiQlKOX0rt3SujHxPnksPKZbaA40=
+github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
 github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
 github.com/mattn/go-isatty v0.0.9/go.mod h1:YNRxwqDuOph6SZLI9vUUz6OYw3QyUt7WiY2yME+cCiQ=
 github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
@@ -206,8 +209,13 @@ github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.1 h1:5TQK59W5E3v0r2duFAb7P95B6hEeOyEnHRa8MjYSMTY=
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
-github.com/tidwall/pretty v1.0.0 h1:HsD+QiTn7sK6flMKIvNmpqz1qrpP3Ps6jOKIKMooyg4=
+github.com/tidwall/gjson v1.14.0 h1:6aeJ0bzojgWLa82gDQHcx3S0Lr/O51I9bJ5nv6JFx5w=
+github.com/tidwall/gjson v1.14.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
+github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
+github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
 github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
+github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
+github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
 github.com/ugorji/go v1.2.7 h1:qYhyWUUd6WbiM+C6JZAUkIJt/1WrjzNHY9+KCIjVqTo=
 github.com/ugorji/go v1.2.7/go.mod h1:nF9osbDWLy6bDVv/Rtoh6QgnvNDpmCalQV5urGCCS6M=
 github.com/ugorji/go/codec v1.2.7 h1:YPXUKf7fYbp/y8xloBqZOw2qaVggbfwMlI8WM3wZUJ0=
@@ -224,6 +232,8 @@ github.com/valyala/fasttemplate v1.2.1/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+
 github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
 github.com/vektah/gqlparser/v2 v2.2.0 h1:bAc3slekAAJW6sZTi07aGq0OrfaCjj4jxARAaC7g2EM=
 github.com/vektah/gqlparser/v2 v2.2.0/go.mod h1:i3mQIGIrbK2PD1RrCeMTlVbkF2FJ6WkU1KJlJlC+3F4=
+github.com/wI2L/jsondiff v0.2.0 h1:dE00WemBa1uCjrzQUUTE/17I6m5qAaN0EMFOg2Ynr/k=
+github.com/wI2L/jsondiff v0.2.0/go.mod h1:axTcwtBkY4TsKuV+RgoMhHyHKKFRI6nnjRLi8LLYQnA=
 github.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c=
 github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=
 github.com/xdg-go/scram v1.0.2/go.mod h1:1WAq6h33pAW+iRreB34OORO2Nf7qel3VV3fjBj+hCSs=
@@ -237,8 +247,9 @@ github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a/go.mod h1:ul22v+Nro/
 github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
 github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
-go.keploy.io/server v0.5.2 h1:3HukrX9kRGB2o/7vHM5wLhzUthGPLp8BipwU+CJITxA=
 go.keploy.io/server v0.5.2/go.mod h1:TL7IcngMUjrpm9uoJbY7dfqfNNOqE5NSgQ+eHh3pp0I=
+go.keploy.io/server v0.5.6 h1:wQbhwyV+zJWBE2Zz8cVncF1Q3g62zdwjOkr3Q1ZJJCM=
+go.keploy.io/server v0.5.6/go.mod h1:/wHYlepZLITJ+MsYnRfRED9K/ENset2JhfzeAqSGnV0=
 go.mongodb.org/mongo-driver v1.8.3 h1:TDKlTkGDKm9kkJVUOAXDK5/fkqKHJVwYQSpoRfB43R4=
 go.mongodb.org/mongo-driver v1.8.3/go.mod h1:0sQWfOeY63QTntERDJJ/0SuKK0T1uVSgKCuAROlKEPY=
 go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
@@ -318,6 +329,7 @@ golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBc
 golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210910150752-751e447fb3d0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9 h1:nhht2DYV/Sn3qOayu8lM+cU1ii9sTLUeBQwQQfUHtrs=

integrations/kgin/v1/gin-v1.go

@@ -71,6 +71,7 @@ func mw(k *keploy.Keploy) gin.HandlerFunc {
 				Mode:   keploy.MODE_TEST,
 				TestID: id,
 				Deps:   k.GetDependencies(id),
+				Mock:   k.GetMocks(id),
 			})
 			resp := captureRespGin(c)
 			k.PutResp(id, resp)