Skip to content

Commit 1e6ee06

Browse files
htemelski-redisndyakovkiryazovi-redis
authored
test(e2e): testing framework upgrade (#3541)
* update e2e test, change script * update script and tests * fixed bdbid parsing * disabled majority of tests, swapped event order * change the config tag * revert test order * fix typo * reenable all e2e tests * change the clonfig flag key for all e2e tests * improve logging for debug purposes of tests * longer deadline for FI in CI * increase waiting for notifications * extend tests * dont fail on flaky third client * fi new params * fix test build * more time for migrating * first wait for FI action, then assert notification * fix test build * fix tests * fix tests * change output * global print logs for tests * better output * fix error format * maybe the notification is already received * second and third client fix * print output if failed * better second and third client checks * output action data if notification is not received * stop command runner * database create / delete actions * database create / delete actions used in tests * fix import * remove example * remove unused var * use different port than the one in env * wait for action to get the response * fix output * fix create db config * fix create db config * use new database for client * fix create db config * db per scenario * less logs, correct check * Add CTRF to the scenario tests (#3545) * add some json ctrf improvements * fix -v * attempt to separate the output --------- Co-authored-by: Nedyalko Dyakov <[email protected]> --------- Co-authored-by: Nedyalko Dyakov <[email protected]> Co-authored-by: kiryazovi-redis <[email protected]> Co-authored-by: Nedyalko Dyakov <[email protected]>
1 parent f7eed76 commit 1e6ee06

14 files changed

+1604
-444
lines changed

‎maintnotifications/e2e/DATABASE_MANAGEMENT.md‎

Lines changed: 363 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,363 @@
1+
# Database Management with Fault Injector
2+
3+
This document describes how to use the fault injector's database management endpoints to create and delete Redis databases during E2E testing.
4+
5+
## Overview
6+
7+
The fault injector now supports two new endpoints for database management:
8+
9+
1. **CREATE_DATABASE** - Create a new Redis database with custom configuration
10+
2. **DELETE_DATABASE** - Delete an existing Redis database
11+
12+
These endpoints are useful for E2E tests that need to dynamically create and destroy databases as part of their test scenarios.
13+
14+
## Action Types
15+
16+
### CREATE_DATABASE
17+
18+
Creates a new Redis database with the specified configuration.
19+
20+
**Parameters:**
21+
- `cluster_index` (int): The index of the cluster where the database should be created
22+
- `database_config` (object): The database configuration (see structure below)
23+
24+
**Raises:**
25+
- `CreateDatabaseException`: When database creation fails
26+
27+
### DELETE_DATABASE
28+
29+
Deletes an existing Redis database.
30+
31+
**Parameters:**
32+
- `cluster_index` (int): The index of the cluster containing the database
33+
- `bdb_id` (int): The database ID to delete
34+
35+
**Raises:**
36+
- `DeleteDatabaseException`: When database deletion fails
37+
38+
## Database Configuration Structure
39+
40+
The `database_config` object supports the following fields:
41+
42+
```go
43+
type DatabaseConfig struct {
44+
Name string `json:"name"`
45+
Port int `json:"port"`
46+
MemorySize int64 `json:"memory_size"`
47+
Replication bool `json:"replication"`
48+
EvictionPolicy string `json:"eviction_policy"`
49+
Sharding bool `json:"sharding"`
50+
AutoUpgrade bool `json:"auto_upgrade"`
51+
ShardsCount int `json:"shards_count"`
52+
ModuleList []DatabaseModule `json:"module_list,omitempty"`
53+
OSSCluster bool `json:"oss_cluster"`
54+
OSSClusterAPIPreferredIPType string `json:"oss_cluster_api_preferred_ip_type,omitempty"`
55+
ProxyPolicy string `json:"proxy_policy,omitempty"`
56+
ShardsPlacement string `json:"shards_placement,omitempty"`
57+
ShardKeyRegex []ShardKeyRegexPattern `json:"shard_key_regex,omitempty"`
58+
}
59+
60+
type DatabaseModule struct {
61+
ModuleArgs string `json:"module_args"`
62+
ModuleName string `json:"module_name"`
63+
}
64+
65+
type ShardKeyRegexPattern struct {
66+
Regex string `json:"regex"`
67+
}
68+
```
69+
70+
### Example Configuration
71+
72+
#### Simple Database
73+
74+
```json
75+
{
76+
"name": "simple-db",
77+
"port": 12000,
78+
"memory_size": 268435456,
79+
"replication": false,
80+
"eviction_policy": "noeviction",
81+
"sharding": false,
82+
"auto_upgrade": true,
83+
"shards_count": 1,
84+
"oss_cluster": false
85+
}
86+
```
87+
88+
#### Clustered Database with Modules
89+
90+
```json
91+
{
92+
"name": "ioredis-cluster",
93+
"port": 11112,
94+
"memory_size": 1273741824,
95+
"replication": true,
96+
"eviction_policy": "noeviction",
97+
"sharding": true,
98+
"auto_upgrade": true,
99+
"shards_count": 3,
100+
"module_list": [
101+
{
102+
"module_args": "",
103+
"module_name": "ReJSON"
104+
},
105+
{
106+
"module_args": "",
107+
"module_name": "search"
108+
},
109+
{
110+
"module_args": "",
111+
"module_name": "timeseries"
112+
},
113+
{
114+
"module_args": "",
115+
"module_name": "bf"
116+
}
117+
],
118+
"oss_cluster": true,
119+
"oss_cluster_api_preferred_ip_type": "external",
120+
"proxy_policy": "all-master-shards",
121+
"shards_placement": "sparse",
122+
"shard_key_regex": [
123+
{
124+
"regex": ".*\\{(?<tag>.*)\\}.*"
125+
},
126+
{
127+
"regex": "(?<tag>.*)"
128+
}
129+
]
130+
}
131+
```
132+
133+
## Usage Examples
134+
135+
### Example 1: Create a Simple Database
136+
137+
```go
138+
ctx := context.Background()
139+
faultInjector := NewFaultInjectorClient("http://127.0.0.1:20324")
140+
141+
dbConfig := DatabaseConfig{
142+
Name: "test-db",
143+
Port: 12000,
144+
MemorySize: 268435456, // 256MB
145+
Replication: false,
146+
EvictionPolicy: "noeviction",
147+
Sharding: false,
148+
AutoUpgrade: true,
149+
ShardsCount: 1,
150+
OSSCluster: false,
151+
}
152+
153+
resp, err := faultInjector.CreateDatabase(ctx, 0, dbConfig)
154+
if err != nil {
155+
log.Fatalf("Failed to create database: %v", err)
156+
}
157+
158+
// Wait for creation to complete
159+
status, err := faultInjector.WaitForAction(ctx, resp.ActionID,
160+
WithMaxWaitTime(5*time.Minute))
161+
if err != nil {
162+
log.Fatalf("Failed to wait for action: %v", err)
163+
}
164+
165+
if status.Status == StatusSuccess {
166+
log.Println("Database created successfully!")
167+
}
168+
```
169+
170+
### Example 2: Create a Database with Modules
171+
172+
```go
173+
dbConfig := DatabaseConfig{
174+
Name: "modules-db",
175+
Port: 12001,
176+
MemorySize: 536870912, // 512MB
177+
Replication: true,
178+
EvictionPolicy: "noeviction",
179+
Sharding: true,
180+
AutoUpgrade: true,
181+
ShardsCount: 3,
182+
ModuleList: []DatabaseModule{
183+
{ModuleArgs: "", ModuleName: "ReJSON"},
184+
{ModuleArgs: "", ModuleName: "search"},
185+
},
186+
OSSCluster: true,
187+
OSSClusterAPIPreferredIPType: "external",
188+
ProxyPolicy: "all-master-shards",
189+
ShardsPlacement: "sparse",
190+
}
191+
192+
resp, err := faultInjector.CreateDatabase(ctx, 0, dbConfig)
193+
// ... handle response
194+
```
195+
196+
### Example 3: Create Database Using a Map
197+
198+
```go
199+
dbConfigMap := map[string]interface{}{
200+
"name": "map-db",
201+
"port": 12002,
202+
"memory_size": 268435456,
203+
"replication": false,
204+
"eviction_policy": "volatile-lru",
205+
"sharding": false,
206+
"auto_upgrade": true,
207+
"shards_count": 1,
208+
"oss_cluster": false,
209+
}
210+
211+
resp, err := faultInjector.CreateDatabaseFromMap(ctx, 0, dbConfigMap)
212+
// ... handle response
213+
```
214+
215+
### Example 4: Delete a Database
216+
217+
```go
218+
clusterIndex := 0
219+
bdbID := 1
220+
221+
resp, err := faultInjector.DeleteDatabase(ctx, clusterIndex, bdbID)
222+
if err != nil {
223+
log.Fatalf("Failed to delete database: %v", err)
224+
}
225+
226+
status, err := faultInjector.WaitForAction(ctx, resp.ActionID,
227+
WithMaxWaitTime(2*time.Minute))
228+
if err != nil {
229+
log.Fatalf("Failed to wait for action: %v", err)
230+
}
231+
232+
if status.Status == StatusSuccess {
233+
log.Println("Database deleted successfully!")
234+
}
235+
```
236+
237+
### Example 5: Complete Lifecycle (Create and Delete)
238+
239+
```go
240+
// Create database
241+
dbConfig := DatabaseConfig{
242+
Name: "temp-db",
243+
Port: 13000,
244+
MemorySize: 268435456,
245+
Replication: false,
246+
EvictionPolicy: "noeviction",
247+
Sharding: false,
248+
AutoUpgrade: true,
249+
ShardsCount: 1,
250+
OSSCluster: false,
251+
}
252+
253+
createResp, err := faultInjector.CreateDatabase(ctx, 0, dbConfig)
254+
if err != nil {
255+
log.Fatalf("Failed to create database: %v", err)
256+
}
257+
258+
createStatus, err := faultInjector.WaitForAction(ctx, createResp.ActionID,
259+
WithMaxWaitTime(5*time.Minute))
260+
if err != nil || createStatus.Status != StatusSuccess {
261+
log.Fatalf("Database creation failed")
262+
}
263+
264+
// Extract bdb_id from output
265+
var bdbID int
266+
if id, ok := createStatus.Output["bdb_id"].(float64); ok {
267+
bdbID = int(id)
268+
}
269+
270+
// Use the database for testing...
271+
time.Sleep(10 * time.Second)
272+
273+
// Delete the database
274+
deleteResp, err := faultInjector.DeleteDatabase(ctx, 0, bdbID)
275+
if err != nil {
276+
log.Fatalf("Failed to delete database: %v", err)
277+
}
278+
279+
deleteStatus, err := faultInjector.WaitForAction(ctx, deleteResp.ActionID,
280+
WithMaxWaitTime(2*time.Minute))
281+
if err != nil || deleteStatus.Status != StatusSuccess {
282+
log.Fatalf("Database deletion failed")
283+
}
284+
285+
log.Println("Database lifecycle completed successfully!")
286+
```
287+
288+
## Available Methods
289+
290+
The `FaultInjectorClient` provides the following methods for database management:
291+
292+
### CreateDatabase
293+
294+
```go
295+
func (c *FaultInjectorClient) CreateDatabase(
296+
ctx context.Context,
297+
clusterIndex int,
298+
databaseConfig DatabaseConfig,
299+
) (*ActionResponse, error)
300+
```
301+
302+
Creates a new database using a structured `DatabaseConfig` object.
303+
304+
### CreateDatabaseFromMap
305+
306+
```go
307+
func (c *FaultInjectorClient) CreateDatabaseFromMap(
308+
ctx context.Context,
309+
clusterIndex int,
310+
databaseConfig map[string]interface{},
311+
) (*ActionResponse, error)
312+
```
313+
314+
Creates a new database using a flexible map configuration. Useful when you need to pass custom or dynamic configurations.
315+
316+
### DeleteDatabase
317+
318+
```go
319+
func (c *FaultInjectorClient) DeleteDatabase(
320+
ctx context.Context,
321+
clusterIndex int,
322+
bdbID int,
323+
) (*ActionResponse, error)
324+
```
325+
326+
Deletes an existing database by its ID.
327+
328+
## Testing
329+
330+
To run the database management E2E tests:
331+
332+
```bash
333+
# Run all database management tests
334+
go test -tags=e2e -v ./maintnotifications/e2e/ -run TestDatabase
335+
336+
# Run specific test
337+
go test -tags=e2e -v ./maintnotifications/e2e/ -run TestDatabaseLifecycle
338+
```
339+
340+
## Notes
341+
342+
- Database creation can take several minutes depending on the configuration
343+
- Always use `WaitForAction` to ensure the operation completes before proceeding
344+
- The `bdb_id` returned in the creation output should be used for deletion
345+
- Deleting a non-existent database will result in a failed action status
346+
- Memory sizes are specified in bytes (e.g., 268435456 = 256MB)
347+
- Port numbers should be unique and not conflict with existing databases
348+
349+
## Common Eviction Policies
350+
351+
- `noeviction` - Return errors when memory limit is reached
352+
- `allkeys-lru` - Evict any key using LRU algorithm
353+
- `volatile-lru` - Evict keys with TTL using LRU algorithm
354+
- `allkeys-random` - Evict random keys
355+
- `volatile-random` - Evict random keys with TTL
356+
- `volatile-ttl` - Evict keys with TTL, shortest TTL first
357+
358+
## Common Proxy Policies
359+
360+
- `all-master-shards` - Route to all master shards
361+
- `all-nodes` - Route to all nodes
362+
- `single-shard` - Route to a single shard
363+

0 commit comments

Comments
 (0)