125 lines
3.6 KiB
Markdown
125 lines
3.6 KiB
Markdown
# DynamoDB Storage Service
|
|
|
|
This package provides an enhanced DynamoDB client wrapper with the following features:
|
|
|
|
## Features
|
|
|
|
### 1. Connection Pooling
|
|
The client uses the AWS SDK's default HTTP client which includes connection pooling automatically. This ensures efficient reuse of TCP connections to DynamoDB.
|
|
|
|
### 2. Retry Logic with Exponential Backoff
|
|
The client is configured with automatic retry logic:
|
|
- **Max Attempts**: 5 retries
|
|
- **Max Backoff**: 20 seconds
|
|
- **Strategy**: Exponential backoff with jitter to prevent thundering herd
|
|
|
|
This handles transient failures gracefully and improves reliability.
|
|
|
|
### 3. Transaction Support
|
|
The `TransactWriteItems` method provides ACID transaction support for multiple write operations:
|
|
```go
|
|
err := client.TransactWriteItems(ctx, &dynamodb.TransactWriteItemsInput{
|
|
TransactItems: []types.TransactWriteItem{
|
|
{
|
|
Put: &types.Put{
|
|
TableName: aws.String("MyTable"),
|
|
Item: map[string]types.AttributeValue{
|
|
"id": &types.AttributeValueMemberS{Value: "item1"},
|
|
},
|
|
},
|
|
},
|
|
// More items...
|
|
},
|
|
})
|
|
```
|
|
|
|
### 4. Batch Operations
|
|
The client provides batch read and write operations with automatic retry of unprocessed items:
|
|
|
|
#### BatchGetItems
|
|
Retrieves multiple items in a single request:
|
|
```go
|
|
output, err := client.BatchGetItems(ctx, &dynamodb.BatchGetItemInput{
|
|
RequestItems: map[string]types.KeysAndAttributes{
|
|
"MyTable": {
|
|
Keys: []map[string]types.AttributeValue{
|
|
{"id": &types.AttributeValueMemberS{Value: "item1"}},
|
|
{"id": &types.AttributeValueMemberS{Value: "item2"}},
|
|
},
|
|
},
|
|
},
|
|
})
|
|
```
|
|
|
|
#### BatchWriteItems
|
|
Writes multiple items in a single request:
|
|
```go
|
|
err := client.BatchWriteItems(ctx, &dynamodb.BatchWriteItemInput{
|
|
RequestItems: map[string][]types.WriteRequest{
|
|
"MyTable": {
|
|
{
|
|
PutRequest: &types.PutRequest{
|
|
Item: map[string]types.AttributeValue{
|
|
"id": &types.AttributeValueMemberS{Value: "item1"},
|
|
},
|
|
},
|
|
},
|
|
},
|
|
},
|
|
})
|
|
```
|
|
|
|
Both batch operations automatically handle unprocessed items with exponential backoff retry logic.
|
|
|
|
## Standard Operations
|
|
|
|
The client also provides wrapped versions of standard DynamoDB operations with automatic retry:
|
|
- `PutItem` - Put a single item
|
|
- `GetItem` - Get a single item
|
|
- `UpdateItem` - Update a single item
|
|
- `DeleteItem` - Delete a single item
|
|
- `Query` - Query items
|
|
|
|
## Usage
|
|
|
|
### Creating a Client
|
|
|
|
```go
|
|
ctx := context.Background()
|
|
client, err := storage.NewDynamoDBClient(ctx, "http://localhost:8000")
|
|
if err != nil {
|
|
log.Fatal(err)
|
|
}
|
|
```
|
|
|
|
For production (AWS DynamoDB), pass an empty string for the endpoint:
|
|
```go
|
|
client, err := storage.NewDynamoDBClient(ctx, "")
|
|
```
|
|
|
|
### Testing
|
|
|
|
The package includes comprehensive tests that can be run against DynamoDB Local:
|
|
|
|
1. Start DynamoDB Local:
|
|
```bash
|
|
docker-compose up -d
|
|
```
|
|
|
|
2. Run tests:
|
|
```bash
|
|
go test -v ./internal/storage
|
|
```
|
|
|
|
Tests will automatically skip if DynamoDB is not available.
|
|
|
|
## Requirements Addressed
|
|
|
|
This implementation addresses the following requirements from the spec:
|
|
- **Requirement 8.1**: Immediate persistence of all changes
|
|
- **Requirement 8.8**: Efficient scaling for 10,000+ items per user
|
|
- **Design requirement**: Retry logic with exponential backoff
|
|
- **Design requirement**: Transaction support for atomic operations
|
|
- **Design requirement**: Batch operations for efficient bulk reads/writes
|
|
- **Design requirement**: Connection pooling for performance
|