aws-cloudformation-dynamodb ยท diff
git:20260223.c6d4db1 to git:20260323.935227a
94 added, 1152 removed. Audit A to A.
---
name: aws-cloudformation-dynamodb
description: Provides AWS CloudFormation patterns for DynamoDB tables, GSIs, LSIs, auto-scaling, and streams. Use when creating DynamoDB tables with CloudFormation, configuring primary keys, local/global secondary indexes, capacity modes (on-demand/provisioned), point-in-time recovery, encryption, TTL, and implementing template structure with Parameters, Outputs, Mappings, Conditions, cross-stack references.
allowed-tools: Read, Write, Bash
---
- # AWS CloudFormation DynamoDB
+ # AWS CloudFormation DynamoDB Patterns
+ Create production-ready NoSQL database infrastructure using AWS CloudFormation templates.
+
## Overview
Create production-ready NoSQL database infrastructure using AWS CloudFormation templates. This skill covers DynamoDB tables, primary keys, secondary indexes (GSI/LSI), capacity modes, auto-scaling, encryption, TTL, streams, and best practices for parameters, outputs, and cross-stack references.
## When to Use
Use this skill when:
- Creating new DynamoDB tables with CloudFormation
- Configuring primary keys (partition key, sort key)
- Creating Global Secondary Indexes (GSI) and Local Secondary Indexes (LSI)
- Setting up capacity modes (on-demand or provisioned)
- Implementing auto-scaling with Application Auto Scaling
- Enabling point-in-time recovery and backup
- Configuring encryption at rest and in transit
- Setting up TTL for automatic data expiration
- Enabling DynamoDB Streams for change data capture
- Organizing templates with Parameters, Outputs, Mappings, Conditions
- Implementing cross-stack references with export/import
- - Using Transform for macros and reuse
## Instructions
Follow these steps to create DynamoDB tables with CloudFormation:
1. **Define Table Parameters**: Specify table name and billing mode
2. **Configure Primary Key**: Set partition key and optional sort key
3. **Add Secondary Indexes**: Create GSIs for alternative access patterns
4. **Configure Encryption**: Enable encryption using KMS keys
5. **Set Up TTL**: Define timestamp attribute for automatic deletion
6. **Enable Streams**: Configure stream for change data capture
7. **Add Auto Scaling**: Implement Application Auto Scaling for provisioned capacity
8. **Create Backup**: Enable point-in-time recovery
- For complete examples, see the [EXAMPLES.md](references/examples.md) file.
+ ## Quick Reference
- ## Examples
+ | Resource Type | Purpose |
+ |---------------|---------|
+ | `AWS::DynamoDB::Table` | Create DynamoDB table |
+ | `AWS::ApplicationAutoScaling::ScalableTarget` | Auto scaling configuration |
+ | `AWS::ApplicationAutoScaling::ScalingPolicy` | Scaling policies |
+ | `AWS::KMS::Key` | KMS key for encryption |
+ | `AWS::IAM::Role` | IAM roles for auto scaling |
+ | BillingMode | `PAY_PER_REQUEST` or `PROVISIONED` |
+ | SSESpecification | Server-side encryption |
- The following examples demonstrate common DynamoDB patterns:
+ ## Examples
- ### Example 1: Table with GSI
+ ### Basic Table with On-Demand Capacity
```yaml
DynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: !Sub "${AWS::StackName}-table"
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: pk
AttributeType: S
- - AttributeName: sk
+ KeySchema:
+ - AttributeName: pk
+ KeyType: HASH
+ ```
+
+ ### Table with Global Secondary Index
+
+ ```yaml
+ DynamoDBTable:
+ Type: AWS::DynamoDB::Table
+ Properties:
+ TableName: !Sub "${AWS::StackName}-table"
+ BillingMode: PAY_PER_REQUEST
+ AttributeDefinitions:
+ - AttributeName: pk
AttributeType: S
- AttributeName: gsi-pk
AttributeType: S
KeySchema:
- AttributeName: pk
KeyType: HASH
- - AttributeName: sk
- KeyType: RANGE
GlobalSecondaryIndexes:
- - IndexName: gsi
+ - IndexName: gsi-index
KeySchema:
- AttributeName: gsi-pk
KeyType: HASH
Projection:
ProjectionType: ALL
```
- ### Example 2: Table with Auto Scaling
-
- ```yaml
- ScalableTarget:
- Type: AWS::ApplicationAutoScaling::ScalableTarget
- Properties:
- MaxCapacity: 100
- MinCapacity: 5
- ResourceId: !Sub "table/${DynamoDBTable}"
- RoleARN: !GetAtt AutoScalingRole.Arn
- ScalableDimension: dynamodb:table:ReadCapacityUnits
- ServiceNamespace: dynamodb
- ```
-
- ### Example 3: Table with TTL
+ ### Table with TTL
```yaml
- DynamoDBTable:
+ SessionTable:
Type: AWS::DynamoDB::Table
Properties:
- TableName: !Sub "${AWS::StackName}-session-table"
+ TableName: !Sub "${AWS::StackName}-sessions"
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: sessionId
AttributeType: S
KeySchema:
- AttributeName: sessionId
KeyType: HASH
TimeToLiveSpecification:
AttributeName: expiresAt
Enabled: true
```
- For complete production-ready examples, see [EXAMPLES.md](references/examples.md).
-
- ## CloudFormation Template Structure
-
- ### Base Template with Standard Format
+ ### Table with Auto Scaling
```yaml
- AWSTemplateFormatVersion: 2010-09-09
- Description: DynamoDB table with GSI and auto-scaling
-
- Metadata:
- AWS::CloudFormation::Interface:
- ParameterGroups:
- - Label:
- default: Table Configuration
- Parameters:
- - TableName
- - BillingMode
- - PrimaryKeyName
- - PrimaryKeyType
- - Label:
- default: Capacity Settings
- Parameters:
- - ReadCapacityUnits
- - WriteCapacityUnits
- - MinReadCapacity
- - MaxReadCapacity
- - TargetUtilizationPercent
-
- Parameters:
- TableName:
- Type: String
- Default: my-dynamodb-table
- Description: Name of the DynamoDB table
-
- BillingMode:
- Type: String
- Default: PAY_PER_REQUEST
- AllowedValues:
- - PAY_PER_REQUEST
- - PROVISIONED
- Description: Billing mode for the table
-
- PrimaryKeyName:
- Type: String
- Default: pk
- Description: Name of the partition key attribute
-
- PrimaryKeyType:
- Type: String
- Default: S
- AllowedValues:
- - S (String)
- - N (Number)
- - B (Binary)
- Description: Type of the partition key attribute
-
- ReadCapacityUnits:
- Type: Number
- Default: 5
- Description: Read capacity units (required for PROVISIONED mode)
-
- WriteCapacityUnits:
- Type: Number
- Default: 5
- Description: Write capacity units (required for PROVISIONED mode)
-
- MinReadCapacity:
- Type: Number
- Default: 5
- Description: Minimum read capacity for auto-scaling
-
- MaxReadCapacity:
- Type: Number
- Default: 100
- Description: Maximum read capacity for auto-scaling
-
- TargetUtilizationPercent:
- Type: Number
- Default: 70
- Description: Target utilization percentage for auto-scaling
-
- Mappings:
- CapacityConfig:
- dev:
- ReadCapacity: 5
- WriteCapacity: 5
- MinRead: 5
- MaxRead: 20
- MinWrite: 5
- MaxWrite: 20
- staging:
- ReadCapacity: 10
- WriteCapacity: 10
- MinRead: 10
- MaxRead: 50
- MinWrite: 10
- MaxWrite: 50
- production:
- ReadCapacity: 25
- WriteCapacity: 25
- MinRead: 25
- MaxRead: 200
- MinWrite: 25
- MaxWrite: 200
-
- Conditions:
- IsProvisioned: !Equals [!Ref BillingMode, PROVISIONED]
- IsDev: !Equals [!Ref Environment, dev]
-
- Resources:
- # DynamoDB Table
- MyDynamoDBTable:
- Type: AWS::DynamoDB::Table
- Properties:
- TableName: !Ref TableName
- BillingMode: !Ref BillingMode
- AttributeDefinitions:
- - AttributeName: !Ref PrimaryKeyName
- AttributeType: !Ref PrimaryKeyType
- - AttributeName: sk
- AttributeType: S
- - AttributeName: gsi_pk
- AttributeType: S
- - AttributeName: gsi_sk
- AttributeType: S
- KeySchema:
- - AttributeName: !Ref PrimaryKeyName
- KeyType: HASH
- - AttributeName: sk
- KeyType: RANGE
- GlobalSecondaryIndexes:
- - IndexName: GSI
- KeySchema:
- - AttributeName: gsi_pk
- KeyType: HASH
- - AttributeName: gsi_sk
- KeyType: RANGE
- Projection:
- ProjectionType: ALL
- ProvisionedThroughput: !If
- - IsProvisioned
- - ReadCapacityUnits: !FindInMap [CapacityConfig, !Ref Environment, ReadCapacity]
- WriteCapacityUnits: !FindInMap [CapacityConfig, !Ref Environment, WriteCapacity]
- - !Ref AWS::NoValue
- ProvisionedThroughput: !If
- - IsProvisioned
- - ReadCapacityUnits: !FindInMap [CapacityConfig, !Ref Environment, ReadCapacity]
- WriteCapacityUnits: !FindInMap [CapacityConfig, !Ref Environment, WriteCapacity]
- - !Ref AWS::NoValue
- StreamSpecification:
- StreamViewType: NEW_AND_OLD_IMAGES
- TimeToLiveSpecification:
- AttributeName: ttl
- Enabled: true
- PointInTimeRecoverySpecification:
- PointInTimeRecoveryEnabled: true
- SSESpecification:
- SSEEnabled: true
- SSEType: AES256
- Tags:
- - Key: Environment
- Value: !Ref Environment
- - Key: Project
- Value: !Ref ProjectName
-
- Outputs:
- TableName:
- Description: Name of the DynamoDB table
- Value: !Ref MyDynamoDBTable
- Export:
- Name: !Sub "${AWS::StackName}-TableName"
-
- TableArn:
- Description: ARN of the DynamoDB table
- Value: !GetAtt MyDynamoDBTable.Arn
- Export:
- Name: !Sub "${AWS::StackName}-TableArn"
-
- TableStreamArn:
- Description: ARN of the DynamoDB table stream
- Value: !GetAtt MyDynamoDBTable.StreamArn
- Export:
- Name: !Sub "${AWS::StackName}-TableStreamArn"
+ ScalableTarget:
+ Type: AWS::ApplicationAutoScaling::ScalableTarget
+ Properties:
+ MaxCapacity: 100
+ MinCapacity: 5
+ ResourceId: !Sub "table/${DynamoDBTable}"
+ RoleARN: !GetAtt AutoScalingRole.Arn
+ ScalableDimension: dynamodb:table:ReadCapacityUnits
+ ServiceNamespace: dynamodb
```
- ## Best Practices for Parameters
-
- ### AWS-Specific Parameter Types
-
- ```yaml
- Parameters:
- # AWS-specific types for validation
- TableName:
- Type: String
- Description: Name of the DynamoDB table
-
- TableArn:
- Type: AWS::DynamoDB::Table::Arn
- Description: ARN of existing DynamoDB table
-
- TableStreamArn:
- Type: AWS::DynamoDB::Table::StreamArn
- Description: Stream ARN of DynamoDB table
-
- KMSKeyArn:
- Type: AWS::KMS::Key::Arn
- Description: KMS key for server-side encryption
+ See [references/complete-examples.md](references/complete-examples.md) for more complete examples including encryption, streams, auto scaling, and production tables.
- IAMRoleArn:
- Type: AWS::IAM::Role::Arn
- Description: IAM role for DynamoDB access
- ```
+ ## Template Structure
- ### Parameter Constraints
+ ### Base Template
```yaml
+ AWSTemplateFormatVersion: 2010-09-09
+ Description: DynamoDB table with GSI and auto-scaling
+
Parameters:
TableName:
Type: String
Default: my-table
- Description: DynamoDB table name
- ConstraintDescription: Must be 3-255 characters, alphanumeric and underscores
- MinLength: 3
- MaxLength: 255
- AllowedPattern: "[a-zA-Z0-9_]+"
-
- ReadCapacityUnits:
- Type: Number
- Default: 5
- Description: Read capacity units
- MinValue: 1
- MaxValue: 40000
- ConstraintDescription: Must be between 1 and 40000
-
- WriteCapacityUnits:
- Type: Number
- Default: 5
- Description: Write capacity units
- MinValue: 1
- MaxValue: 40000
- ConstraintDescription: Must be between 1 and 40000
-
BillingMode:
Type: String
Default: PAY_PER_REQUEST
- Description: Billing mode
- AllowedValues:
- - PAY_PER_REQUEST
- - PROVISIONED
- ```
- ## Outputs and Cross-Stack References
-
- ### Export/Import Patterns
-
- ```yaml
- # Stack A - Database Stack
- AWSTemplateFormatVersion: 2010-09-09
- Description: DynamoDB table infrastructure stack
-
Resources:
- MyDynamoDBTable:
+ DynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
- TableName: !Sub "${AWS::StackName}-table"
- BillingMode: PROVISIONED
- AttributeDefinitions:
- - AttributeName: pk
- AttributeType: S
- - AttributeName: sk
- AttributeType: S
- KeySchema:
- - AttributeName: pk
- KeyType: HASH
- - AttributeName: sk
- KeyType: RANGE
- ProvisionedThroughput:
- ReadCapacityUnits: 10
- WriteCapacityUnits: 10
- SSESpecification:
- SSEEnabled: true
+ TableName: !Ref TableName
+ BillingMode: !Ref BillingMode
Outputs:
TableName:
- Description: Name of the DynamoDB table
- Value: !Ref MyDynamoDBTable
- Export:
- Name: !Sub "${AWS::StackName}-TableName"
-
+ Value: !Ref DynamoDBTable
TableArn:
- Description: ARN of the DynamoDB table
- Value: !GetAtt MyDynamoDBTable.Arn
- Export:
- Name: !Sub "${AWS::StackName}-TableArn"
-
- TableStreamArn:
- Description: Stream ARN for Lambda triggers
- Value: !GetAtt MyDynamoDBTable.StreamArn
- Export:
- Name: !Sub "${AWS::StackName}-TableStreamArn"
- ```
-
- ```yaml
- # Stack B - Application Stack (imports from Stack A)
- AWSTemplateFormatVersion: 2010-09-09
- Description: Application stack using DynamoDB table
-
- Parameters:
- DatabaseStackName:
- Type: String
- Default: database-stack
- Description: Name of the database stack
-
- Resources:
- # Lambda function that uses DynamoDB
- DataProcessorFunction:
- Type: AWS::Lambda::Function
- Properties:
- FunctionName: !Sub "${AWS::StackName}-processor"
- Runtime: python3.11
- Handler: handler.handler
- Code:
- S3Bucket: !Ref CodeBucket
- S3Key: lambda/processor.zip
- Environment:
- Variables:
- TABLE_NAME: !ImportValue
- !Sub "${DatabaseStackName}-TableName"
- Policies:
- - DynamoDBReadPolicy:
- TableName: !ImportValue
- !Sub "${DatabaseStackName}-TableName"
- - DynamoDBWritePolicy:
- TableName: !ImportValue
- !Sub "${DatabaseStackName}-TableName"
-
- # Lambda trigger from DynamoDB Streams
- StreamProcessorFunction:
- Type: AWS::Lambda::Function
- Properties:
- FunctionName: !Sub "${AWS::StackName}-stream-processor"
- Runtime: python3.11
- Handler: stream_handler.handler
- Code:
- S3Bucket: !Ref CodeBucket
- S3Key: lambda/stream-processor.zip
- EventSourceMapping:
- EventSourceArn: !ImportValue
- !Sub "${DatabaseStackName}-TableStreamArn"
- FunctionName: !Ref StreamProcessorFunction
- StartingPosition: LATEST
- ```
-
- ### Nested Stacks for Modularity
-
- ```yaml
- AWSTemplateFormatVersion: 2010-09-09
- Description: Main stack with nested DynamoDB stacks
-
- Resources:
- # Nested stack for tables
- TablesStack:
- Type: AWS::CloudFormation::Stack
- Properties:
- TemplateURL: https://s3.amazonaws.com/bucket/dynamodb-tables.yaml
- TimeoutInMinutes: 15
- Parameters:
- Environment: !Ref Environment
- TableNamePrefix: !Ref TableNamePrefix
-
- # Nested stack for auto-scaling
- AutoScalingStack:
- Type: AWS::CloudFormation::Stack
- Properties:
- TemplateURL: https://s3.amazonaws.com/bucket/dynamodb-autoscaling.yaml
- TimeoutInMinutes: 15
- Parameters:
- Environment: !Ref Environment
- TableName: !GetAtt TablesStack.Outputs.MainTableName
- ReadCapacityUnits: !GetAtt TablesStack.Outputs.ReadCapacity
- WriteCapacityUnits: !GetAtt TablesStack.Outputs.WriteCapacity
- ```
-
- ## DynamoDB Tables with Advanced Configurations
-
- ### Table with GSI and LSI
-
- ```yaml
- AWSTemplateFormatVersion: 2010-09-09
- Description: DynamoDB table with multiple GSIs and LSIs
-
- Parameters:
- Environment:
- Type: String
- Default: dev
- AllowedValues:
- - dev
- - staging
- - production
-
- Resources:
- # DynamoDB Table with indexes
- OrdersTable:
- Type: AWS::DynamoDB::Table
- Properties:
- TableName: !Sub "${AWS::StackName}-orders"
- BillingMode: PROVISIONED
- AttributeDefinitions:
- # Primary key attributes
- - AttributeName: customer_id
- AttributeType: S
- - AttributeName: order_date
- AttributeType: S
- # GSI attributes
- - AttributeName: status
- AttributeType: S
- - AttributeName: order_total
- AttributeType: N
- - AttributeName: created_at
- AttributeType: S
- # LSI attributes
- - AttributeName: ship_date
- AttributeType: S
- KeySchema:
- - AttributeName: customer_id
- KeyType: HASH
- - AttributeName: order_date
- KeyType: RANGE
- # Global Secondary Indexes
- GlobalSecondaryIndexes:
- # GSI for status-based queries
- - IndexName: StatusIndex
- KeySchema:
- - AttributeName: status
- KeyType: HASH
- - AttributeName: order_date
- KeyType: RANGE
- Projection:
- ProjectionType: ALL
- ProvisionedThroughput:
- ReadCapacityUnits: 10
- WriteCapacityUnits: 10
- # GSI for total-based queries
- - IndexName: TotalIndex
- KeySchema:
- - AttributeName: status
- KeyType: HASH
- - AttributeName: order_total
- KeyType: RANGE
- Projection:
- ProjectionType: ALL
- ProvisionedThroughput:
- ReadCapacityUnits: 10
- WriteCapacityUnits: 10
- # Local Secondary Indexes
- LocalSecondaryIndexes:
- - IndexName: ShipDateIndex
- KeySchema:
- - AttributeName: customer_id
- KeyType: HASH
- - AttributeName: ship_date
- KeyType: RANGE
- Projection:
- ProjectionType: ALL
- ProvisionedThroughput:
- ReadCapacityUnits: 20
- WriteCapacityUnits: 20
- StreamSpecification:
- StreamViewType: NEW_AND_OLD_IMAGES
- SSESpecification:
- SSEEnabled: true
- Tags:
- - Key: Environment
- Value: !Ref Environment
- - Key: DataClassification
- Value: confidential
- ```
-
- ### On-Demand Capacity Table
-
- ```yaml
- Resources:
- # On-demand capacity table
- EventsTable:
- Type: AWS::DynamoDB::Table
- Properties:
- TableName: !Sub "${AWS::StackName}-events"
- BillingMode: PAY_PER_REQUEST
- AttributeDefinitions:
- - AttributeName: event_id
- AttributeType: S
- - AttributeName: event_type
- AttributeType: S
- - AttributeName: timestamp
- AttributeType: S
- KeySchema:
- - AttributeName: event_id
- KeyType: HASH
- GlobalSecondaryIndexes:
- - IndexName: TypeTimestampIndex
- KeySchema:
- - AttributeName: event_type
- KeyType: HASH
- - AttributeName: timestamp
- KeyType: RANGE
- Projection:
- ProjectionType: ALL
- StreamSpecification:
- StreamViewType: KEYS_ONLY
- SSESpecification:
- SSEEnabled: true
- PointInTimeRecoverySpecification:
- PointInTimeRecoveryEnabled: true
- ```
-
- ## Auto-Scaling Configuration
-
- ### Application Auto Scaling for Provisioned Tables
-
- ```yaml
- AWSTemplateFormatVersion: 2010-09-09
- Description: DynamoDB table with auto-scaling
-
- Parameters:
- Environment:
- Type: String
- Default: dev
- AllowedValues:
- - dev
- - staging
- - production
-
- Resources:
- # DynamoDB Table
- MyTable:
- Type: AWS::DynamoDB::Table
- Properties:
- TableName: !Sub "${AWS::StackName}-table"
- BillingMode: PROVISIONED
- AttributeDefinitions:
- - AttributeName: pk
- AttributeType: S
- - AttributeName: sk
- AttributeType: S
- - AttributeName: gsi_pk
- AttributeType: S
- KeySchema:
- - AttributeName: pk
- KeyType: HASH
- - AttributeName: sk
- KeyType: RANGE
- GlobalSecondaryIndexes:
- - IndexName: GSI
- KeySchema:
- - AttributeName: gsi_pk
- KeyType: HASH
- Projection:
- ProjectionType: ALL
- ProvisionedThroughput:
- ReadCapacityUnits: 5
- WriteCapacityUnits: 5
- ProvisionedThroughput:
- ReadCapacityUnits: 5
- WriteCapacityUnits: 5
-
- # Scalable Target for Table Read Capacity
- ReadScalingTarget:
- Type: AWS::ApplicationAutoScaling::ScalableTarget
- Properties:
- MaxCapacity: !FindInMap [CapacityConfig, !Ref Environment, MaxReadCapacity]
- MinCapacity: !FindInMap [CapacityConfig, !Ref Environment, MinReadCapacity]
- ResourceId: !Sub "table/${MyTable}"
- RoleARN: !GetAtt AutoScalingRole.Arn
- ScalableDimension: dynamodb:table:ReadCapacityUnits
- ServiceNamespace: dynamodb
-
- # Scalable Target for Table Write Capacity
- WriteScalingTarget:
- Type: AWS::ApplicationAutoScaling::ScalableTarget
- Properties:
- MaxCapacity: !FindInMap [CapacityConfig, !Ref Environment, MaxWriteCapacity]
- MinCapacity: !FindInMap [CapacityConfig, !Ref Environment, MinWriteCapacity]
- ResourceId: !Sub "table/${MyTable}"
- RoleARN: !GetAtt AutoScalingRole.Arn
- ScalableDimension: dynamodb:table:WriteCapacityUnits
- ServiceNamespace: dynamodb
-
- # Scalable Target for GSI Read Capacity
- GSIScalingTarget:
- Type: AWS::ApplicationAutoScaling::ScalableTarget
- Properties:
- MaxCapacity: 100
- MinCapacity: 5
- ResourceId: !Sub "table/${MyTable}/index/GSI"
- RoleARN: !GetAtt AutoScalingRole.Arn
- ScalableDimension: dynamodb:index:ReadCapacityUnits
- ServiceNamespace: dynamodb
-
- # Scaling Policy for Read Capacity
- ReadScalingPolicy:
- Type: AWS::ApplicationAutoScaling::ScalingPolicy
- Properties:
- PolicyName: !Sub "${AWS::StackName}-read-scaling"
- PolicyType: TargetTrackingScaling
- ScalingTargetId: !Ref ReadScalingTarget
- TargetTrackingScalingPolicyConfiguration:
- PredefinedMetricSpecification:
- PredefinedMetricType: DynamoDBReadCapacityUtilization
- TargetValue: 70
- ScaleInCooldown: 60
- ScaleOutCooldown: 60
-
- # Scaling Policy for Write Capacity
- WriteScalingPolicy:
- Type: AWS::ApplicationAutoScaling::ScalingPolicy
- Properties:
- PolicyName: !Sub "${AWS::StackName}-write-scaling"
- PolicyType: TargetTrackingScaling
- ScalingTargetId: !Ref WriteScalingTarget
- TargetTrackingScalingPolicyConfiguration:
- PredefinedMetricSpecification:
- PredefinedMetricType: DynamoDBWriteCapacityUtilization
- TargetValue: 70
- ScaleInCooldown: 60
- ScaleOutCooldown: 60
-
- # Auto Scaling IAM Role
- AutoScalingRole:
- Type: AWS::IAM::Role
- Properties:
- RoleName: !Sub "${AWS::StackName}-dynamodb-autoscaling"
- AssumeRolePolicyDocument:
- Version: "2012-10-17"
- Statement:
- - Effect: Allow
- Principal:
- Service: application-autoscaling.amazonaws.com
- Action: sts:AssumeRole
- ManagedPolicyArns:
- - arn:aws:iam::aws:policy/service-role/DynamoDBAutoscaleRole
-
- Mappings:
- CapacityConfig:
- dev:
- MinReadCapacity: 5
- MaxReadCapacity: 20
- MinWriteCapacity: 5
- MaxWriteCapacity: 20
- staging:
- MinReadCapacity: 10
- MaxReadCapacity: 50
- MinWriteCapacity: 10
- MaxWriteCapacity: 50
- production:
- MinReadCapacity: 25
- MaxReadCapacity: 200
- MinWriteCapacity: 25
- MaxWriteCapacity: 200
- ```
-
- ## DynamoDB Streams and Lambda Integration
-
- ### Table with Stream for Lambda Trigger
-
- ```yaml
- AWSTemplateFormatVersion: 2010-09-09
- Description: DynamoDB table with stream for Lambda trigger
-
- Resources:
- # DynamoDB Table with Stream
- OrdersTable:
- Type: AWS::DynamoDB::Table
- Properties:
- TableName: !Sub "${AWS::StackName}-orders"
- BillingMode: PROVISIONED
- AttributeDefinitions:
- - AttributeName: order_id
- AttributeType: S
- - AttributeName: customer_id
- AttributeType: S
- - AttributeName: status
- AttributeType: S
- KeySchema:
- - AttributeName: order_id
- KeyType: HASH
- GlobalSecondaryIndexes:
- - IndexName: CustomerIndex
- KeySchema:
- - AttributeName: customer_id
- KeyType: HASH
- - AttributeName: status
- KeyType: RANGE
- Projection:
- ProjectionType: ALL
- ProvisionedThroughput:
- ReadCapacityUnits: 10
- WriteCapacityUnits: 10
- StreamSpecification:
- StreamViewType: NEW_AND_OLD_IMAGES
-
- # Lambda Function for processing stream
- StreamProcessorFunction:
- Type: AWS::Lambda::Function
- Properties:
- FunctionName: !Sub "${AWS::StackName}-stream-processor"
- Runtime: python3.11
- Handler: handler.process_event
- Code:
- S3Bucket: !Ref CodeBucket
- S3Key: lambda/stream-processor.zip
- Timeout: 300
- Role: !GetAtt LambdaExecutionRole.Arn
-
- # Event Source Mapping
- StreamEventSource:
- Type: AWS::Lambda::EventSourceMapping
- Properties:
- EventSourceArn: !GetAtt OrdersTable.StreamArn
- FunctionName: !Ref StreamProcessorFunction
- StartingPosition: TRIM_HORIZON
- BatchSize: 100
- MaximumBatchingWindowInSeconds: 60
- DestinationConfig:
- OnFailure:
- Destination: !GetAtt DeadLetterQueue.Arn
- Enabled: true
-
- # Dead Letter Queue for failed events
- DeadLetterQueue:
- Type: AWS::SQS::Queue
- Properties:
- QueueName: !Sub "${AWS::StackName}-stream-dlq"
- MessageRetentionPeriod: 86400
-
- # Lambda Execution Role
- LambdaExecutionRole:
- Type: AWS::IAM::Role
- Properties:
- RoleName: !Sub "${AWS::StackName}-lambda-role"
- AssumeRolePolicyDocument:
- Version: "2012-10-17"
- Statement:
- - Effect: Allow
- Principal:
- Service: lambda.amazonaws.com
- Action: sts:AssumeRole
- ManagedPolicyArns:
- - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
- Policies:
- - PolicyName: !Sub "${AWS::StackName}-dynamodb-policy"
- PolicyDocument:
- Version: "2012-10-17"
- Statement:
- - Effect: Allow
- Action:
- - dynamodb:GetRecords
- - dynamodb:GetShardIterator
- - dynamodb:DescribeStream
- - dynamodb:ListStreams
- Resource: !GetAtt OrdersTable.StreamArn
- - Effect: Allow
- Action:
- - sqs:SendMessage
- Resource: !GetAtt DeadLetterQueue.Arn
- ```
-
- ## TTL Configuration
-
- ### Table with Time-to-Live
-
- ```yaml
- Resources:
- # Table with TTL for automatic data expiration
- SessionsTable:
- Type: AWS::DynamoDB::Table
- Properties:
- TableName: !Sub "${AWS::StackName}-sessions"
- BillingMode: PAY_PER_REQUEST
- AttributeDefinitions:
- - AttributeName: session_id
- AttributeType: S
- - AttributeName: user_id
- AttributeType: S
- KeySchema:
- - AttributeName: session_id
- KeyType: HASH
- GlobalSecondaryIndexes:
- - IndexName: UserIndex
- KeySchema:
- - AttributeName: user_id
- KeyType: HASH
- Projection:
- ProjectionType: ALL
- StreamSpecification:
- StreamViewType: NEW_IMAGE
- TimeToLiveSpecification:
- AttributeName: ttl
- Enabled: true
- SSESpecification:
- SSEEnabled: true
-
- # TTL for 24-hour expiration (example)
- SessionCleanupFunction:
- Type: AWS::Lambda::Function
- Properties:
- FunctionName: !Sub "${AWS::StackName}-session-cleanup"
- Runtime: python3.11
- Handler: handler.cleanup
- Code:
- S3Bucket: !Ref CodeBucket
- S3Key: lambda/session-cleanup.zip
- Role: !GetAtt LambdaExecutionRole.Arn
-
- # Scheduled rule for TTL cleanup
- SessionCleanupRule:
- Type: AWS::Events::Rule
- Properties:
- Name: !Sub "${AWS::StackName}-session-cleanup"
- ScheduleExpression: "rate(1 hour)"
- State: ENABLED
- Targets:
- - Id: !Ref SessionCleanupFunction
- Arn: !GetAtt SessionCleanupFunction.Arn
- ```
-
- ## Encryption and Security
-
- ### Table with Customer Managed KMS Key
-
- ```yaml
- AWSTemplateFormatVersion: 2010-09-09
- Description: DynamoDB table with customer-managed encryption
-
- Resources:
- # KMS Key for encryption
- DynamoDBKMSKey:
- Type: AWS::KMS::Key
- Properties:
- KeyName: !Sub "${AWS::StackName}-dynamodb-key"
- Description: KMS key for DynamoDB encryption
- KeyPolicy:
- Version: "2012-10-17"
- Statement:
- - Sid: Enable IAM policies
- Effect: Allow
- Principal:
- AWS: !GetAtt IAMRole.Arn
- Action:
- - kms:*
- Resource: "*"
- - Sid: Allow DynamoDB to use the key
- Effect: Allow
- Principal:
- Service: dynamodb.amazonaws.com
- Action:
- - kms:Encrypt
- - kms:Decrypt
- - kms:GenerateDataKey
- - kms:GenerateDataKeyWithoutPlaintext
- Resource: "*"
-
- # DynamoDB Table with customer-managed encryption
- SensitiveDataTable:
- Type: AWS::DynamoDB::Table
- Properties:
- TableName: !Sub "${AWS::StackName}-sensitive-data"
- BillingMode: PROVISIONED
- AttributeDefinitions:
- - AttributeName: id
- AttributeType: S
- - AttributeName: category
- AttributeType: S
- KeySchema:
- - AttributeName: id
- KeyType: HASH
- ProvisionedThroughput:
- ReadCapacityUnits: 10
- WriteCapacityUnits: 10
- SSESpecification:
- SSEEnabled: true
- SSEType: KMS
- KMSMasterKeyId: !Ref DynamoDBKMSKey
- PointInTimeRecoverySpecification:
- PointInTimeRecoveryEnabled: true
-
- # IAM Role for accessing encrypted table
- IAMRole:
- Type: AWS::IAM::Role
- Properties:
- RoleName: !Sub "${AWS::StackName}-data-access-role"
- AssumeRolePolicyDocument:
- Version: "2012-10-17"
- Statement:
- - Effect: Allow
- Principal:
- Service: lambda.amazonaws.com
- Action: sts:AssumeRole
- Policies:
- - PolicyName: !Sub "${AWS::StackName}-data-access"
- PolicyDocument:
- Version: "2012-10-17"
- Statement:
- - Effect: Allow
- Action:
- - dynamodb:GetItem
- - dynamodb:PutItem
- - dynamodb:UpdateItem
- - dynamodb:DeleteItem
- - dynamodb:Query
- - dynamodb:Scan
- Resource: !GetAtt SensitiveDataTable.Arn
- - Effect: Allow
- Action:
- - kms:Decrypt
- - kms:GenerateDataKey
- Resource: !GetAtt DynamoDBKMSKey.Arn
- ```
-
- ## Conditions and Transform
-
- ### Conditions for Environment-Specific Resources
-
- ```yaml
- AWSTemplateFormatVersion: 2010-09-09
- Description: DynamoDB with conditional resources
-
- Parameters:
- Environment:
- Type: String
- Default: dev
- AllowedValues:
- - dev
- - staging
- - production
-
- EnableEncryption:
- Type: String
- Default: true
- AllowedValues:
- - true
- - false
-
- Conditions:
- IsProduction: !Equals [!Ref Environment, production]
- IsDev: !Equals [!Ref Environment, dev]
- EnableEncryption: !Equals [!Ref EnableEncryption, true]
- EnablePITR: !Not [!Equals [!Ref Environment, dev]]
- EnableStream: !Or [!Equals [!Ref Environment, staging], !Equals [!Ref Environment, production]]
-
- Resources:
- # Main table
- MyTable:
- Type: AWS::DynamoDB::Table
- Properties:
- TableName: !Sub "${AWS::StackName}-table"
- BillingMode: !If [IsProduction, PROVISIONED, PAY_PER_REQUEST]
- AttributeDefinitions:
- - AttributeName: pk
- AttributeType: S
- - AttributeName: sk
- AttributeType: S
- KeySchema:
- - AttributeName: pk
- KeyType: HASH
- - AttributeName: sk
- KeyType: RANGE
- ProvisionedThroughput: !If
- - IsProduction
- - ReadCapacityUnits: 25
- WriteCapacityUnits: 25
- - !Ref AWS::NoValue
- StreamSpecification: !If
- - EnableStream
- - StreamViewType: NEW_AND_OLD_IMAGES
- - !Ref AWS::NoValue
- SSESpecification: !If
- - EnableEncryption
- - SSEEnabled: true
- SSEType: AES256
- - !Ref AWS::NoValue
- PointInTimeRecoverySpecification: !If
- - EnablePITR
- - PointInTimeRecoveryEnabled: true
- - !Ref AWS::NoValue
+ Value: !GetAtt DynamoDBTable.Arn
```
- ### Transform for SAM Template
-
- ```yaml
- AWSTemplateFormatVersion: 2010-09-09
- Transform: AWS::Serverless-2016-10-31
-
- Description: Using SAM Transform for DynamoDB
-
- Globals:
- Function:
- Timeout: 30
- Runtime: python3.11
- Environment:
- Variables:
- TABLE_NAME: !Ref DataTable
-
- Resources:
- # SAM Simple Table (creates table with on-demand capacity)
- DataTable:
- Type: AWS::Serverless::SimpleTable
- Properties:
- TableName: !Sub "${AWS::StackName}-data"
- PrimaryKey:
- Name: id
- Type: String
- ProvisionedThroughput:
- ReadCapacityUnits: 5
- WriteCapacityUnits: 5
- SSESpecification:
- SSEEnabled: true
-
- # Lambda function with DynamoDB access
- DataHandler:
- Type: AWS::Serverless::Function
- Properties:
- FunctionName: !Sub "${AWS::StackName}-handler"
- Handler: handler.handler
- CodeUri: lambda/
- Policies:
- - DynamoDBCrudPolicy:
- TableName: !Ref DataTable
- Events:
- Api:
- Type: Api
- Properties:
- Path: /data
- Method: post
- ```
+ See [references/advanced-configuration.md](references/advanced-configuration.md) for detailed Parameters, Mappings, Conditions, Outputs, IAM roles, and cross-stack references.
## Best Practices
- ### Security
-
- - Enable server-side encryption (SSE) for all tables
- - Use customer-managed KMS keys for sensitive data
- - Enable point-in-time recovery for production tables
- - Use IAM policies with minimum necessary permissions
- - Enable VPC endpoints for private table access
- - Configure AWS CloudTrail for auditing
-
- ### Performance
-
- - Choose partition keys with uniform distribution
- - Use GSIs for alternative access patterns
- - Monitor consumed capacity and throttled requests
- - Use auto-scaling for variable workloads
- - Consider on-demand capacity for unpredictable traffic
- - Implement proper data modeling for query patterns
-
- ### Monitoring
-
- - Enable CloudWatch metrics with 1-minute granularity
- - Create alarms for throttled requests
- - Monitor table and index capacity utilization
- - Use AWS DynamoDB Accelerator (DAX) for read-heavy workloads
- - Implement proper error handling and retries
-
- ### Cost Optimization
-
- - Use on-demand capacity when appropriate
- - Right-size provisioned capacity based on metrics
- - Use auto-scaling to handle peak loads
- - Consider TTL for automatic data cleanup
- - Archive old data to S3 with Data Pipeline or Glue
-
- ## CloudFormation Stack Management Best Practices
-
- ### Stack Policies
+ 1. **Use PAY_PER_REQUEST** for development/testing and unpredictable workloads
+ 2. **Enable Point-In-Time Recovery** for production tables
+ 3. **Use KMS encryption** for sensitive data (SSE-KMS)
+ 4. **Configure auto-scaling** for provisioned capacity tables
+ 5. **Design GSIs carefully** - each GSI consumes capacity
+ 6. **Use TTL** for automatic data expiration (sessions, cache)
+ 7. **Enable Streams** for change data capture and analytics
+ 8. **Tag resources** for cost allocation and organization
+ 9. **Export outputs** for cross-stack references
+ 10. **Use Conditions** for environment-specific configurations
- ```yaml
- Resources:
- DynamoDBTable:
- Type: AWS::DynamoDB::Table
- Properties:
- TableName: !Sub "${AWS::StackName}-table"
+ ## Common Troubleshooting
- # Stack policy to protect table from deletion
- StackPolicy:
- Type: AWS::CloudFormation::StackPolicy
- Properties:
- PolicyDocument:
- Version: '2012-10-17'
- Statement:
- - Effect: Allow
- Principal: "*"
- Action: "Update:*"
- Resource: "*"
- - Effect: Deny
- Principal: "*"
- Action:
- - Update:Delete
- Resource:
- - LogicalId: DynamoDBTable
- ```
+ **Table already exists**: Use unique table names or stack deletion policy
+ **GSI creation fails**: Verify attribute definitions include GSI attributes
+ **Auto-scaling not working**: Check IAM role permissions and service-linked role
+ **TTL not expiring**: Ensure TTL attribute is Number type, not String
+ **Streams not enabled**: Can only enable streams during table creation
+ **Encryption errors**: Verify KMS key exists in same region as table
- ### Drift Detection
+ ## Related Skills
- ```bash
- # Detect drift on a stack
- aws cloudformation detect-drift --stack-name my-dynamodb-stack
+ - [aws-cloudformation-security](../aws-cloudformation-security/) - Security best practices for DynamoDB
+ - [aws-cloudformation-lambda](../aws-cloudformation-lambda/) - Lambda triggers for DynamoDB Streams
+ - [aws-cloudformation-iam](../aws-cloudformation-iam/) - IAM roles for DynamoDB access
- # Get resource drift status
- aws cloudformation describe-stack-resource-drifts \
- --stack-name my-dynamodb-stack
- ```
+ ## References
- ## Related Resources
+ ### Complete Examples
+ - **[references/complete-examples.md](references/complete-examples.md)** - Basic tables, GSI, LSI, TTL, encryption, PITR, auto-scaling, production tables, global tables, streams
- - [DynamoDB Documentation](https://docs.aws.amazon.com/dynamodb/)
- - [AWS CloudFormation User Guide](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/)
- - [DynamoDB Best Practices](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/best-practices.html)
- - [Application Auto Scaling](https://docs.aws.amazon.com/autoscaling/application/userguide/what-is-application-auto-scaling.html)
+ ### Advanced Configuration
+ - **[references/advanced-configuration.md](references/advanced-configuration.md)** - Parameters, Mappings, Conditions, auto scaling policies, streams, TTL, global tables, outputs, IAM roles, deletion policies
## Constraints and Warnings
- ### Resource Limits
-
- - **Table Size Limits**: Maximum 40 GB per item; no limit on number of items per table
- - **Partition Key Limits**: Composite keys have specific size limits (attributes combined must be under KB limit)
- - **GSI Limits**: Maximum 20 Global Secondary Indexes per table
- - **LSI Limits**: Maximum 5 Local Secondary Indexes per table (same partition key as base table)
-
- ### Throughput Constraints
-
- - **Read/Write Capacity**: Exceeding provisioned capacity results in throttling
- - **Auto Scaling Limits**: Auto scaling takes time to adjust; instant spikes cause throttling
- - **On-Demand Mode**: On-demand tables have unlimited capacity but may throttle if table exceeds account limits
- - **Partition Key Design**: Poor partition key distribution causes hot partitions and throttling
-
- ### Operational Constraints
-
- - **Table Creation**: Creating tables with GSIs takes longer than single-table creation
- - **Index Updates**: Adding GSIs to existing tables requires rebuilding the entire table
- - **Stream Limits**: DynamoDB Streams have a 24-hour retention limit; extended retention requires Lambda or Firehose
- - **TTL Processing**: TTL deletes are not instantaneous; may take up to 48 hours
-
- ### Security Constraints
-
- - **Encryption**: Once enabled, encryption at rest cannot be disabled
- - **Point-in-Time Recovery**: PITR cannot be disabled once enabled on a table
- - **Global Tables**: Cross-region replication requires specific IAM permissions
- - **Conditional Writes**: Condition expressions that fail due to resource constraints still consume write capacity
-
- ### Cost Considerations
-
- - **Provisioned Capacity**: Pay for provisioned capacity even if unused
- - **On-Demand Pricing**: On-demand mode can be significantly more expensive for predictable workloads
- - **GSI Costs**: Each GSI incurs separate read/write capacity costs
- - **Storage Costs**: Encrypting data adds storage overhead; consider this when estimating costs
- - **Data Transfer**: Inter-region replication for global tables incurs data transfer costs
-
- ## Additional Files
-
- For complete details on resources and their properties, see:
- - [REFERENCE.md](references/reference.md) - Detailed reference guide for all DynamoDB CloudFormation resources
- - [EXAMPLES.md](references/examples.md) - Complete production-ready examples
+ - **Table names**: Must be unique per region (globally unique for global tables)
+ - **GSI limits**: Maximum 5 GSIs per table (for single-region tables)
+ - **LSI limits**: Maximum 5 LSIs per table, same partition key as table
+ - **Capacity limits**: On-demand has default account limits (40,000 RCUs, 40,000 WCUs)
+ - **Auto-scaling**: Requires PROVISIONED billing mode (not PAY_PER_REQUEST)
+ - **Point-in-time recovery**: Can only be enabled during table creation
+ - **Streams**: Can only be enabled during table creation
+ - **Encryption**: KMS keys must be in same region as table
+ - **TTL**: Attribute must be Number type, measured in seconds
+ - **Throughput**: Each GSI shares provisioned throughput with table
+ - **Item size limit**: Maximum 400 KB per item
+ - **Hot partition**: Design keys to avoid hot partition issues