This content originally appeared on DEV Community and was authored by Pepe
When deploying an EKS cluster with self managed node groups using the Terraform AWS EKS module, a common issue arises where the nodes simply aren’t created. In one case study, the root cause was identified as Terraform attempting to locate a custom AMI in an incorrect catalog. The solution was straightforward: explicitly specify the ami_type parameter.
Overview of the Issue
The Terraform configuration was designed to set up an EKS cluster complete with self managed node groups. However, despite proper settings for VPCs, subnets, and IAM roles, the node groups were never created. The investigation revealed that Terraform was mistakenly searching for a custom AMI in a catalog that didn’t contain the desired image.
Diagnosing the Problem
I was trying to build cluster nodes with AMI based on Amazon Linux 2023. However, AWS Console error message:
Launch template nodegroup-launch-template-123456 should not specify an instance profile. The noderole in your request will be used to construct an instance profile.
and the terraform plan output looks a little bit strange.
Acquiring state lock. This may take a few moments...
module.eks.data.aws_partition.current[0]: Reading...
module.eks.module.kms.data.aws_partition.current[0]: Reading...
module.eks.data.aws_caller_identity.current[0]: Reading...
module.eks.data.aws_iam_policy_document.assume_role_policy[0]: Reading...
data.aws_caller_identity.current: Reading...
module.eks.module.kms.data.aws_caller_identity.current[0]: Reading...
module.eks.module.kms.data.aws_partition.current[0]: Read complete after 0s [id=aws]
module.eks.data.aws_partition.current[0]: Read complete after 0s [id=aws]
module.eks.data.aws_iam_policy_document.assume_role_policy[0]: Read complete after 0s [id=xxxxxx]
module.eks.data.aws_caller_identity.current[0]: Read complete after 0s [id=xxxxxx]
module.eks.data.aws_iam_policy_document.custom[0]: Reading...
module.eks.data.aws_iam_session_context.current[0]: Reading...
data.aws_caller_identity.current: Read complete after 0s [id=xxxxxx]
module.eks.data.aws_iam_policy_document.custom[0]: Read complete after 0s [id=xxxxxx]
module.eks.module.kms.data.aws_caller_identity.current[0]: Read complete after 0s [id=xxxxxx]
module.eks.module.self_managed_node_group["custom_nodegroup "].data.aws_partition.current: Reading...
module.eks.module.self_managed_node_group["custom_nodegroup "].data.aws_partition.current: Read complete after 0s [id=aws]
module.eks.module.self_managed_node_group["custom_nodegroup "].data.aws_caller_identity.current: Reading...
module.eks.module.self_managed_node_group["custom_nodegroup "].data.aws_iam_policy_document.assume_role_policy[0]: Reading...
module.eks.module.self_managed_node_group["custom_nodegroup "].data.aws_caller_identity.current: Read complete after 0s [id=xxxxxx]
module.eks.module.self_managed_node_group["custom_nodegroup "].data.aws_iam_policy_document.assume_role_policy[0]: Read complete after 0s [id=xxxxxx]
module.eks.data.aws_iam_session_context.current[0]: Read complete after 0s [id=arn:aws:sts::xxxxxx:assumed-role/eks-user/eks-1234-123456]
module.eks.module.self_managed_node_group["custom_nodegroup "].data.aws_ssm_parameter.ami[0]: Reading...
module.eks.module.self_managed_node_group["custom_nodegroup "].data.aws_ssm_parameter.ami[0]: Read complete after 0s [id=/aws/service/eks/optimized-ami/1.32/amazon-linux-2/recommended/image_id]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
It was trying to locate AMI in the wrong catalog.
[id=/aws/service/eks/optimized-ami/1.32/amazon-linux-2/recommended/image_id]
According to terraform EKS module documentation here, you should use values from official AWS documentation here.
amiType
If the node group was deployed using a launch template with a custom AMI, then this is CUSTOM. For node groups that weren't deployed using a launch template, this is the AMI type that was specified in the node group configuration.
Type: String
Valid Values:
AL2_x86_64 | AL2_x86_64_GPU | AL2_ARM_64 | CUSTOM | BOTTLEROCKET_ARM_64 | BOTTLEROCKET_x86_64 | BOTTLEROCKET_ARM_64_NVIDIA | BOTTLEROCKET_x86_64_NVIDIA | WINDOWS_CORE_2019_x86_64 | WINDOWS_FULL_2019_x86_64 | WINDOWS_CORE_2022_x86_64 | WINDOWS_FULL_2022_x86_64 | AL2023_x86_64_STANDARD | AL2023_ARM_64_STANDARD | AL2023_x86_64_NEURON | AL2023_x86_64_NVIDIA
From the documentation, we have to use CUSTOM value, but it is not true. There is a github issue: #3094. Terraform is not accepting the CUSTOM value. Instead of CUSTOM it has to be value, which the image is based on. In my example, it is AL2023_x86_64_STANDARD.
Fixed version:
variable "ami_type" {
description = "type of AMI to be used"
type = string
default = "AL2023_x86_64_STANDARD"
}
self_managed_node_groups = {
custom_nodegroup = {
name = local.cluster_name
use_name_prefix = false
subnet_ids = var.subnet_ids
additional_security_group_ids = [aws_security_group.eks_sg.id]
instance_type = var.instance_type
ami_type = var.ami_type
ami_id = var.ami_id
key_name = var.key_name
asg_min_size = 1
asg_desired_capacity = 1
asg_max_size = 2
launch_template = {
create_launch_template = true
name = "nodegroup-launch-template"
use_name_prefix = true
description = "Self managed node group launch template"
}
enable_monitoring = true
create_iam_role_policy = true
iam_role_name = local.cluster_name
iam_role_use_name_prefix = false
iam_role_description = "Self managed node group role"
create_security_group = true
security_group_name = local.cluster_name
security_group_use_name_prefix = false
revoke_rules_on_delete = true
create_before_destroy = true
security_group_description = "Self managed node group security group"
security_group_tags = {
Purpose = "Protector of the kubelet"
}
}
}
tags = local.tags
}
Conclusion
In this case, the failure to create self managed node groups was not due to security groups or IAM roles alone—it was the misdirection of Terraform attempting to retrieve a custom AMI from the wrong catalog. The simple act of specifying the ami_type parameter resolved the issue by guiding Terraform to the correct image repository.
By ensuring consistent configuration across all parameters and paying close attention to details like AMI selection, you can avoid similar pitfalls and ensure a successful EKS cluster deployment using Terraform.
This content originally appeared on DEV Community and was authored by Pepe
Pepe | Sciencx (2025-03-04T07:07:36+00:00) Troubleshooting Self Managed Node Groups in Terraform EKS. Retrieved from https://www.scien.cx/2025/03/04/troubleshooting-self-managed-node-groups-in-terraform-eks/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.