Pre-requisites for DataFactory deployment using AWS

This User-guide will walk you through the various Pre-requisites steps for DataFactory tool deployment using Amazon Web Service.

Create User

1. Create an IAM role in the AWS root account and give it admin access.
2. Log in to the AWS root account, in the Services, search for Identify and Access Management (IAM).
3. On the left pane, under ‘Access management’, select ‘Users’, click ‘Add Users’, and Add user.
4. Select the required AWS access type and click Next: Permissions to create a user.

To view the steps involved in creating a user:

Refer: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html

Create Role

1. In the AWS console: create two new roles ‘eksClusterRole’ and,’ AmazonWorkerNodeRole’
2. In the IAM service on the left pane select ‘Roles’, and click ‘Create role’.
3. Select ‘AWS service’ as the Trusted entity type.
4. Type in and select ‘EKS’ as the Amazon service, select “EKS - Cluster” and use the default Permissions policies.
5. Enter the Role details. Name it “eks-cluster-role”.
6. Click Create role.
7. Similarly, follow the above steps to create another Role Name “AmazonWorkerNodeRole”, select ‘ec2’ as an Amazon service, and add the following three AWS-managed policies. 
   1. 1. AmazonEKSWorkerNodePolicy
      2. AmazonEC2ContainerRegistryReadOnly
      3. AmazonEKS_CNI_Policy

Create Policy

1. In the IAM service on the left pane select ‘Policies’, and click ‘Create policy’.
2. Add another inline policy named ‘ClusterAutoScaler’.
3. Copy the provided JSON for this Permission.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:DescribeLaunchConfigurations",
                "autoscaling:DescribeTags",
                "autoscaling:SetDesiredCapacity",
                "autoscaling:TerminateInstanceInAutoScalingGroup",
                "ec2:DescribeLaunchTemplateVersions"
            ],
            "Resource": "*"
        }
    ]
}

Note: These policy ARNs and IAM user ARN should be noted for further use.

To view the steps involved in creating a policy:

Refer: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create-console.html

Create Policies:

1. EksClusterFullAccess:
   1. Go to the “Policies” option
   2. Click “Create policy” then click the “JSON” option
   3. Remove existing code and paste below JSON code click the “Next Tags”, give a tag, and click “Next”.
   4. To create a Policy, provide a name for the policy “EksClusterFullAccess” and create a policy.

Note: Enter your organization’s ‘AWS Account ID’ and a ‘User Name’ in the JSON code.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "eks:*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "eks:UpdateClusterVersion",
            "Resource": "arn:aws:eks:*:AWS Account ID:cluster/<dev>"
        },
        {
            "Effect": "Allow",
            "Action": [
                "eks:*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": "eks.amazonaws.com"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "eks:DescribeCluster",
                "eks:ListClusters"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "eks:DescribeNodegroup",
                "eks:ListNodegroups",
                "eks:DescribeCluster",
                "eks:ListClusters",
                "eks:AccessKubernetesApi",
                "ssm:GetParameter",
                "eks:ListUpdates",
                "eks:ListFargateProfiles"
            ],
            "Resource": "*"
        },
        {
            "Sid": "ViewOwnUserInfo",
            "Effect": "Allow",
            "Action": [
                "iam:GetUserPolicy",
                "iam:ListGroupsForUser",
                "iam:ListAttachedUserPolicies",
                "iam:ListUserPolicies",
                "iam:GetUser"
            ],
            "Resource": [
                "arn:aws:iam::*:user/$aws:User-Name"
            ]
        },
        {
            "Sid": "NavigateInConsole",
            "Effect": "Allow",
            "Action": [
                "iam:GetGroupPolicy",
                "iam:GetPolicyVersion",
                "iam:GetPolicy",
                "iam:ListAttachedGroupPolicies",
                "iam:ListGroupPolicies",
                "iam:ListPolicyVersions",
                "iam:ListPolicies",
                "iam:ListUsers"
            ],
            "Resource": "*"
        }
    ]
}

Add the following permissions to the User:

1. Select the user and click “Add permissions”, then. select the option “Attach existing policies directly” and attach the following policies to the user.

• AmazonRDSFullAccess
• AmazonEC2FullAccess
• AutoScalingFullAccess
• AmazonEC2ContainerRegistryFullAccess
• AmazonS3FullAccess
• CloudWatchFullAccess
• AmazonEC2ContainerServiceAutoscaleRole
• AmazonVPCFullAccess

Create a group and attach policies to the group:

2. Click the “User groups”, click “Create group” and provide a name for the group “eks-cluster-full-access-group” and Add permissions to the group.

• AmazonEKSClusterPolicy
• AmazonEKSWorkerNodePolicy
• AmazonEKSServicePolicy
• AmazonEKS_CNI_Policy
• AmazonEKSFargatePodExecutionRolePolicy
• AmazonEKSVPCResourceController
• ClusterAutoScaler
• EksClusterFullAccess

To view the steps involved in creating a group:

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_groups_create.html

3. Assign this user to the “eks-cluster-full-access-group” group.