Skip to content

Deploy OKD 4.10 Cluster⚓︎

This instruction provides detailed information on the OKD 4.10 cluster deployment in the AWS Cloud and contains the additional setup necessary for the managed infrastructure.

A full description of the cluster deployment can be found in the official documentation.

Prerequisites⚓︎

Before the OKD cluster deployment and configuration, make sure to check the prerequisites.

Required Tools⚓︎

  1. Install the following tools listed below:

  2. Create the AWS IAM user with the required permissions. Make sure the AWS account is active, and the user doesn't have a permission boundary. Remove any Service Control Policy (SCP) restrictions from the AWS account.

  3. Generate a key pair for cluster node SSH access. Please perform the steps below:

    • Generate the SSH key. Specify the path and file name, such as ~/.ssh/id_ed25519, of the new SSH key. If there is an existing key pair, ensure that the public key is in the ~/.ssh directory.
      ssh-keygen -t ed25519 -N '' -f <path>/<file_name>
      
    • Add the SSH private key identity to the SSH agent for a local user if it has not already been added.
      eval "$(ssh-agent -s)"
      
    • Add the SSH private key to the ssh-agent:
      ssh-add <path>/<file_name>
      
  4. Build the ccoctl tool:

    • Clone the cloud-credential-operator repository.
      git clone https://github.com/openshift/cloud-credential-operator.git
      
    • Move to the cloud-credential-operator folder and build the ccoctl tool.
      cd cloud-credential-operator && git checkout release-4.10
      GO_PACKAGE='github.com/openshift/cloud-credential-operator'
      go build -ldflags "-X $GO_PACKAGE/pkg/version.versionFromGit=$(git describe --long --tags --abbrev=7 --match 'v[0-9]*')" ./cmd/ccoctl
      

Prepare for the Deployment Process⚓︎

Before deploying the OKD cluster, please perform the steps below:

Create AWS Resources⚓︎

Create the AWS resources with the Cloud Credential Operator utility (the ccoctl tool):

  1. Generate the public and private RSA key files that are used to set up the OpenID Connect identity provider for the cluster:

    ./ccoctl aws create-key-pair
    
  2. Create an OpenID Connect identity provider and an S3 bucket on AWS:

    ./ccoctl aws create-identity-provider \
    --name=<NAME> \
    --region=<AWS_REGION> \
    --public-key-file=./serviceaccount-signer.public
    

    where:

    • NAME - is the name used to tag any cloud resources created for tracking,
    • AWS_REGION - is the AWS region in which cloud resources will be created.
  3. Create the IAM roles for each component in the cluster:

    • Extract the list of the CredentialsRequest objects from the OpenShift Container Platform release image:

      oc adm release extract \
      --credentials-requests \
      --cloud=aws \
      --to=./credrequests \
      --quay.io/openshift-release-dev/ocp-release:4.10.25-x86_64
      

      Note

      A version of the openshift-release-dev docker image can be found in the Quay registry.

    • Use the ccoctl tool to process all CredentialsRequest objects in the credrequests directory:
      ccoctl aws create-iam-roles \
      --name=<NAME> \
      --region=<AWS_REGION> \
      --credentials-requests-dir=./credrequests
      --identity-provider-arn=arn:aws:iam::<AWS_ACCOUNT_ID>:oidc-provider/<NAME>-oidc.s3.<AWS_REGION>.amazonaws.com
      

Create OKD Manifests⚓︎

Before deploying the OKD cluster, please perform the steps below:

  1. Download the OKD installer.

  2. Extract the installation program:

    tar -xvf openshift-install-linux.tar.gz
    
  3. Download the installation pull secret for any private registry. This pull secret allows to authenticate with the services that are provided by the authorities, including Quay.io, serving the container images for OKD components. For example, here is a pull secret for Docker Hub:

    The pull secret for the private registry
    {
      "auths":{
        "https://index.docker.io/v1/":{
          "auth":"$TOKEN"
        }
      }
    }
    
  4. Create a deployment directory and the install-config.yaml file:

    mkdir okd-deployment
    touch okd-deployment/install-config.yaml
    

    To specify more details about the OKD cluster platform or to modify the values of the required parameters, customize the install-config.yaml file for the AWS. Please see below an example of the customized file:

    install-config.yaml - OKD cluster’s platform installation configuration file
    apiVersion: v1
    baseDomain: <YOUR_DOMAIN>
    credentialsMode: Manual
    compute:
    - architecture: amd64
      hyperthreading: Enabled
      name: worker
      platform:
        aws:
          rootVolume:
            size: 30
          zones:
            - eu-central-1a
          type: r5.large
      replicas: 3
    controlPlane:
      architecture: amd64
      hyperthreading: Enabled
      name: master
      platform:
        aws:
          rootVolume:
            size: 50
          zones:
            - eu-central-1a
          type: m5.xlarge
      replicas: 3
    metadata:
      creationTimestamp: null
      name: 4-10-okd-sandbox
    networking:
      clusterNetwork:
      - cidr: 10.128.0.0/14
        hostPrefix: 23
      machineNetwork:
      - cidr: 10.0.0.0/16
      networkType: OVNKubernetes
      serviceNetwork:
      - 172.30.0.0/16
    platform:
      aws:
        region: eu-central-1
        userTags:
          user:tag: 4-10-okd-sandbox
    publish: External
    pullSecret: <PULL_SECRET>
    sshKey: |
      <SSH_KEY>
    

    where:

    • YOUR_DOMAIN - is a base domain,
    • PULL_SECRET - is a created pull secret for a private registry,
    • SSH_KEY - is a created SSH key.
  5. Create the required OpenShift Container Platform installation manifests:

    ./openshift-install create manifests --dir okd-deployment
    
  6. Copy the manifests generated by the ccoctl tool to the manifests directory created by the installation program:

    cp ./manifests/* ./okd-deployment/manifests/
    
  7. Copy the private key generated in the tls directory by the ccoctl tool to the installation directory:

    cp -a ./tls ./okd-deployment
    

Deploy the Cluster⚓︎

To initialize the cluster deployment, run the following command:

./openshift-install create cluster --dir okd-deployment --log-level=info

Note

If the cloud provider account configured on the host does not have sufficient permissions to deploy the cluster, the installation process stops, and the missing permissions are displayed.

When the cluster deployment is completed, directions for accessing the cluster are displayed in the terminal, including a link to the web console and credentials for the kubeadmin user. The kubeconfig for the cluster will be located in okd-deployment/auth/kubeconfig.

Example output
...
INFO Install complete!
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/home/myuser/install_dir/auth/kubeconfig'
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.mycluster.example.com
INFO Login to the console with the user: "kubeadmin", and password: "4vYBz-Ee6gm-ymBZj-Wt5AL"
INFO Time elapsed: 36m22s:

Warning

The Ignition config files contain certificates that expire after 24 hours, which are then renewed at that time. Do not turn off the cluster for this time, or you will have to update the certificates manually. See OpenShift Container Platform documentation for more information.

Log Into the Cluster⚓︎

To log into the cluster, export the kubeconfig:

  export KUBECONFIG=<installation_directory>/auth/kubeconfig

Manage OKD Cluster Without the Inbound Rules⚓︎

In order to manage the OKD cluster without the 0.0.0.0/0 inbound rules, please perform the steps below:

  1. Create a Security Group with a list of your external IPs:

    aws ec2 create-security-group --group-name <SECURITY_GROUP_NAME> --description "<DESCRIPTION_OF_SECURITY_GROUP>" --vpc-id <VPC_ID>
    aws ec2 authorize-security-group-ingress \
    --group-id '<SECURITY_GROUP_ID>' \
    --ip-permissions 'IpProtocol=all,PrefixListIds=[{PrefixListId=<PREFIX_LIST_ID>}]'
    
  2. Manually attach this new Security Group to all master nodes of the cluster.

  3. Create another Security Group with an Elastic IP of the Cluster VPC:

    aws ec2 create-security-group --group-name custom-okd-4-10 --description "Cluster Ip to 80, 443" --vpc-id <VPC_ID>
    aws ec2 authorize-security-group-ingress \
     --group-id '<SECURITY_GROUP_ID>' \
     --protocol all \
     --port 80 \
     --cidr <ELASTIC_IP_OF_CLUSTER_VPC>
    aws ec2 authorize-security-group-ingress \
     --group-id '<SECURITY_GROUP_ID>' \
     --protocol all \
     --port 443 \
     --cidr <ELASTIC_IP_OF_CLUSTER_VPC>
    
  4. Modify the cluster load balancer via the router-default svc in the openshift-ingress namespace, paste two Security Groups created on previous steps:

    The pull secret for the private registry
    apiVersion: v1
    kind: Service
    metadata:
      name: router-default
      namespace: openshift-ingress
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: "tag_name=some_value"
        service.beta.kubernetes.io/aws-load-balancer-security-groups: "<SECURITY_GROUP_IDs>"
        ...
    

Optimize Spot Instances Usage⚓︎

In order to optimize the usage of Spot Instances on the AWS, add the following line under the providerSpec field in the MachineSet of Worker Nodes:

providerSpec:
  value:
    spotMarketOptions: {}