GCR Image from External Kubernetes

GCR Image from External Kubernetes

Disclaimer: This blog contains opinions about Google technology. While I work at Google, this is my personal blog. Opinions stated here are my own, not those of my company.

Let’s say you use Google Cloud Registry (GCR) to store your images. If you are running your workloads on GKE (Google Kubernetes Engine) that is in the same project as GCR, you should have access by default. GKE clusters are created with read-only permissions for Storage buckets. However, what if you are not running Kubernetes on GCP? Or, what if you are simply running it from a different GCP project? In that case you need to configure imagePullSecret. This is one of the scenarios made simple with Config Connector – creating imagePullSecret to pull GCR Image from external Kubernetes can be done declaratively in a few steps. And if you are looking for general discussion about benefits of using Config Connector, start with this post.

Setup: GCP project, GCR image, external K8s cluster and Config Connector

We’ll start by configuring your non-GKE K8s cluster. When I tested this, I created AWS EKS cluster using the instructions here, but I’m assuming that you already have a cluster.

Next, let’s create a GCP project, install Config Connector on your cluster, and create a service account that Config Connector will use to access GCR bucket. Again, if you already have a project, feel free to only use the part of this script that configures Config Connector access.

export SA_EMAIL="cnrm-system@${PROJECT_ID}.iam.gserviceaccount.com"

# create project
gcloud projects create $PROJECT_ID --name="$PROJECT_ID"
gcloud config set project $PROJECT_ID

# to provision Config Connector, create cnrm-system service account and export the tkey
gcloud iam service-accounts create cnrm-system --project ${PROJECT_ID}
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member "serviceAccount:${SA_EMAIL}" --role roles/owner
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member "serviceAccount:${SA_EMAIL}" --role roles/storage.admin
gcloud iam service-accounts keys create --iam-account "${SA_EMAIL}" ./key.json

# install Config Connector
kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user $(gcloud config get-value account)
curl -X GET -sLO --location-trusted https://us-central1-cnrm-eap.cloudfunctions.net/download/latest/infra/install-bundle.tar.gz
rm -rf install-bundle
tar zxvf install-bundle.tar.gz
kubectl apply -f install-bundle/

# give cnrm-system namespace permissions to manage GCP
kubectl create secret generic gcp-key --from-file ./key.json --namespace cnrm-system

# annotate the namespace
kubectl annotate namespace default "cnrm.cloud.google.com/project-id=${PROJECT_ID}" --overwrite

Let’s now upload your image to GCR bucket, unless you already have an image on GCR. In this example I’m using an image with a simple node app that returns custom message from its endpoint.

docker pull bulankou/node-hello-world:latest
docker tag bulankou/node-hello-world gcr.io/[PROJECT_ID]/node-hello-world
docker push gcr.io/[PROJECT_ID]/node-hello-world

Configure Image Pull Secret

Now, when we have you image on GCR, let’s have Config Connector take ownership of the bucket. This is done by create a K8s configuration representing the bucket with the same name, usually artifacts.[PROJECT_ID].appspot.com. As usual, apply this yaml with kubectl apply.

apiVersion: storage.cnrm.cloud.google.com/v1alpha2
kind: StorageBucket
  name: artifacts.[PROJECT_ID].appspot.com

In addition to the bucket, let’s create ServiceAccount, IAMPolicy and ServiceAccountKey:

apiVersion: iam.cnrm.cloud.google.com/v1alpha1
kind: IAMServiceAccount
  name: gcr-sa
  displayName: Service Account for GCR access
apiVersion: iam.cnrm.cloud.google.com/v1alpha1
kind: IAMPolicy
  name: gcr-bucket-policy
    apiVersion: storage.cnrm.cloud.google.com/v1alpha2
    kind: StorageBucket
    name: artifacts.[PROJECT_ID].appspot.com
    - role: roles/storage.objectViewer
        - serviceAccount:gcr-sa@[PROJECT_ID].iam.gserviceaccount.com
kind: IAMServiceAccountKey
  name: gcr-sa-key
  publicKeyType: TYPE_X509_PEM_FILE
  keyAlgorithm: KEY_ALG_RSA_2048
    name: gcr-sa

IAMServiceAccountKey resource will automatically create a secret with the same name. Now we just need to create a docker-registry type secret that will take its key.json field as password:

kubectl create secret docker-registry gcr-docker-key \
  --docker-server=https://gcr.io \
  --docker-username=_json_key \
  --docker-email=user@example.com \
  --docker-password="$(kubectl get secret gcr-sa-key -o go-template=$'{{index .data "key.json"}}' | base64 --decode)"

We are done configuring the secret to pull the image. Now we can use it in our pod:

apiVersion: v1
kind: Pod
  name: node-app-pod
  - name: node-app-container
    image: gcr.io/[PROJECT_ID]/node-hello-world
    imagePullPolicy: Always
    - name: HELLO_MESSAGE
      value: "Hello from GCR!"
    - containerPort: 8080
      - name: gcr-docker-key

This is all today! We looked how you create a secret to pull a GCR image from external Kubernetes workload. The source code for this example is here.

116 thoughts on “GCR Image from External Kubernetes”

  1. Magnificent items from you, man. I have remember your stuff previous to and you’re simply too wonderful. I really like what you’ve got here, really like what you’re saying and the way by which you say it. You make it entertaining and you still take care of to keep it wise. I cant wait to read much more from you. This is actually a great web site.| а

  2. Have you ever thought about publishing an ebook or guest authoring on other sites? I have a blog based on the same subjects you discuss and would love to have you share some stories/information. I know my subscribers would value your work. If you are even remotely interested, feel free to shoot me an e mail.| а

  3. Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter updates. I’ve been looking for a plug-in like this for quite some time and was hoping maybe you would have some experience with something like this. Please let me know if you run into anything. I truly enjoy reading your blog and I look forward to your new updates.| а

  4. I have been exploring for a bit for any high quality articles or blog posts on this kind of area . Exploring in Yahoo I at last stumbled upon this website. Studying this information So i’m satisfied to convey that I’ve a very excellent uncanny feeling I found out exactly what I needed. I so much surely will make sure to do not omit this website and give it a look on a continuing basis.| а

  5. I blog often and I genuinely appreciate your information. Your article has truly peaked my interest. I am going to bookmark your site and keep checking for new information about once per week. I subscribed to your RSS feed as well.| а

  6. Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter updates. I’ve been looking for a plug-in like this for quite some time and was hoping maybe you would have some experience with something like this. Please let me know if you run into anything. I truly enjoy reading your blog and I look forward to your new updates.| а

  7. After I initially left a comment I appear to have clicked the -Notify me when new comments are added- checkbox and from now on whenever a comment is added I get 4 emails with the same comment. Perhaps there is a means you are able to remove me from that service? Thank you!| а

  8. Good day I am so happy I found your blog, I really found you by error, while I was looking on Yahoo for something else, Nonetheless I am here now and would just like to say thanks for a marvelous post and a all round interesting blog (I also love the theme/design), I don’t have time to read through it all at the moment but I have bookmarked it and also added in your RSS feeds, so when I have time I will be back to read a great deal more, Please do keep up the great work.| а

  9. Heya i’m for the first time here. I came across this board and I find It really useful & it helped me out a lot. I hope to give something back and aid others like you helped me.

  10. Write more, thats all I have to say. Literally, it seems as though you relied on the video to make your point. You clearly know what youre talking about, why throw away your intelligence on just posting videos to your blog when you could be giving us something enlightening to read?| а

  11. What i do not understood is in truth how you’re not actually much more neatly-preferred than you may be right now. You’re very intelligent. You already know therefore significantly with regards to this matter, made me for my part consider it from a lot of numerous angles. Its like men and women are not fascinated except it’s something to do with Girl gaga! Your personal stuffs nice. At all times maintain it up!| а

  12. You actually make it appear really easy along with your presentation but I in finding this matter to be actually something which I think I’d by no means understand. It seems too complex and very extensive for me. I am taking a look ahead for your subsequent publish, I will try to get the grasp of it!| а

  13. Unquestionably believe that that you said. Your favorite reason appeared to be at the web the easiest factor to consider of. I say to you, I certainly get annoyed whilst people think about worries that they just don’t recognize about. You controlled to hit the nail upon the highest and defined out the whole thing with no need side effect , other people could take a signal. Will probably be again to get more. Thanks| а

  14. My programmer is trying to convince me to move to .net from PHP. I have always disliked the idea because of the expenses. But he’s tryiong none the less. I’ve been using Movable-type on numerous websites for about a year and am anxious about switching to another platform. I have heard great things about blogengine.net. Is there a way I can import all my wordpress content into it? Any help would be greatly appreciated!| а

  15. I precisely had to say thanks all over again. I do not know what I might have undertaken in the absence of these methods discussed by you directly on such a question. This was a very traumatic matter in my view, nevertheless noticing the very skilled tactic you treated it forced me to weep with gladness. I’m happy for this service and then hope that you comprehend what a powerful job you have been carrying out training people today through the use of a web site. More than likely you’ve never encountered any of us.

  16. You actually make it seem so easy with your presentation but I find this matter to be actually something which I think I would never understand. It seems too complicated and very broad for me. I’m looking forward for your next post, I will try to get the hang of it!| а

Leave a Comment