Running selenium Grid using Docker + Kubernetes

Hopefully, these days more and more corporates are moving to SaaS based solutions (ie. Saucelabs and alike) for their test automation infrastructure. But for some reason, if you are running your own on-premise infrastructure using Selenium grid, continue to read.

Recently, I was doing a side project that was completely NodeJs stack for webapi, UI and everything. While I’m doing non-windows stuff, thought its wise to use Google Container Engine to host, run and maintain my containers. Part of this whole thing, I had to do little screen scrapping and can’t use something like Saucelabs. That’s where Gcloud and Kubernetes  came in very handy. Below is the setup..

Before we go too far, little bit Kubernetes 101 to set the context

Pod – group of one or more running containers.
Pod template –  describes the desired state of a single container. Desired state is important, coz, Kubernetes will use this description to make sure that the current state of the pod matches the desired state definition.
Volume – to persist the data across container lifecycle
Labels: key-value pair attached to each object in Kubernetes to group the resources together.
Replication Controller:  pod template + number of desired replicas = Replication controller.
Services: provide abstracted static IP address for pods selected by labels

 

 

Refer this get started with Google container engine, setup your cluster and node. I’m assuming that your node is setup and ready to go.

We are going to setup a Selenium hub and two selenium chrome nodes and configure them to talk to each other.

Selenium Grid hub setup

Hub Replication Controller – in essence, create a pod from the selenium-hub v2.50.0 image with ONE replication and expose port 4444 for tests to be queued.

Screen Shot 2016-02-21 at 9.28.53 PM.png

While this will start running the pods, there is no standard way to access this from outside. So, next 2 steps

Create Service

Screen Shot 2016-02-21 at 9.38.39 PM.png

Now, this will create a service but still not exposed to the Internet for you Queue the tests. So, run below command

Expose Service 

kubectl expose rc selenium-hub –name=selenium-hub-external —    labels=”app=selenium-hub,external=true” –type=“LoadBalancer”

Get services  – to get external IP, run below command

kubectl get services

You should see an output something similar (I’ve masked Cluster IP and External IPs)

Screen Shot 2016-02-21 at 9.42.15 PM.png

Test by hitting the external IP

http://IP:4444/grid/console

Selenium node-chrome setup

Goal is to setup 2 pods, expose port 5900 for selenium node <–> hub communication

Screen Shot 2016-02-21 at 9.54.01 PM.png

Now, if you go back to your grid console, you should see something like this

Screen Shot 2016-02-21 at 9.55.53 PM.png

Depending on your VMs capacity, you could replicate more agent pods to run tests. This works out fantastic for me and I hope this could save some money if you are still using on-premise selenium grid setup.

Perhaps, you could setup and tear down selenium-nodes on demand based on the requirements provided by your test suite instead of running agents 24/7.

All this source controlled here in Github

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s