Cloud Operations Interview

What does a cloud computing expert need to know? How do you hire a cloud computing expert? Competition for operations & DBAs is fierce, so you’ll want to know how to find the best.

If you’re a systems administrator or ops guy, you may want to prepare for an interview for such a position. Meanwhile, if you’re a director of it or operations, a recruiter, or a manager in HR, you’ll want to have some idea of how to find the right candidate.

Here’s my guide to doing just that.

Questions to Ask During Cloud Operations Interview

Here are the questions that must need to be asked during the cloud operations interview:

1. Solid Unix Systems Administrator

At the top of the list, a cloud operation expert needs to understand Unix and more importantly Linux. Here are some sample questions to get the conversation moving:

What Is Web Operations and What Have You Done Day-To-Day?

Prepare some stories.

What’s Your Favorite Feature of the Linux Kernel?

This is an open ended question, but a systems administrator should have some knowledge here. The kernel is the most basic piece of software that runs when a computer boots up, whether it is a desktop or a server. This piece of software coordinates everything, manages resources, and directs traffic.

Name Some Distributions of Linux. What Is a Distro?

Linux is built by a collaborative team of thousands on the internet. That’s what makes it open source. The distributions include the operating system, along with a collection of software to go along with it. 

All the supporting utilities, libraries, and servers must be compiled and held in a repository. That’s what makes up a distribution. Debian, Redhat, and Ubuntu are a few popular ones.

A cloud operations expert needs to have a wide ranging skillset, from unix administration, architecture, scalability, database & webserver administration, troubleshooting & performance, and load & stress testing. You’ll also want someone who has learned hard lessons from some failures, has some war stories to tell, and has a hard nose for stability.

What’s the Difference Between Apache and Nginx?

These two pieces of software are both webservers, that is they respond to the HTTP protocol and can serve HTML pages. They also have a myriad of plugins to support different languages and features.

The difference? Nginx (pronounced engine-X) is a newer incarnation. It’s been rearchitected from the ground up, building on all the things learned from Apache over the years. Its tighter, more efficient code, and easier to configure.

You might also enjoy our Intro to EC2 Cloud Deployments Guide.

What Is a Key Value Store? Examples?

There are lots of examples of these types of databases. They are a very simple memory cache that can interface with most applications. Memcache is a popular example of a key value store. Redis, CouchDB and Voldemort can also do this.

What Is a Page Cache? Reverse Proxy Cache? Examples?

These are all the same thing. They are basically a very minimal webserver without all the plugins or bells and whistles. You put one of these in front of your webserver to handle all the easy stuff and speed up overall throughput. Varnish is a popular example.

What Filesystem Do You Prefer?

This is a bit arcane, but one should have some opinions here. xfs is a popular filesystem, though ext3 and ext4 are also common. Emphasize the journaling aspect here. Journaling means that if you pull the cord or your server crashes, the filesystem can recover upon reboot. It does this by journaling changes, much how a database keeps a redolog cache of recent changes to database tables.

Command Line Tools

There are lots of commands in the day-to-day toolbox of a web ops expert. Here are some examples:

rsync (pronounced our-sync) – sync files between servers & do checksums to allow easy restarts scp (pronounced s-c-p) – secure copy, similar to rsync but no checksums, so less reliable curl (pronounced kurl) – diagnose & test urls and HTTP from the command line cron (pronounced cron) – run commands at scheduled times ssh (pronounced s-s-h) – secure shell, the most basic tool to reach a cloud server ifconfig (pronounced if-config) – check the network interfaces on the server vi/emacs (pronounced v-i and e-macks) – terminal editors, to modify config files uptime (pronounced up-time) – display the current load average of the server top (pronounced top) – interactive display of system metrics like memory, load, swap & processes ps (pronounced p-s) – shows running processes on the server /var/log/messages – essential system logfile

What Are Application Servers? How Are They Different From Webservers?

Tomcat & Glassfish are two examples of application servers. These handle heavier weight languages & applications like Java. Application server on some level is just a more heavyduty webserver and these days Apache can be thought of as an application server also.

2. Cloud concepts

What Is Virtualization? What Is a Hypervisor?

Virtualization allows you to run one or more computers within a computer. You can do virtualization on a desktop, sharing network, memory, CPU and disk resources among a number of virtual servers.

But more importantly in cloud computing or IaaS offerings, you can do virtualization at the datacenter level. The hypervisor layer is a datacenter virtualization technology that provisions server resources, and balances shared network and disk resources.

What is an Image?

In Amazon the world, the AMI or amazon machine image is a snapshot of a server state at one moment in time. This image is taken at the block level, and includes the master block record, the first block on disk that a server boots from.

All that is the state of a server, when it is shutdown, is what is stored on disk or in this image. All config files, logfiles, and anything else writing to disk.

What is Multi-Tenant?

This means that there are multiple servers sharing resources. The tenants are the customers who each want to get the server, cpu, memory, network and disk that they paid for.

What Is the Downside to Shared Resources?

Contention for resources is always the challenge. If your fellow tenants are not very thirsty, this can work to your advantage. But if they’re also heavy users, the hypervisor layer has manage the balancing act.

You may get a spike of disk I/O at one point, but later get a dearth. This can cause a relational database like MySQL or Oracle to suddenly look stalled.

What is Instance-Store? What is Ebs?

Instance store servers were Amazon’s original offering, where servers had their own local (and slow) storage. This storage was ephemeral, so all machine state was lost on reboot. These servers also boot slowly.

EBS also known as elastic block storage is a virtualized storage option, similar to NAS or NFS. You can create arbitrary chunks of storage, and attach them to servers, all from command line APIs. Cool!

What is a Virtual Private Cloud?

With the VPC offering, Amazon drops a router into your existing datacenter. You can then provision virtual servers to your hearts content, and they all appear to be servers in your existing datacenter. Elastically scale, within the network and security model you’re already using.

What Is a Hybrid Approach to Cloud Adoption?

Keeping your investments in hardware and datacenter is obviously an appealing option for firms that have large existing environments. A hybrid approach with a VPC allows you to get your feet wet, but still keep essential applications on physical servers.

What is Amazon EC2?

Elastic Compute Cloud refers to the virtual servers you spinup in Amazon Web Services.

What is Amazon RDS, Oracle RDS, and MySQL RDS?

Amazon has various relational and non-relational database offerings. RDS stands for relational database service.

RDS or roll your own – which is better? Here are some use cases to help you decide.

What is Multi-az?

Amazon’s infrastructure offering isn’t just a single datacenter with servers. The beauty of what they’ve built is that they offer a number of datacenters (called availability zones) in each of many regions such as Northern Virginia, Oregon and Singapore.

Incidentally multi-az is a key feature to how businesses can protect themselves from failure. Amazon recently had an outage, but AirBNB, Reddit & Foursquare didn’t have to fail.

What Does a Cdn Do? How Does It Work? Examples?

A CDN is a content delivery network. Remember all those files that make up a webpage? Images, video, CSS files?

Turns out serving these components from servers *closer* to your customer, make their webpages load much faster. CDNs are networks of servers that hold the content of your pages, and serve them faster.

It works by replacing content paths with a special one from your provider. A simple change in your code will allow content to dynamically load from across the web. Cool!

CloudFront is Amazon’s offering coupled with S3 for file storage. Akamai is another big provider.

Here Are Some Questions To Pique Your Interest:

• Why does the API battle between Amazon & Eucalyptus (FOSS) matter?
• Do you use command line tools? why?
• What can go wrong with backups? how do we test them?
• Should we encrypt filesystems in the cloud? what are the risks?
• Should we use offsite backups?
• What is DRBD?
• Why is auditing important? access control?
• What is load balancing? why is it difficult with databases?
• How do you perform a benchmark? perform load testing?
• Why use a package manager? can we install from source?

Conclusion

So, that’s all the questions which can be asked during a cloud operations interview. However, we’re not done yet. We also have hit on other important skills a cloud ops expert should have including scripting, database administration (Our MySQL Interview Guide), scalability, performance, configuration management, metrics, monitoring, and some all-important war stories!