1,258 CEOs responded with the following:

43% – Our talent-related expenses rose more than expected
31% – We weren’t able to innovate effectively
29% – We were unable to pursue a market opportunity
24% – We cancelled or delayed a key strategic initiative
24% – We couldn’t achieve growth forecasts in overseas markets
24% – We couldn’t achieve growth forecasts in the country where we are based
21% – Our production and/or service delivery quality standards fell

Read the full study here: http://humancapitalleague.com/Home/22153

read more here:

http://online.wsj.com/article/SB10001424052970203750404577173031991814896.html

First, expand the Options link under each question and check the box to make the question only accept text answers.

When the candidate reaches this question, they will only see a text box – no video required. In the candidate management dashboard, there’s now an icon specifying the type of answer (top is text answer, bottom is video answer):

This feature is already deployed and ready for use today – give it a try! If you’re not yet a customer, get started today for free and stay tuned for more great features coming soon!

We’re happy to introduce a feature that many customers have asked for: automated phone interviews. Now, if a candidate doesn’t have a web-cam or are experiencing problems configuring their machine, they have the option of continuing with a phone-based interview (a text-only option is also available).

In the phone-based interview, candidates call in to our toll free number, where the questionnaire will be read to them one question at a time. The candidates’ answers are recorded and uploaded to our servers. The recruiting team can then listen to the answers right from their Active Interview management console, just like they would with video answers.

Click through to hear the results of an example phone interview:

http://cglee.activeinterview.com/share/3218caa05965512061cb3d8b85

For our existing customers, no action is necessary to activate this – it’s available immediately to all your candidates. This is just one of many features in the pipeline, so stay tuned to this blog or our Twitter for updates!

First of all, props to Marco for writing that post.  It’s a selfless act to blog about your technical processes and methods, and especially when it opens you up to “armchair hacker” criticism.  Sharing of these kinds of different methods is really valuable to us DIY-ers / lean start-up guys and gals.  It’s in that context, and in the context of horror stories like Mag.no.lia (total data loss) and Github (testing vs production db wires crossed) and CodingHorror.com (images lost) that ActiveInterview just audited our data protection & backup methods.  These are smart people running awesome sites –  if it can happen to them, it’s worth stepping back for a few days to review what we’re doing to prevent a similar scenario from happening to us.

Our service at Active Interview does require an additional layer of backup that (I think) is different from Instapaper, since our application works around not just data stored in a DB, but also a large amount of files associated to candidate interviews (videos, video thumbnails, resumes) as well as our customers file data (company logos, etc).

Our constraints:

1. We outsource our storage and server infrastructure to Amazon (read our AWS Case Study).
2. We’re a lean start-up. Backup are managed by developers (and we like it that way).
3. We use mostly open-source / commodity technologies.

7 Rules for Data Protection (and how we achieve them)

1. All user generated file data should live in at-least two secure places at once.

Even though we use S3 for our user generated content data store, and S3 has data redundancy baked-in, we go a step further and backup data “off-site” from Amazon with another vendor at a non-Amazon data center.  We’re hedging against not just our vendor’s technical issues, but also operator error on our end.  Whenever we move data between servers, all transfers occur over SSH (password-less) with scp/Rsync.  Rsync has a variety of friendly option to help you from overwriting/deleting files on your target server, which is a nice precaution.  We regularly ping our backup server and use a Webmin system check to test that it’s there and healthy.

2. Database data should live in two secure places at once.

At the simplest level, this can be solved via Master/Slave replication.  This works great for if the master dies, but isn’t going to help if erroneous code deletes data on Master, as that DELETE will quickly replicate over to the Slave.  Binlogging as Marco describes can solve this problem in simple cases, as you can rewind/edit/replay the binlog.  Just in-case we can’t restore properly from a rewind/replay-ing of the binlog, we also do aggressive (hourly) snapshots of the database.  Note that you should Flush/Lock the database before you do a SQL dump, to ensure data consistency.  With MySQLdump you can use mysqldump with –flush-logs and –lock-all-tables.  It’s also important to recognize that mysqldump is only a realistic option when your database is fairly small.  Dumping data is fairly quick with large DBs, but it’s the import time that will kill you.  For backing up and restoring sizable databases, look into using a binary snapshot tool like InnoDB hotbackup.

As we take our hourly db sql dumps, we copy them over to a S3 bucket dedicated to backup files.  And then we Rsync the same db backup file to the offsite (non-Amazon) server.

3. In-case of disaster, be able to restore everything to a running state in less than 3 hours.

There are several ways we can do this, depending on the nature of the disaster.

• If it’s a DB loss, we can flip over to the slave server. Or restore from hourly snapshot, which ensures we only lose at most one hour of data.
• If it’s user genereated content loss from S3, we can Rsync from our off-site backup server.
• If our EC2 instance goes down for some reason, we can bounce back with our original server drive attached to a new instance.  This is possible because we’re using an EBS root volume (you should probably be using this too, instead of EC2′s original “instance store”).  EC2 was initially designed such that your root data drive would not survive instance termination, accidental or otherwise.  With an EBS root drive, the data drive exist independent of the EC2 instance  and you able to detach/re-attach an EBS volume to any instance.
• If for some reason the “live” production EBS drive cannot be re-attached to a new EC2 instance, not to worry.  We capture EBS volume snapshots of the entire drive twice a day with 14 days of archives, using this excellent, lightweight & chron-able script from Eric Hammond: http://alestic.com/2009/09/ec2-consistent-snapshot.  Note that for your EBS snapshot to be bullet-proof using that script, you should use an XFS filesystem, which allows locking.  Eric’s script can be used with or without the XFS file systems, though if you don’t use XFS, you run the risk of not having a consistent file system on EBS volume restore.
• And finally, if all of the above fail to provide a path to restore the web-application server, we have literally every step needed to provision a new instance, from scratch, fully documented in a Python Fabric script.  This “fabfile” automates the process of provisioning a new server with all necessary packages, all user accounts, services and dependencies with one simple command: “fab -h <target>”.  Capistrano/ Bundler and RVM provide the final automation for easy deployment and automatic dependency management of our Rails application layer.  If you are on Rails 2.x, start using RVM/Bundler today!

4. Document & audit the recovery process at least once a month.

We document everything needed to recover any part of our data or infrastructure in a Google doc that is shared among the dev team.  The responsibility to audit the backup/restore process rotates each month across team members.  We run through the restore process every month… cost of firing up a new EC2 instance for that afternoon ~$3.  Cheaper if you use spot prices 

5. Database permission should be as granular as possible.

This mean the primary webapp does NOT use root access.  This is a pretty common development practice that frighteningly seems to creep into production setups.  Spend the extra 60 seconds setting up a non-root account w/ appropriate ACLs.  Since we use DB backup scripts, those scripts have their own users with read-only (SELECT only) rights.   Remote access from the office to the production DB is allowed, but only through password-less SSH, and again only with a SELECT only db account.  Basically if you are going to insert/delete/edit anything in the production DB, you’re going to have to go through root access from the shell.  We logwatch all server activity to monitor any suspicious behavior.

6. File level/S3 permissions should be granular as possible.

Pretty obvious, but you don’t want to have write or execute access to any part of the application or data directories on the web-server for system users that don’t need that access.  You’re playing with fire if you “chmod 777″.  Similarly, when files are copied over to S3 buckets, the appropriate API call is made to set the correct ACLs on that S3 object.

7.  Prevent production data leakage.

As a part of Marcos posted method, backups are also sent to his home computer.  I strongly disagree with that process.  As mentioned by many comments on HN, this is a direct vector for data loss.  Even if your service doesn’t appear to store anything confidential, IMO there is a reasonable and implicit expectation from consumers that their data is not residing on a home or laptop computer.  Since people re-use password all the time, all it takes is a lost/stolen table of passwords mapped to web-mail email addresses to create some real havoc, despite the application not seeming to store anything confidential.  Even many encrypted passwords are vunerable given enough time.

Unrelated to backups: It’s reasonable that developers may want production data on their development boxes to play with, which may be laptops they take home with them.  In this case, we’re thinking of writing a script that either scrubs or randomizes the data when it gets pulled down from production into a dev environment.

So there you have it –  7 simple rules for data protection.  Here is a corresponding diagram of our described backup architecture:

Read the full article here: The video job interview: When your face precedes the face-to-face.

“
The virtual interview was the only way Bryan Stuart could keep his non-profit organization’s budget in check when he hired 35 people for One Laptop Per Child’s internship program. It also was much easier to get candidates to submit recorded videos, which he could review at his home office in Wilmington, Ind. whenever he had spare time.

With plans to deploy interns around the world to co-ordinate laptop distribution programs, language skills were critical. While many cited proficiency in multiple languages on their resumes, Mr. Stuart knew there was one way to test those claims: one of the interview questions required applicants to answer in a second language.

That question startled applicant Federico Volio, a student in Atlanta, Ga. who was visiting his family in Costa Rica at the time, but it also impressed him.

“It really just gives you a good impression of how much they care about who you are,” he says.

He rattled off an answer in Spanish with ease which, in part, won him the internship.

”

“It’s going to be a long day people.”

We’re big fans of Amazon’s amazing cloud services. And like some other exceptional webapps, we use them for much of the underlying infrastructure that powers Active Interview. In our case, AWS is the building block of our video capture, storage and serving features. We were more than happy to participate in a case study for them and are really happy with the final writeup:

Read the full report here: http://aws.amazon.com/solutions/case-studies/active-interview/