Alex writes:

I would LOVE to see PyPI become a place that Mozilla felt confident it could use to deploy Python software.

I am happy about this conversation-starter and would like to contribute some of our reasoning for using a “vendor library” over pip requirements files. I, too, am not a huge fan of the vendor library, but understand how it solves some of the problems we’ve been having with plain PyPI at Mozilla.

Let’s discuss some of our main concerns and how the vendor library solves (or does not solve) them for us.

Risky dependence on PyPI and other third-party servers

In the past, PyPI has not always been the most stable of services. Even though PyPI has been vastly improved meanwhile, not all libraries we use can be pulled from PyPI, some come from code repositories on various services. Pulling in files at deploy time from numerous places all over the Internet is a huge risk, particularly if your deployment requires downtime and you need to be done as soon as possible. In other words: A setup that doesn’t allow us to (mostly) bundle one neat package before actually performing the deployment to our servers is a huge risk we’re not willing to take lightly.

Some, but not all of this, could be mitigated with a private PyPI mirror, though that comes with a maintenance overhead and does not help for non-PyPI libraries at all.

We deploy to, potentially, many servers

Second part of the “one neat package” issue: Running pip install on one server is fun and games. On 50? Not so much. It takes a long time, even if you run it on all of them in parallel. Quite possibly, a few of them will have hiccups (network trouble, for example) in the process, so good luck finding out which ones failed and manually fixing those before moving on.

Package “migrations”

Imagine an upgrade to an already-deployed application. How do you add two libraries, remove three, and upgrade five existing ones? Installing and upgrading might work with pip, for uninstalling we’d (probably?) need to write some sort of migration script, hope it all runs smoothly on all servers, then carry on. Or we blow away the virtualenv altogether for every push and reinstall all libraries from scratch.

This is many things, none of which is elegant.

Contributors

We want it to be reasonably easy for contributors to set up our code and start developing. With the vendor library, it basically boils down to a few git commands, while pip install is a much longer process. In addition, the “migration problem” is present here too: The developer has to constantly monitor changes to the requirements files (or realize that the source of an unexpected, new error is a missing library) and run pip as appropriate. In contrast, updating the vendor library is part of the regular pull-from-git process any developer has to do anyway to stay up to date with the main repository.

Vagrant mitigates some of these concerns by automating the setup process, but not all.

Compiled packages

One thing neither PyPI nor vendor solve is the deployment of packages with binary components (usually: C libraries with a Python wrapper). For that reason, we keep those libraries to a bare minimum and prefer pure Python variants wherever feasible. The few unavoidable compiled libraries (such as Jinja2 or py-bcrypt) we deploy to our servers through RPM packages (by sticking these requirements into a common Puppet file shared by all our webservers). That’s not too elegant, but as these cases are somewhat rare, it works. In any case, a server running the actual pip command to install these packages needs to have a full compiler toolchain, which production web servers are unlikely to have.

What it boils down to

The base concerns here circle around robustness and scalability, both for deployments to real-world servers and for various, globally distributed developers.

Really, the key is minimal human involvement: We love our developers, and we love our sysadmins. Any part of a requirements system with a high chance of repeatedly requiring human involvement is broken and needs fixed.

If you have any suggestions on how to improve this setup, I am all ears. As always, Mozilla’s code and processes are Open Source so we are happy to listen to your ideas!

This is me, reading a Firefox-themed QR Code with my cell phone.

After getting the idea from this excellent blog post, I made a QR code pointing to mozilla.org/firefox, sporting a nice little Firefox logo in the middle.

For the geeks among you who would like the 411, this works because QR codes have a certain degree of redundancy for error correction. The logo in the middle is considered an “error” and thus, the rest of the code is used to puzzle together the information in the code anyway.

Try it out with your smart phone (on Android, with an app like Barcode Scanner), it really works!

My Facebook session is SSL-encrypted by default, but Words with Friends requires me to disable this encryption. This is wrong on many levels. Most notably, if I disable “secure browsing” on Facebook altogether, even only for this session, my session cookie will be sent in plain text over the wire (or worse, on Wifi, over the air). If I do this at a coffee shop or airport, this is a great invitation for every evildoer in the general vicinity to hijack my Facebook account.

While I appreciate Facebook’s transparency in the matter, I find it upsetting that companies like Zynga wouldn’t account for Facebook users on SSL. By encouraging people to access Facebook over an unencrypted connection, they are foolishly endangering user data and are demonstrating an utter disregard for user privacy.

I wish Facebook enabled SSL encryption by default, and furthermore required third party apps to be served over SSL. You can’t have it both ways: Either you don’t handle user data, then you don’t need to care about encryption. Or you do handle user data (and yes, a session cookie counts!), then you need to properly secure it. I am tired of software makers weaseling themselves out of their self-imposed responsibility.

Update: As pointed out in the comments, moving to HTTPS for apps is on Facebook’s developer roadmap. I appreciate it!

“What, Mozilla makes websites? I thought all you did was Firefox.”

It’s usually followed by a second question: “what, your websites are open source, too?”

The basic misconception here is that Firefox is Mozilla’s mission. This is not true. Mozilla’s mission is outlined, in broad strokes, in the Mozilla Manifesto, and the core of Mozilla’s mission is to make the Internet better for the users (which goes beyond the Web and includes technologies like email, for example). And we believe that the best Internet for the users is one that inherently supports openness and choice.

Now don’t get me wrong. Firefox is important. Because the Web is the most visible thing that people like you and I are using the Internet for nowadays, Firefox is our most important tool to make the Internet better for the users. But it is not enough. The Web is not television. On the Web, the users are also the producers.

And this is why Mozilla’s mission goes beyond Firefox, beyond Thunderbird, deeply into the development and privacy space. A participatory Web based on Open standards, powerful APIs, along with the inherent freedom of choice and users’ control over their own personal data are what Mozilla is all about. Apart from Firefox, don’t be surprised to see Mozilla write state-of-the art, open source web applications and developer tools, be involved in the development of various open standards and play an important role in many other spaces that are relevant to the Internet today.

Oh no! Yesterday was the first day in 85 days of “Project 365″ that I failed to take and post a picture that same day. I blame Minecraft: During the day I was busy, but usually catch up on the photo taking before the end of day. But then I gave Minecraft a shot: It’s a fantastic little game about converting a world made of blocks into something awesome (and all the while staying safe from the monsters that come out at night). I like the gameplay and how–in this xbox and PS3-saturated world of ours–it’s a game that totally pulls off the 8-bit graphics.

Apologies for the delayed post and I’ll try to be on top of it again (unless I get sucked into the game agai………

Twitter’s fail whale has a little friend, a fail robot. Not sure how the two relate to each other, but it seems the whale comes out when twitter struggles under the sheer load of tweets, while the fail robot denotes service errors not caused by load?

Now, evil tongues would claim that there’s so much fail at Twitter that a single fail pet just doesn’t do it justice… But, knowing how challenging it is to scale a service to millions of users, I wouldn’t quite dare to say so — and therefore, I welcome the poor injured robot to my fail pet collection. Quick! To the fail pet emergency room!

Thanks to jabba for taking a screenshot for me!

What’s awesome is that Tumblr actually did end up using his artwork on their error page. This is what it looks like:

Mad props to the Oatmeal and Tumblr for this stellar addition to my fail pet collection!

And thanks to michaelk for pointing it out to me!

In the wake of a little server outage here at Mozilla, here’s my version of the comic:

(Based on the above xkcd web comic. Licensed under a CC by-nc license.)

But sometimes it’s the smallest things that have the biggest impact. Besides these hyped features, HTML5 also introduces a number of semantic form fields. Before, the only textual input the web knew was, well, plain text. It was up to the web application developer to enforce certain rules around that, like making sure the input is a number, or not empty, or even a valid website address (URL).

Firefox 4 understands these new input types and helps the user by enforcing correct values even before the users submits the form. By handling validation on the client, this enables a consistent form validation UI across websites and keeps the user from constantly submitting forms and wait for the server-side form validation to pass or fail. (NB: This does not relieve the developers of performing server-side checks in order to ensure the security of their web application).

Here is what this looks like in a recent prototype of the Firefox Input site:

Another fun little feature, also pictured, is the placeholder text attribute. The grayed-out placeholder in a text box shows you an example of what you might enter into this field. Rather than explaining correct values in a huge label or a side note next to the field, developers can show their users much more easily what data they would like them to enter into the form fields.

All of this makes for fewer mistakes entering data into web forms, which is both beneficial to the user (getting the job done faster) and the developer (collecting better data). Win-win!

For much more detailed on HTML5 forms, placeholders, validation, etc., take a look at Mark Pilgrim’s excellent Dive Into HTML5. Also, don’t miss out on Anthony Ricaud’s in-depth description of HTML5 forms in Firefox on the Mozilla Hacks blog.

Since I started working at Mozilla headquarters, my job interview volume has drastically increased — both in person and phone. As I had rarely conducted any job interviews before that, it was as much a learning experience for me as it was a critical part of their job search for the applicants. Initially, I was perhaps just as nervous as the interviewee.

The experience was mixed: Sometimes, candidates come up with surprising answers, elegant solutions to simple problems — or even just show that they have grasped a problem and its solutions from beginning to end.

Perhaps just as often, however, the experience is also increasingly frustrating. One of the worst things to notice is that the candidate can’t code. Seriously? Yes, seriously. Given a relatively simple programming question, they fail to come up with a solution either at all, or worse, they produce a solution that reveals shocking gaps in their knowledge of basic algorithms, understanding of code, or other qualities you would hope for in someone who makes software for a living and claims to have done so for awhile.

Another thing that disappoints me in candidates is when it turns out, they don’t have professional goals. Sure, you can’t be prepared for every question in the world. But if you are looking for a job, yet you are unsure where you want to go if you do get the position, how can this possibly convince me that you are the right person for the job?

For our web development team, this makes me wonder: Do people consider “web” in front of “developer” to be a weakening qualifier? Or more generally, does software engineering have a secret reputation of not being a real profession, with real professional requirements?

Mozilla is perhaps one of the fewer companies without a strict degree requirement: Some of our brightest minds have no formal college education, yet are incredibly successful and valued members of our community. The reason why this works out is that the Mozilla project, with many more volunteers than employees, is a meritocracy: If you do awesome things, people will respect you and turn to you for more awesomeness. If you aren’t doing a good job, your impact on the community will be minute (and stay that way).

Maybe, though, this is sometimes mistaken as an excuse not to be really, really good. It’s not about knowing everything there is about writing software. But if they are applying for one of the best positions in the industry, where they have to earn their respect rather than show off their formal credentials, shouldn’t they at least try a little harder?

If I could give a few tips to applicants across the Web industry, to perhaps raise their chances of getting a job, and to improve the interview experience for both interviewer and interviewee, I would say:

• Know your basics. There is a reason they teach algorithms and data structures very, very early in college. And complexity and logic. The works. Even more so, know your Web. If you can’t explain the nature of an HTTP request, how can you know your own applications by heart? If you don’t know how to secure a web application, how can you protect the privacy of your users?
• Know why you’re awesome. When an interviewer goes home that day, they want to feel excited about hiring you. Give them a reason. That’s no invitation to be snotty — but one to not hold back on exciting things there are to know about you professionally.

What’s your experience with interviewing and hiring people? What are the things you want to see in an applicant? And what have you done to find the people who are right for you? I am interested in hearing your opinions in the comments.