Wednesday, January 1, 2014

Unstandardized standards are the worst: sendmail

Implementing software to replace legacy systems is always a challenge, especially when you're dealing with a system with as much legacy as sendmail, which was first introduced as delivermail in 1979.ref

Each UNIX vendor, it seems, rewrote or heavily customized sendmail. This has lead to sometimes conflicting implementations.

Case in point: -t

Normally, you invoke sendmail(8) with a series of arguments indicating the subject of a message, the recipients, etc. When invoked this way, the command expects a message on standard input, waits for EOF, and then sends your message along.

However, sometimes you don't want to have to fiddle with command-line parameters; you've already written a perfectly fine message with headers.

-t is generally passed to sendmail when you want to build a message envelope from an already-formatted message, with headers, etc. For example, if you had a file foo.txt with a body like this:
From: Luke Faraone 
To: John Smith 
Subject: Hello, world!

Hi there.

you could send the message with a simple invocation of cat foo.txt | sendmail -t. The system would take care of ensuring a Message-id was appended if appropriate, and queue the message to be sent. However, it is when you do slightly more complex invocations of sendmail that things get ambiguous.

It turns out that implementations differ on what exactly it means when you use -t in combination with naming destination addresses after the arguments to sendmail. exim4's documentation describes the situation in greater detail:
extract_addresses_remove_   argumentsUse: mainType: booleanDefault: true
According to some Sendmail documentation (Sun, IRIX, HP-UX), if any addresses are present on the command line when the -t option is used to build an envelope from a message’s To:Cc: and Bcc: headers, the command line addresses are removed from the recipients list. This is also how Smail behaves. However, other Sendmail documentation (the O’Reilly book) states that command line addresses are added to those obtained from the header lines. When extract_addresses_remove_arguments is true (the default), Exim subtracts argument headers. If it is set false, Exim adds rather than removes argument addresses.

Thus, there's basically no mechanism for a program to know which behaviour to expect. God forbid two programs are installed on a system that expect different behaviours!

It appears that the default behaviour of Ruby is the opposite of what exim4 (Debian's default mail client) expects. This has resulted in numerous bug reports. Some replies suggest changing exim4's defaults, while others advocate overriding ActionMailer and friends to use sendmail -i instead, without -t.

That said, its not really clear who's wrong here; at no point does there appear to have been a definitive specification for sendmail, and as such we can hope for defined behaviour by common custom at best, and a sea of incompatibility bugs at worst. Amusingly, POSIX standards have nothing to say on this subject of sendmail at all; it defines that a mailx command must exist, but says that its sending mode may be implementation-specific.

As Matthew Garrett writes, there's not enough gin in the world.

Saturday, November 9, 2013

Why I use my bank's mobile site on my desktop

(or, cutting out bloat by using a platform where bloat won't fly)

Let me start off by saying I'm generally a huge fan of my bank, USAA. Their offerings are free of hidden fees, their phone support excellent, and the perks they provide are competitive. They don't have the best savings interest rates, but you can always find a better deal online to park money not actively in your checking account.

However, USAA's website is a behemoth. My account page took about 8 seconds to fully load, downloading 1.4MiB of content.

The "My Accounts" page you're redirected to after logging in.
It is frequently buggy; whenever I log in via Google Chrome on Ubuntu 12.04 I land on a page with a URL beginning with "https://www.usaa.com/inet/gas_bank/AccountBannerAjax" and a bunch of GET parameters like "currentaccountkey" and "accnumber" with values like "encrypted12a1f4dd1[…]". The server returns a 200 OK, promises a Content-Length of 20, but then actually returns zero bytes. After navigating to the homepage and clicking a button, I end up getting logged in, but I wonder what percentage of their userbase are experiencing this problem?

For some strange reason, I get a lot of checks. It appears that nobody else informed the banking system that it's 2013, and the easiest mechanism for people to send money without paying fees is still on paper. To its credit, USAA made remote deposit of checks available to all customers in 2006, when it was mostly an offering limited to businesses. However, it seems like they haven't updated their web workflow since then. 


Using it on the web still requires using a signed Java applet (itself discouraged by CMU's CERT) that does the incredibly complex task of… letting you select a file from your computer and upload it to their servers. At least, that's what I think it does, because any time I chose "Run", my browser complained a few minutes later that the tab had stopped responding. Regardless of functionality, you can accomplish almost anything their site could currently be doing with HTML5 and a third party service if they want to crop images locally.

Spinning after logging
in on Android
USAA's mobile app for Android has another host of problems; I haven't been able to log into it for 2 weeks, and when I chatted with someone today I was told they were "doing some maintenance this weekend", so I should try again in a few hours once that's finished.

I googled around a bit for some way to perhaps make the applet work in Ubuntu (which admittedly is not a supported platform), and came upon a Facebook thread where a rep suggested using the mobile web site.

A breath of fresh air
I loaded it in my browser, and was amazed at how well it functioned. Obviously designed for higher-end devices (It didn't even load in one WAP emulator I tried), the mobile web interface was a refreshing breath of fresh air. It scaled well to a full-screen device (see below), loaded quickly, and gave me all the information I would have wanted out of the normal web interface.

Most notably: remember the whole "upload a check" workflow that required a buggy Java applet on the main website? We get bog-standard HTML form fields, no additional magic. There goes any theories about the Java client doing some magic validation or prep of the image; here, all they're getting is the images and my session cookie.


I'm still shocked at whoever thought a My Yahoo!-style homepage was the best layout for a bank, but props to the web developers who managed to make a mobile interface that was both pretty and allowed me to work around broken functionality in their implementations on every other platform I had access to.

But why was the mobile web interface the least bloated? Easy. On the desktop, you generally have a nice pipe, or if not, the user knows it and won't be too upset if your site is just as slow as other sites similarly situated. On mobile, the user downloaded all the code already, so the only latency should be the API requests against the server, right?

On the mobile web users have come to expect relatively speedy mobile-optimised sites and there's less screen real estate to do fancy things that get in the way of content. For many sites, that's a huge improvement. Of course, it would be really nice if more banks supported open protocols for interactions (USAA has a read-only, limited-duration OFX feed), but I would settle for a better web interface.

So tl;dr: USAA, please make www.usaa.com redirect to m.usaa.com, kthxbai.

Saturday, July 20, 2013

Joining the Debian FTPTeam

I'm pleased to say that I have joined the Debian FTPTeam as of the Friday before last. See Joerg Jaspert's announcement on debian-devel-announce.

The FTPTeam is responsible for maintaining the Debian software archive, and ensures that new software in Debian is high-quality and compliant with our policies.

As an "ftpassistant", I (along with PaulScottGergely, and others) will be helping to process the NEW queue, which is currently at a whopping 297 packages. Here's hoping we'll be able to get that number down over the coming weeks!

Wednesday, April 3, 2013

Teaching free/open source to high school students

A few weeks ago I taught a class on Open Source: Contributing to free culture (catalog entry) for Spark, a one-day program put on by the student-run MIT Educational Studies Program. I was fortunate to have two helpful co-teachers, Tyler Hallada and Jacob Hurwitz, who assisted with the lesson plan and the in class lecture.

We ended up teaching 3 sessions of the 1hr 50min class that Saturday, with about 10 students in each session.

I was pretty impressed by the quality of the students; a number of them had used GNU/Linux before, but even those who hadn't were able to gain something from the experience. The class was broken up into three segments:

  1. Lecture on a brief history of open source and the free software movement
  2. Small research project on an open source project
  3. Lab where students could work through OpenHatch's training missions
The point was to mix up what could otherwise be a very boring lecture.

I think we might have missed the mark on the last bit, as I get the feeling that we didn't end up giving the students good actionables. While the quality of OpenHatch is high and the organization's campus outreach programs are amazing, skills practice only goes so far without clear direction to apply said skills. I'll be following up with the class participants to see how they're progressing on their own open source contributor journey, and will post updates if I have any.

While not an OpenHatch event, if this sort of thing interests you, OpenHatch runs a series of events like this one and has a mailing list for discussing planning and sharing best practices. Subscribe and say hi!

The presentation is enclosed below, and of course is licensed under CreativeCommons Attribution-ShareAlike 3.0. [PDF]


Monday, October 8, 2012

Where I've gone off to

For those of you back at my university, you may have noticed I'm not there this semester.

Google's 2012 FEP team

I spent this past summer at Google, Inc, developing internal tools for the AdSense team, specifically Google Programmable Ads. Being at Google was great; I loved my team and my intern cohort, the rest of the interns in the Freshman Engineering Practicum.

I'm not currently at Google; my internship ended at the beginning of August. Although it was a blast, all good things must come to an end.

I'm not at Mason, either. I was super excited to return to university, and was just about to buy my books and get ready to move in when I got an email from a former coworker at Ksplice, a startup Oracle acquired while I interned there last year. He was starting a new company which would focus on business communications. I'd be working with a bunch of my former coworkers, and based on what I had heard of the company's plans I was confident in their ability to make an awesome product.

Needless to say, when they decided to offer me a position working there full time, I jumped on it.

From an academic point of view, Mason didn't really have much of a mechanism to support this. Co-ops are uncommon there, and not really supported for more than one semester; a full year away had never been done, according to our career services. My department was fully supportive, however, so we managed to find a way to make it work. This involved filling out some oddly-named forms, such as Special Registration for Graduation Request, which the registrar asserted was the right form, trust us on this one.

Myself and Obey Arthur Liu
at a SIPB hackathon
But that's all neither here nor there. I'm now up in Cambridge, MA, working with awesome people (including but not limited to tabbott, wdaherjesstess, and keegan), and exploring the city. I'm hanging out with MIT SIPB, helping with the maintenance of Sugar Labs' servers in E15 and spending more time working on various open source projects.

To my friends at Mason: I miss you all. I know regardless of how the next year+ turns out, it'll be one one hell of a ride.

Sunday, April 15, 2012

AMPED Status Update




This semester, I created an Android application which processes approval of transactions on the internet in conjunction with a web interface. I walked through the AMPED web interface and server code with my faculty mentor, and we conducted a security analysis of the protocol used in the project. The end result is an application which can be used to handle arbitrary authentication requests, as generated by a server. It performs secure authentication of these requests, and cryptographically signs approved transactions as per the initial proposal. I also produced a reusable Java library which can be incorporated in other projects.
Source code for all parts of the project are available on the web. The code has been released under the GNU General Public License, which allows for free use, modification, and redistribution under certain terms.


The application is static, and programmatic provisioning for, say, wide deployment, isn’t possible at the present time. In future research I would like to investigate mechanisms for securely provisioning new devices so that we could limit dependence on a secure channel for initial communication, while maintaining the non-repudiation capabilities of AMPED. For example, some combination of a QR code (for an initial shared secret) and a device’s cellular radio could be used to ensure a secure key exchange. After receiving a shared secret, the device would then generate a secret key and transfer the public portion to the server, using the shared secret as an ephemeral key for this purpose. This is similar to what’s used in Google Authenticator, but GA requires both the server and the client be able to generate valid authentication tokens, preventing non-repudiation.








Integrating AMPED into existing software systems would also be potentially fruitful, as this would allow us to see how the project would function under normal usage conditions. To this end, Dr. Simon has expressed interest in the possibility of using a future iteration of AMPED in the administration of wireless sensor networks.


AMPED was developed with funding from the George Mason University Office of Student Scholarship’s Spring 2012 research grant programme, URSP.

Friday, December 16, 2011

Semester in review

You have successfully logged in. Please do NOT close this page, or you will lose access. Open a new tab or window to connect to Internet sites. Please notice that you will only have LIMITED network access until you upgrade your system.
I've been pretty quiet online these past few months. My first term at George Mason has been a blast; but I've been super busy.


In that time I've joined Student Government as the Undersecretary for Information Technology, where I've worked on improving our campus residential and wireless network along with the really awesome people at ITU TSD.

I became a UTA for the Computer Science I class at my university, and helped organise several late-night extended tutoring sessions in the days before project deadlines.

I also took over part of the maintenance responsibilities for the Computer Science department's faculty supercomputing cluster, and began work on developing a cluster for student use.

During this winter break it looks like I'll  be in Cambridge for a couple weeks during IAP. Among other things, I'm running an event with SIPB.

Next semester, I'll be participating in the Undergraduate Research Scholars Program extending my work on mobile device approval-based authentication with Dr. Robert Simon. Here's to the future!