Macchiato

This past June, I attended my very first WWDC. The conference, the people and the parties were all amazing, and it was definitely a highlight. Inspired by the spirit of the conference, and all the new technologies presented, I set out to conquer my laziness and build and ship a new app.

I’m a huge fan of Markdown. So much so that I write nearly everything in it. From emails and notes, to documentation and blog posts. Unfortunately, writing Markdown meant one of two things for me: either launching TextEdit and switching it to plain text mode, or launching TextMate and writing in a code editor. Neither were really suited to the task.

To remedy this, I built Macchiato. I made full use of Lion’s new technologies. In fact, Macchiato only works on Lion. You’ve got full-screen mode, auto-save and versioning. The internals of the app uses NSRegularExpression, sandboxing, Automatic Reference Counting, and several other Lion-only APIs.

Macchiato is about being the very best at doing one thing: writing in Markdown. I’ve tried to keep an emphasis on usability, design and typography. I wanted to make it a joy to use, and for me it did the trick. I use it every day.

Check out Macchiato!

Paul Carr:

To make money — real money — at this game you have to attract millions, or tens of millions, of users. And when you’re dealing with those kinds of numbers, it’s literally impossible not to treat your users as pieces of data. It’s ironic, but depressingly unsurprising, that web 2.0 is using faux socialization and democratization to create a world where everyone is reduced to a number on a spreadsheet.

I’m having a hard time keeping the quoting-paul-carr to actual content ratio reasonable, but as with so much of his writing, this is spot on.

(tip of the hat to Nik Fletcher for reblogging this first)

I do not pride myself in my skills as a programmer. Complex algorithms scare me, and I stay away from them as much as I can. Rather, what I am good at is creating elegant solutions to problems. I’m good at imagining how things could work together, how something could be improved through programming. I’m good at building real-life products and streamlining processes. Breadth, rather than depth.

I also do not pride myself in knowing a language fully. I am good at understanding concepts, code and documentation, and when asked a specific question in a job interview I am not afraid to say that I don’t know, but could easily figure it out with 5min and access to the internet.

What I’m getting at is that I value resourcefulness, scrappiness and creativity more than knowledge and intelligence. This is why I’m not taking a computer science course, but rather am instead studying alternate approaches to problem-solving in the world of design. I think it will be a lot more valuable in my life in building an intellectually well-rounded personality, and an ability to pick up new skills quickly and bend them to suit my purposes. In the end, what truly matters is what I’m able to create, not how intelligent I may be.

Running a Modern Startup on PHP

I originally wrote this for the ChartBoost Blog.

In the modern world of agile startups and silicon valley, the buzz is all about Ruby, Python, and whatever the latest cool programming language or framework to come out is. Older technologies don’t get much love, and PHP especially has a bad reputation. In this post, I’m gonna go over why and how we use PHP as a modern technology, and the various other tools, techniques and development methodologies we employ to run as agilely and elegantly.

PHP

PHP is regarded as a clumsy and amateurish technology, best left to development newbies and $5-an-hour consultants. It’s almost bad enough to make me feel shame when I tell people we run on PHP. However, I don’t think this reputation is entirely deserved.

The language itself is, after Perl, the oldest language to be adopted en-mass as a web technology. Its roots are as a text pre-processor, and over the past 16 years it has evolved from that into something much broader. Many of its fault stems from the way it has evolved, rather than being designed the way it is today from the ground up.

I’m not going to argue PHP is the best language—it clearly isn’t. Frankly, it’s a mess. There’s no consistency in function and class names, even within the core library itself. The Object-Oriented features were tacked on at a later point and, while they’re getting better, are somewhat fragile. Here at ChartBoost, the core requirement is that we run on at least PHP 5.3, which introduced Late Static Bindings. Before that, building serious object-oriented code in PHP was impossible.

Even for all its faults, PHP remains a major player online, and some of the most impressive technology companies (like Facebook) are using it. PHP remains one of the fastest language to code with, deploy and execute. Lastly, while this is mostly due to personal preference, I find its C-inspired syntax to be one of the best in the web development world. Braces, parenthesis and semicolons make it extremely clear what the code is doing, as opposed to Ruby’s mess of symbols, implied parenthesis and lack of statement endings.

MVC

It’s paramount for a modern web app to run on an MVC (Model-View-Controller) architecture. Unfortunately, PHP offers very little in terms of modern and agile MVC frameworks. The big ones (CodeIgniter, Symphony, etc.) are extremely bloated and actually tend to get in your way more than help. Also, most impose their vision of what the model & database layers should look like.

Paraglide

Fortunately, one framework stands out from the pack. Paraglide is a minimalist framework that takes care of routing requests to controllers, rendering views, and little else. It offers the basics in terms of setting up the environment, providing basic helpers and organizing your code. It also works on the command line and from a shell (more on this later.)

Believe me when I say this, but Paraglide in mind-blowingly cool. It makes coding in PHP as elegant, and in some ways even more elegant, than the equivalent in Rails. It’s faster and lighter weight than Rails, but is easily extensible and works with pretty much any other code or package you throw its way.

MongoDB

Another decision core to our design ideals was the choice of MongoDB as our main model layer. Mongo is an incredibly powerful and scalable database system. It’s fundamentally different from MySQL in that it is at its core a key-value store. Mongo is so incredibly efficient that we have in fact completely skipped the usually required step of using Memcached. Mongo also offers greater reliablility and safety than MySQL with features such as failure-proof replica sets, and a querying interface that’s invulnerable to injection attacks. Avoiding SQL altogether has also been extremely pleasant. One of Mongo’s biggest advantages is easy and powerful scaling through replica sets. When a node goes down, or is added, Mongo will automatically recognize it and rebalance itself, without causing any downtime. There is no single-point-of-failure.

MongoModel

A pet project of mine has been MongoModel, and it is what we use as the third leg of our architecutre. MongoModel is an ORM which uses Mongo as its datastore, and adds features vital to a full-featured web application. It provides object-mapping, relationships, validations and is extremely elegant to use. Much like with Rails’ ActiveRecord, sensible defaults are deduced, and it’s schema-agnostic. You do not need to setup or even define what your database looks like. Rather, you just use the objects and MongoModel takes care of everything else.

Unit Testing

While we don’t practice Test-Driven-Development, we do have unit tests in place. PHP does not provide an elegant test library, so we built our own (soon-to-be open-sourced.)

Shell Development & Scripting

Paraglide is, to my knowledge, the only PHP framework that works in command-line scripts and from an interactive shell. Script functionality is extremely important in order to run cron scripts and various other maintenance and administration tasks. Interactive shell access is a boon for quick development and debugging. We use PHP-Shell to interact with our code directly from the command line. This is quite similar to Rails’ script/console.

Git

Everything we do is stored in Git. Git’s virtues are well-known within the community, so I will only say that git has been incredibly useful in keeping track of our code, the history, and for collaboration. We even use git as a wiki, to keep track of our documentation and various other internal documents.

GitHub

All our git repositories are hosted on GitHub. The main value of this, besides the hosting and gorgeous user interface, has been to use the social features to keep track of who’s been doing what. GitHub makes it really easy to have an overview of what’s happening. It also manages user accounts and rights beautifully.

Capistrano

Our main server-side code lives in a Git repository. We have dedicated branches for production code. We use Capistrano for deployments. The git repository has a dedicated branch for production code, which we merge to as we deploy stuff. A script will automatically run unit tests on anything that is pushed to production.

Amazon Web Services

ChartBoost relies on Amazon Web Services’ many products, including EC2 for cloud servers, S3 for data storage, SES for emailing and various other smaller services. This lets us pay for how much we use only, and allows for simple and fast scaling. We have an image ready to be deployed to new nodes, so we can handle any traffic thrown at our app.

Communications & Internal Tools

Last but not least, there’s the tools we use internally to keep in sync. Lighthouse takes care of our bug-tracking needs, while its companion, Tender handles support. We use Campfire for group chats, and AIM for one-on-ones. Google Apps & Gmail take care of our emails. Also check out companion Mac apps Lighthouse Keeper for Lighthouse, and Propane for Campfire.

If you read this far, you now have a good overview of the various tools and techniques we use to code agilely at ChartBoost. Even though we chose an unpopular language to base our technology on, I think it has helped us tremendously. With this post, I hope to spread the love again for PHP and these various ways of using it in a modern environment.

The Azure License: Meaningful Attribution

I’m updating and re-releasing this from my old blog. Feel free to use the license in any project. No need to attribute me, the license itself is released into the public domain.

Open-source licensing can be a real pain. Some licenses are nearly impossible to decipher, while some (namely—the GNU GPL) are just pure evil.

I have been trying to find a software license which, like the Creative Commons Attribution license, would let the licensee do pretty much anything with the software, except it would require attribution in a meaningful way. That is to say, a non-intrusive mention in the documentation or about box.

The MIT license came closest to this, and it is the base on which the Azure License was written.

A good way to give attribution, as required by the license, would be a friendly “Contains code by Copyright Holder [linked]” or “Special thanks to Copyright Holder [linked]” in the about box.

The Azure License

Copyright (c) {Year} {Copyright Holder}

Attribute to {Copyright Holder} - {url}

You (the licensee) are hereby granted permission, free of charge,
to deal in this software or source code (this "Software") without
restriction, including without limitation the rights to use, copy,
modify, merge, publish, distribute, and/or sublicense this
Software, subject to the following conditions:

You must give attribution to the party mentioned above, by name and
by hyperlink, in the about box, credits document and/or
documentation of any derivative work using a substantial portion of
this Software.

You may not use the name of the copyright holder(s) to endorse or
promote products derived from this Software without specific prior
written permission.

THIS SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THIS SOFTWARE OR THE USE OR OTHER DEALINGS IN THIS
SOFTWARE.

http://license.azuretalon.com/

Meet ChartBoost

We’re a small team of Tapulous alumni who banded together to package up everything we learnt about marketing iPhone apps and turn it into an awesome product. We’ve worked on building, running and marketing hit apps and, through trial and error, figured out what worked and what didn’t. Now, we’re building a service that brings together the best cross-promotion and marketing techniques.

I haven’t had a chance to mention this yet, but I’ve left Tapulous to join two good friends of mine, Sean and Maria, at an exciting new startup in San Francisco city. I’m very excited to see what we can build here, and look forward to sharing some of what I learn on this blog in the future.

(Source: chartboost)

I Can Crack Your App With Just A Shell (And How To Stop Me)

Well, not you specifically, but by you I mean the average Mac developer. It’s too easy to crack Mac apps. Way too easy. By walking through how I can hack your app with only one Terminal shell, I hope to shed some light on how this is most commonly done, and hopefully convince you to protect yourself against me. I’ll be ending this article with some tips to prevent this kind of hack.

Disclaimer: I am fervently against software piracy, and I do not participate in piracy. Some will view this article as an endorsement of piracy, but rest assured that it is not. However, I do not believe that obscurity and ignoring the problem is an acceptable solution.

In order to follow along you’re going to need a few command line utilities. You’re going to need the Xcode tools installed. And lastly, you’re going to need an app to operate on. I chose Exces, a shareware App I wrote a long time ago.

Let’s start by making sure we have the two utilities we need: otx and class-dump. I like to use Homebrew as my package manager of choice. Note that I will use command line utilities only, including vim. If you prefer GUIs, feel free to use your code editor of choice, HexFiend and otx’s GUI app.

$ sudo brew install otx
$ sudo brew install class-dump

The first step is to poke into the target app’s headers, gentlemanly left intact by the unwitting developer.

$ cd Exces.app/Contents/MacOS
$ class-dump Exces | vim

Browse around, and find the following gem:

@interface SSExcesAppController : NSObject
{
[...]
    BOOL registred;
[...]
- (void)verifyLicenseFile:(id)arg1;
- (id)verifyPath:(id)arg1;
- (BOOL)registred;

What do we have here?! A (badly spelt) variable and what looks like three methods related to registration. We can now focus our efforts around these symbols. Let’s continue poking by disassembling the source code for these methods.

$ otx Exces -arch i386

Note that Exces is a universal binary, and that we need to ensure we only deal with the active architecture. In this case, Intel’s i386. Let us find out what verifyLicenseFile: does.

-(void)[SSExcesAppController verifyLicenseFile:]
[...]
+34  0000521e  e8c21e0100              calll       0x000170e5                    -[(%esp,1) verifyPath:]
+39  00005223  85c0                    testl       %eax,%eax
+41  00005225  0f84e2000000            je          0x0000530d
[...]
+226  000052de  c6472c01                movb        $0x01,0x2c(%edi)              (BOOL)registred
[...]

This is not straight Objective-C code, but rather assembly—what C compiles into. The first part of each line, the offset, +34, shows how many bytes into the method the instruction is. 0000521e is the address of the instruction within the program. e8c21e0100 is the instruction in byte code. calll 0x000170e5 is the instruction in assembly language. -[(%esp,1) verifyPath:] is what otx could gather the instruction to represent in Obj-C from the symbols left within the binary.

With this in mind, we can realize that verifyLicenseFile: calls the method verifyPath: and later sets the boolean instance variable registred. We can guess that verifyPath: is probably the method that checks the validity of a license file. We can see from the header that verifyPath: returns an object and thus would be way too complex to patch. We need something that deals in booleans.

Let’s launch Exces in the gdb debugger and check when verifyLicenseFile: is called.

$ gdb Exces 
(gdb) break [SSExcesAppController verifyLicenseFile:]
Breakpoint 1 at 0x5205
(gdb) run

No bite. The breakpoint is not hit on startup. We can assume that there’s a good reason why verifyLicenseFile: and verifyPath: are two separate methods. While we could patch verifyLicenseFile: to always set registred to true, verifyLicenseFile: is probably called only to check license files entered by the user. Quit gdb and let’s instead search for another piece of code that calls verifyPath:. In the otx dump, find the following in awakeFromNib:

-(void)[SSExcesAppController awakeFromNib]
[...]
+885  00004c8c  a1a0410100              movl        0x000141a0,%eax               verifyPath:
+890  00004c91  89442404                movl        %eax,0x04(%esp)
+894  00004c95  e84b240100              calll       0x000170e5                    -[(%esp,1) verifyPath:]
+899  00004c9a  85c0                    testl       %eax,%eax
+901  00004c9c  7409                    je          0x00004ca7
+903  00004c9e  8b4508                  movl        0x08(%ebp),%eax
+906  00004ca1  c6402c01                movb        $0x01,0x2c(%eax)              (BOOL)registred
+910  00004ca5  eb7d                    jmp         0x00004d24                    return;
[...]

The code is almost identical to verifyLicenseFile:. Here’s what happens:

• verifyPath: is called. (+894 calll)
• A test happens based on the result of the call. (+899 testl)
• Based on the result of the text, jump if equal. (+901 je) A test followed by a je or jne (jump if not equal) is assembly-speak for an if statement.
• The registred ivar is set, if we have not jumped away.

Since awakeFromNib is executed at launch, we can safely assume that if we override this check, we can fool the app into thinking it’s registered. The easiest way to do that is to change the je into a jne, essentially reversing its meaning.

Search the dump for any jne statement, and compare it to the je:

+901  00004c9c  7409                    je          0x00004ca7
+14  00004d9f  7534                     jne         0x00004dd5                    return;

7409 is the binary code for je 0x00004ca7. 7534 is a similar binary code. If we simply switch the binary code for the je to 7534, at address 00004c9c, we should have our crack. Let’s test it out in gdb.

$ gdb Exces 
(gdb) break [SSExcesAppController awakeFromNib]
Breakpoint 1 at 0x4920
(gdb) r
(gdb) x/x 0x00004c9c
0x4c9c <-[SSExcesAppController awakeFromNib]+901>:  0x458b0974

We break on awakeFromNib so we’re able to fiddle around while the app is frozen. x/x reads the code in memory at the given address.Now here’s the confusing thing to be aware of: endianness. While on disk, the binary code is normal, intel is a little-endian system which puts the most significant byte last, and thus reverses every four-byte block in memory. so while the code at address 0x4c9c is printed as 0x458b0974, it’s actually 0x74098b45. We recognize the first two bytes 7409 from earlier.

We need to switch the first two bytes to 7534. Let’s start by disassembling the method so we can better see our way around. Find the relevant statement:

0x00004c9c <-[SSExcesAppController awakeFromNib]+901>:  je     0x4ca7 <-[SSExcesAppController awakeFromNib]+912>

Now let’s edit code in memory.

(gdb) set {char}0x00004c9c=0x75
(gdb) x/x 0x00004c9c
0x4c9c <-[SSExcesAppController awakeFromNib]+901>:  0x458b0975
(gdb) set {char}0x00004c9d=0x34
(gdb) x/x 0x00004c9c
0x4c9c <-[SSExcesAppController awakeFromNib]+901>:  0x458b3475

Here we set the first byte at 0x00004c9c. By simply counting in hexadecimal, we know that the next byte goes at address 0x00004c9d, and set it as such. Let’s disassemble again to check if the change was done right.

(gdb) disas
0x00004c9c <-[SSExcesAppController awakeFromNib]+901>:  jne    0x4cd2 <-[SSExcesAppController awakeFromNib]+955>

Whoops, we made a mistake and changed the destination of the jump from +912 to +955. We realize that the first byte (74) of the byte code stands for the je/jne and the second byte is the offset, or how many bytes to jump by. We should only have changed 74 to 75, and not 09 to 34. Let’s fix our mistake.

(gdb) set {char}0x00004c9c=0x75
(gdb) set {char}0x00004c9d=0x09

And check again…

0x00004c9c <-[SSExcesAppController awakeFromNib]+901>:  jne    0x4ca7 <-[SSExcesAppController awakeFromNib]+912>

Hooray! This looks good! Let’s execute the app to admire our crack.

(gdb) continue

Woot! Victory! We’re in, and the app thinks we’re a legitimate customer. Time to get wasted and party! (I recommend Vessel nightclub in downtown San Francisco.) Well, not quite. We still need to make our change permanent. As it currently stands, everything will be erased as soon as we quit gdb. We need to edit the code on disk, in the actual binary file. Let’s find a chunk of our edited binary big enough that it likely won’t be repeated in the whole binary.

(gdb) x/8x 0x00004c9c
0x4c9c <-[SSExcesAppController awakeFromNib]+901>:  0x458b0975  0x2c40c608  0x8b7deb01  0xa4a10855
0x4cac <-[SSExcesAppController awakeFromNib]+917>:  0x89000141  0x89082454  0x89042444  0x26e82414

That’s the memory representation of the code, a whole 8 blocks of four bytes starting at 0x00004c9c. Taking endianness into account, we must reverse them and we get the following:

0x75098b45  0x08c6402c  0x01eb7d8b  0x5508a1a4
0x41010089  0x54240889  0x44240489  0x1424e826

The very first byte of the series is the 74 that we switched into 75. By changing it back, we can deduce the original binary code to be:

0x74098b45  0x08c6402c  0x01eb7d8b  0x5508a1a4
0x41010089  0x54240889  0x44240489  0x1424e826

Let’s open the binary in a hex editor. I used vim, but feel free to use any hex editor at this point. HexFiend has a great GUI.

(gdb) quit
$ vim Exces

This loads up the binary as ascii text, which is of little help. Convert it to hex thusly:

:%!xxd

vim formats hex like this:

0000000: cafe babe 0000 0002 0000 0012 0000 0000  ................

The first part, before the colon, is the address of block. Following it are 16 bytes, broken off in two-byte segments. Incidentally, every Mach-O binary starts with the hex bytes cafebabe. Drunk Kernel programmers probably thought it’d be funny. Now that we have our beautiful hex code loaded up, let’s search for the first two bytes of our code to replace:

/7409

Shit. Too many results to make sense of. Let’s add another two bytes. Search for “7409 8b45” instead and boom, only one result:

001fc90: 0089 4424 04e8 4b24 0100 85c0 7409 8b45  ..D$..K$....t..E

Edit it to the following:

001fc90: 0089 4424 04e8 4b24 0100 85c0 7509 8b45  ..D$..K$....t..E

Convert it back to binary form, then save and quit:

:%!xxd -r
:wq

And… We’re done! To check our work, launch the app in gdb, break to [SSExcesAppController awakeFromNib] and disassemble.

$ gdb Exces 
(gdb) break [SSExcesAppController awakeFromNib]
Breakpoint 1 at 0x4c90
(gdb) r
(gdb) disas

Admire our work:

0x00004c9c <-[SSExcesAppController awakeFromNib]+901>:  jne    0x4ca7 <-[SSExcesAppController awakeFromNib]+912>

Quit gdb and relaunch the app from the Finder, and bask in your leet glory.

How you can stop me

Objective-C makes it really easy to mess with an app’s internals. Try to program the licensing mechanism for your app in pure C, that will already make it harder for me to find my way around your binary. Also read this older article of mine on three easy tips—stripping debug symbols, using PT_DENY_ATTACH, and doing a checksum of your binary—you can implement to make it a whole lot harder for your app to be cracked.

A truly skilled hacker will always find his way around your protection, but implementing a bare minimum of security will weed out 99% of amateurs. I am not a skilled hacker—yet with some very basic knowledge I tore this apart in no time. Implementing the various easy tips above takes very little time, yet would have made it enough of a pain for me that I would have given up.

The Ultimate Solution For Xcode Auto-Versioning With Git

After struggling with several suboptimal solutions for years, I have finally come to find the best Xcode versioning solution for git users. First off, tip of the hat to Marcus Zarra and Johannes Gilger for posting their solutions, which inspired me in my search for the ultimate solution.

A couple advantages that make this solution better than those I’ve used in the past:

• It’s completely filesystem independent. Save for the git binary location requirement, this would work across any Mac with no additional setup. (It should also be quite easy to edit the script to detect git using which.)
• It works across clones and systems.
• Because the version is the current git SHA1 hash, it always refer to a specific commit you can get back to when debugging later.
• It processes the version number at every build immediately. Some of the solutions I’ve used in the past required a double-build, because of Xcode’s tendency to run scripts after the preprocessor. Not so here.
• No duplication of code in projects with multiple targets.
• Works for iPhone, Mac App Store and Mac apps.

So without further ado, my solution: I rely on an external target of type Shell Script which I call Versioning. Every other target sets Versioning as a Direct Dependency, ensuring its script is run before the preprocessor. Versioning contains the following Run Script:

cd "$PROJECT_DIR"
VERSION=`/usr/local/bin/git rev-parse --short HEAD`

cd "$PROJECT_TEMP_DIR"
echo "#define GIT_VERSION $VERSION" > revision.prefix

cd "$PROJECT_DIR"
touch Info.plist

In Info.plist, the CFBundleShortVersionString is set to GIT_VERSION. In the project’s main build settings, Preprocess Info.plist is turned on and Info.plist Preprocessor Prefix File is set to $PROJECT_TEMP_DIR/revision.prefix.

My Mac App Store Plans

The recently unveiled Mac App Store is, unfortunately, an affront to Mac developers. Importing the iOS’ closed model and applying it to the Mac, the new store carries with it all of the major disadvantages of its smartphone equivalent. I can’t stand the thought of losing thirty percent of my revenue, in return for which I get harsh restrictions on what I may do, week-long approval queues (even for critical updates,) a public expecting a ridiculously low price, months of revenue held off by Apple, and the risk of being cut out from said revenue at any time.

I’m not even going to get into how I will lose control over my own customers because Apple won’t let me send them promotional emails or sign up for an account. It won’t let me give away my software either, or have flexible pricing structure, educational discounts, volume discounts, promotions or giveaways.

Most excruciatingly, though, the App Store itself rocks. It’s beautifully designed and as a user, it is pure joy to use. Unfortunately, I am not willing to sign my soul to the devil to be a part of it. I was willing to deal with the limitations on the iOS platform, because it was presented as a new option, an is still somewhat reasonable on a smartphone. However, on the Mac, where we’ve been restriction-free for decades, this is most unwelcome. So here are my plans regarding the App Store:

1. My apps will always be available from my own store. This will function the same way it always has: Sparkle updates and email-based licenses and trial versions.

2. My apps will also be available on the Mac App Store (provided they get approved), crippled however Apple may wish. They will have a 30% price premium over the standard price, to make up for Apple’s cut. I will make sure that the standalone version is never inferior to the App Store version and I will advertise the cheaper, standalone version heavily.