At that point, we had 88 submissions, and we just initiated an online "focus group" to help us select a winner from our top 10. We did that, and now I'll review the project wrap up process provided by crowdSPRING. The process is divided into four steps:

• Step 1. Pick a winner.
• Step 2. Approve your proof
• Step 3. Approve your final files
• Step 4. Rate your creative.

Here's how it worked for us, by step. 

Step 1. Pick a winner.

This step seems like it should be easy. Unfortunately our team couldn't agree on a single design, so we decided to award $200 for the second place design as well. We did this by adding a second award for $200 through the crowdSPRING site, which cost us a total of $230 ($200 + the 15% award fee). Here are the two designs that we picked:

We liked this one because it is simple and readable even at small resolutions like 16x16.  The image of the paper clip suggests organizing bits of paper, so it's appropriate for DoxCycle.

This is the corresponding splash screen:

In the end we had to get a friend with Adobe Illustrator make some minor adjustments to the image to fix the readability of some of the type, but more on that later.

Our second choice took a similar concept, but turned the "D" into a sheet of paper with the edge curling up.  We bought this one because we having a second icon gives us an option for DoxCycle version two.  The paper clip is also a little more identifiable than the previous icon.

Here's the corresponding splash screen:

Step 2. Approve your proof

This step in the wrap up is an opportunity to fix any remaining problems with the final design. We iterated with both the design winners over the course of a day or two. I was a little disappointed in the speed of the responses in this step. Neither of our designers responded as quickly as they had during the competition. It may have been that my expectations were too high.

Step 3. Approve your final files

This step is similar to Step 2, except with final files. It should have been straight forward, but both designers required two days. One forgot to provide the files in the Adobe Illustrator format that we had requested. A few iterations sorted that out, since the designers do not get paid until you accept the final files.

One annoyance is that once you approve your final files, there is no way to contact the designer to make even minor changes. We had some indecision about the capitalization of our product name, so I had to find someone else with Adobe Illustrator to make some these changes to the accepted final files.  Had we worked with an agency or a freelance graphic designer, we could have addressed these late changes more quickly, and probably more cost effectively.

Step 4. Rate your designer

crowdSPRING asks you to rate the designer on a scale of 0 to 100. You can also leave a comment if you wish. I rated both our designers 88, and left them both a thank-you note.

Once you complete the process, you have the option to initiate future projects directly with your designer, without the competition phase.

Final Thoughts

By the time we finished the process, we spent a little over $1200 to get two sets of icons and splash screens. From design brief to final files took just over ten days.

We'll continue to work with our agency for larger and more strategic projects. We'll probably do some crowdsourcing as well, but before we start another project, I'd like to investigate a third option: finding a local freelance graphic designer. If you know a good one, leave me a note in the comments!

"You need to start a blog. It's good for Search Engine Optimization and blogging will improve your web presence for Doxcycle and Trilogy Software."

I paused for a second, and then responded. "First of all, I don't like writing. I'm not good at it. It takes me forever to write a paragraph, and when I'm done I'm not happy with it"

"Second, what am I going to write about? Third, who is going to read it?"

Steve went back to his work, and I thought that the conversation was over. Case closed.

About a week later Steve came back to me and tried again:

"You need to start a blog.  It's your social responsibility to other software entrepreneurs. You read Seth Godin and Eric Lippert, right?  You should be paying it forward. Write about your journey. Tell a story for the next generation of people who want to start their own business."

Again I responded:

"You're just making the task even more daunting. I don't have insights on the scale of either Seth or Eric. I ask you again, who is going to read this blog?"

Steve replied, "You are." And with that, he returned to his work.

I didn't think about what he said again until later in the week, when my Mother came over to help out with my children. Mom reminded me of a story from when I was growing up:

When I was about twelve years old, we lived on a farm nine miles east of Carstairs, Alberta. That spring, Mom asked me to clean the leaves, gravel and debris that collected in the lawn on either side of our 400-metres-long driveway. She knew I was an aspiring businessman, so she offered me $20 to do it.

I needed the $20 to buy some tools that I wanted so I agreed. I grabbed the rake and headed to the end of the driveway to start working. I raked for about 45 minutes, and then looked up towards the house and saw an ominous 360 meters of leaves and gravel. It was discouraging.

I put my head down again and got back to work. A few minutes later mom brought me a drink and offered to help.

"This looks impossible", I said, ready to give up on the $20.

"You are looking at it the wrong way", Mom says, and then pauses for effect. "Look back and see all you've done."

I looked back and saw a couple garbage bags full of leaves, and about 45 meters of neat and tidy lawn on either side of the driveway.  It was much more motivating.

"It looks pretty good!" I agreed. So I set back to work, and every fifteen minutes or so looked back on the work that I'd done as an inspiration to keep going.

It worked. I kept at it and finished the job. (Thanks Mom!) 

At Trilogy Software, we're near the beginning of our project. When I look at our product road map, I see a lot of work, including some problems we don't yet know how to solve. It feels ominous, and sometimes a bit discouraging.

And, that's when I heed Mom's advice. I blog for myself. Recording the challenges and the victories of a small startup. And on those discouraging days that come along now and then, I've got something to remind me how much we've done already.

Do things right.

and

Do the right things.

Of these, it turns out the “Doing the right things” is the more difficult one.  There are many things we are capable of doing.  We need to distil all those items down and focus on the ones that will make a difference to our customers.

Our company, Trilogy Software, released our “VIP” edition of Doxcycle today (free until January for the first 500 registrants).   Doxcycle is “Intelligent Source Document Management for Canadian Tax Professionals”.  It takes T-Slips, charitable donations receipts, RRSP receipts and other tax documentation and organizes it into an indexed, compact PDF file that you can use to prepare a tax return or keep as an archive.  It's very handy should you ever require back up documentation for CRA.

We've been working on Doxcycle since October of 2010. Over the past year, we've made hundreds of decisions about how it will work, the features that we'll present and how we are going to market and sell it. We’ve spent significant effort deciding what the “right things” are.

We are a small group, so we've not held any formal design meetings or followed any product plan laid out by a knowledgeable product manager. Instead, all our team members are involved in monthly meetings with potential customers. We invite our customers to meet us in the coffee shop downstairs, and then we go up to the "big" (relatively speaking) office that three of our team members share.  In these customer meetings, we usually spend 10 or 15 minutes discussing some of the things that are difficult to manage in their business. Then, we give a five or ten minute demo of a prototype we've developed. We follow that with a 30 minute discussion of other ideas that could improve our prototype, or might improve the processes in their business. We ask questions, and avoid making statements.  We draw a few pictures on the whiteboard. When we run out of space on the whiteboard, we make notes on the windows.  In all, we'll chat for an hour or maybe an hour and a half, and then we'll thank our guest for their time.

We stay together as a team for another 30 minutes or so, and make a few more notes. We draw stars beside the things that are important. Over the next few days, the good ideas get underlined and highlighted. The ideas that can't work, don't fit our present design or aren't important get scratched out or erased. The ideas that remain after two or three days get recorded in our task tracking software and assigned to a developer.  They'll evolve into the next prototype, or, if we've got it right, will survive to form a part of a future release.

Today’s Doxcycle release is a culmination of all these discussions, as well as 163 completed tasks and 267 fixed bugs.  To request a free trial, click here.

We’re also pleased that we can now incorporate your feedback, and the feedback of your colleagues, in our future releases of Doxcycle.

There are two ways for you to be heard:

First, visit the support page of our website.  There, you can submit ideas, bugs, problems and even praise if like.  This is a public forum where others can comment on your ideas and build on them.

Second, you can use our in product feedback.  This is a great way to help us improve our slip recognition, or fix a bug that may be unique to your system.  All the comments and files you provide using this method are encrypted before they are sent to us, and we keep them in an encrypted database for utmost security.

To use the in product feedback, you can click the “@” symbol in the “quick” menu in the upper left hand corner of Doxcycle:

Or, you can provide feedback which includes a specific page of your document using the “page menu”:

In either case, you’ll be given this screen, where you can enter your confidential feedback:

We’re looking forward to your comments and promise to “Do the right things” with the feedback you offer.

For our project, we used crowdSPRING.com. There are other sites that provide this service as well, with 99designs.com being the most commonly known. 

Sign Up

At crowdSPRING, the sign up process is very simple. Creating an account requires a user name, an email address, a password and an indication of whether you are primarily a buyer or a designer. I did this in 30 seconds.

Budget 

Before creating a project, I calculated what it would cost. Listing a project on the crowdSPRING site costs $129. The advance promotion plan makes your project a "featured" project and provides an invitation to top creatives for $99. And keeping your gallery of design entries private is another $49.  My receipt also shows a project fee of $90, which is 15% of my “award” amount (more on that later). For my project I paid the listing fee of $129, the project fee of $90, the top promotion plan of $99, and the privacy charge of $49. My total was $367 before allowing for an "award", which is the fee the successful designer will get.

I spent time browsing other projects to get a sense of how other companies are using crowdSPRING. For about half of the projects, the creative brief and all the submissions can be viewed publicly. I found a similar project to ours, and used their brief for inspiration. They had a healthy number of submissions, so I copied their award amount of $600 as well.

Awards can be "guaranteed", which is basically a confirmation that at least one design will be selected, and at least one designer will get paid for their submission. I guaranteed my award, because I believe crowdSPRING’s promise that guaranteed projects will attract more designers, and because I think it's fair that designers should get paid.

In total, my project fees were $967. I may have been able to use a smaller award or cut some of the extra fees.  However, I doubt you could get anything of quality done for much less than $500 in total.  Maybe I'll try the cheaper version another time.

The Brief

Writing the brief is probably the most difficult part of the project (for me, anyways). Normally, we use a full service agency so I don't have to do this part. With the agency model, the guys from Watermark (www.wtrmrk.com) drop by our office. We chat, and Charles transcribes my rambling stream of consciousness into a comprehensive brief that effectively captures what I want.

Unfortunately, in the crowdsource model, I have to do this myself. Here's the brief that I wrote for our project.

What do you do?

We are a software company that designs tools for small to medium size accounting firms, and seasonal tax professionals

What is your industry?

Accounting and Tax Preparation

Describe the target audience for your icon design

Our target audience is small to medium accounting firms.

What 3 things would you like to communicate to your audience through your icon design?

1. We simplify the gathering, organizing and archiving of your source tax documents (W2 slips (in the US), T4 slips (in Canada), business expense receipts and so on)
2. We have a professional solution. (Our app looks like Microsoft Office 2010)
3. We’re hip and have a friendly, professional attitude.

How many different icons do you need designed?

1. An icon that will appear on a Windows Desktop (optimize for Windows 7)
2. A splash screen that will appear when DoxCycle starts up. The splash screen needs to include the word DoxCycle, and the resolution should be about 300x200. The splash screen does not need to be rectangular, and should have some transparency. The corporate green (see attached files) should be incorporated.

Do you have specific size requirements for the icons?

We need an icon that works in a few specific resolutions: 16x16, 24x24, 32x32, 64x64, 128x128 and 256x256

For the splash screen, only one resolution plus the adobe illustrator file is required.

What icon design styles do you like?

DropBox, Angry Birds

What colors do you want to see in your icon design?

See attached Trilogy_Logos.pdf for our corporate palette and font details.
(This file shows our logo)

The DoxCycle logo does not have to include the Trilogy logo colors, but we need to create a family feeling.

What colors do you NOT want to see in your icon design?

no murky or dirty colors

What adjectives should best describe your icon design?

Friendly, Professional

Do you have additional requirements or links you'd like to share?

Internally we've been experimenting with using a paper clip as the focal point of our icon. We like the simplicity of this image, but haven't been able to create clear and compelling icons at all the required sizes.

We're willing to have a simpler version of the icon that works at the 16x16 and 24x24 (for example, a stylized "D" for DoxCycle)

I spent about an hour writing the project brief. I wanted to make it as clear and concise as possible.  Someone who writes briefs regularly could probably do it in much less time than it took me.

After I finished the brief I uploaded the details of our logo, including font choice and pantone colours. And, I uploaded a PowerPoint presentation from a recent sales call to provide some context of what our product does. crowdSPRING calls all the uploaded files "Supporting Materials".

Inviting Designers

The next step is to invite designers to participate in the project. I did this by browsing through the public projects on crowdSPRING. Leave your mouse cursor on any submission you like, and you'll get a popup menu, with one of the options being "invite to project". I did this five or six times before I noticed that I could just "invite all top Creatives ". This option shows on my "Details" screen, and is available because I choose to pay the extra $99 for the "advanced promotion" package. I haven't decided if the "advanced promotion" is worth $99 yet.

Crowdsourcing vs. Outsourcing

I have to admit that when I imagined crowdsourcing, I imagined graphic designers in turtlenecks working away on their MacBook Airs in local coffee shops. While that may be the case for some designers, the majority of crowdsourcing is really micro-outsourcing. Most of the designers on my project are working in Asian time zones. And typically, they seem to submit their work before or after they go to their day job.

We wanted rapid and frequent iteration of concepts, so I needed to take time to provide feedback just before I go to bed, and again first thing after I get up.  Some submissions are done during my workday, but most seem to happen overnight or in the evening. This is not a problem for me, but it is something for which you may want to allow.

The other implication of outsourcing is that the brief needs to be clear and understandable by someone from another culture.  Our product is used to organize Canadian tax source documents, and my brief made reference to T4, T5 and W2 slips. While these documents are familiar to most Canadians/Americans, I'm pretty sure that none of crowdSPRING designers had ever seen them, so I uploaded a few samples to the "Supporting Materials" folder.

Feedback

The first few submissions were discouraging. They all seemed to be different graphic treatments of our logo. I'm not sure if this is because my brief was unclear, or because the designers didn't read it fully.  In any event, I added a few clarifying notes to the brief and the problem seemed to go away.

The crowdSPRING feedback system is fairly straight forward. Submissions are rated from 1 star (not close) through to five stars (love it!). There is also a "No Thanks" button which basically declines the submission. I used this button a couple of times when the designer clearly did not read the brief.  Typically the designer then withdraws their submission and makes it unavailable for further review.

In addition to the rating system, there is a spot to provide private feedback for the designer. I used this system to provide some encouragement, and to micro manage the design process. I made requests like change the font, merge in a concept from another submission, move this element one pixel to the left, etc. Generally I don't like to make these kinds of requests of skilled designers, but in this case it seemed necessary. 

Revising the brief

In addition to providing private feedback to the designers, you can also "revise your brief". You can't make any changes to the information you provided, but you can add to it. For instance, last night I added this not to my brief:

To help with voting, it would be great to see the splash screen on top of a cluttered Windows 7 desktop and the corresponding icon showing in the Windows 7 taskbar.

If anyone has questions about this request, let me know.

This was probably the most impressive part of the process. Within 30 minutes of updating my brief, two of my five designers provided updated submissions.

Project Stats

One of my favorite things about crowdSPRING is the project stats. crowdSPRING rates buyer participation on projects as "Low", "Average", "High" and "Amazing". This allegedly helps designers decide which projects to participate in. As a competitive guy, I took it upon myself to get the best possible rating. Fortunately, it's not that hard to do.  Just rate every submission in a timely fashion, and provide a few lines of feedback. A summary of the Project stats is prominently displayed on the gallery page, which looks like this

crowdSPRING also provides details of the average number of submissions, average number of designers and average number of comments. My project scored highly on almost every metric except for number of designers. crowdSPRING claims the number of designers participating in an average project is 12.  My project has only 5 designers participating.

The Focus Group

The final stage of the process is the "Voting" or the "focus group". crowdSPRING will guide you through selecting finalists, providing a message for voters, and final review. Once you've published your "Voting" page, crowdSPRING provides a quick way to share the link through both Facebook and Twitter.

Support 

I sent notes to crowdSPRING support on two occasions. Both times my requests were resolved promptly and professionally.

Quirks

The crowdSPRING website has a few quirks that are annoying, but surmountable. 

First, the submission images are too small. I’d like to see more detail. I assume this is to prevent me from using images without selecting and paying a designer.

Second, crowdSPRING wants me to rate my submissions as soon as I open my project. It brings the first unrated submission to the foreground and forces me to close the window to get an overview of the new submissions before I start to rate them. This could be improved.

Summary

Here's how crowdSPRING stacks up against an agency, in my view.

Of course, your mileage may vary.

Conclusion

In addition to the $967 in project fees, I've spent an hour writing the brief, and another 7 hours reviewing submissions, providing comments and supplementing the materials, for a total of eight hours.  For an equivalent project at a full service agency, I would expect to pay about three times that in fees, but I'd only spend a quarter of the time providing direction and feedback.

There is a place for both crowdsourcing and a high quality agency. When we need strategic concepts like "build an ad campaign to reach small accounting offices" or "create messages that will resonate seasonal tax preparers", we'll definitely stick with Watermark. For work that sets the direction of our brand, the full service agency wins again.  For tactical graphical elements, like icons, or creating print materials when good branding is already in place, we'll consider crowdsourcing.

Watch this blog later in the week to see the winning design.

Hiring in the early days of any new company has a much greater impact than your future hires.  The third employee, for instance, increases the staff level by 50%, and comprises 33% of the staff total until the fourth employee joins.  Employee number three will have a significant impact on productivity and the mood of the office in those early days. 

When we hire our "Customer Service Manager" at Trilogy Software, we will hire someone not only to "set the direction and policies" for Customer Service Department, but also to do Customer Service.  It will be a one-person department, at least for a few months. That means we need to find someone that can manage a department of 5 or 10 people at some time in the future, and who is also willing to answer the phone, write email responses, process refunds and adjust invoices now. 

The process of hiring is one that is difficult to codify. It requires a judgment call and "gut instincts".   As a software engineer, just about everything else I do at work has a verifiably correct answer, and I can confirm that I'm right within a few minutes. My SQL query, my XAML layout and my C# formulas can all be tested and verified as correct or incorrect within a few minutes of completing my work. Hiring, unfortunately, is not that way. Our hiring team will make a judgment call, and we may not know if we've made the right choice for a few months.

To be sure to make the best judgment call and the most accurate "gut instinct" possible we follow these guidelines:

Prospects should be interviewed by at least two peers, and no more than two managers.

Ideally, interviews will be conducted by the hiring manager and a few members from the team where the prospect will be working. The composition of the interview team should roughly reflect the composition of the future employee's working day. Typically, an employee will be spent most of their time with their teammates, and a relatively small amount of time with their manager. The composition of the interview team should reflect that.

Interviewers should prepare their interview plan in advance, and ask questions that invite a story.

I've probably done a few hundred interviews by now, and I still spend at least fifteen minutes in advance of the interviews preparing some questions. The questions don't have to be exhaustive, and I can go off script when I need to, but I've always found it useful. I ask open-ended questions, ones that will allow the prospect to tell me a story. A good way to do this is to ask questions that start with the words "tell me about", or  "tell me about a time" and then fill in the blanks:

"Tell me about a time when your work was criticised."

"Tell me about a time when you went above and beyond the call of duty to help a customer"

"Tell me about a time you missed a deadline"

Then, your applicants should describe a Situation or Task, what they did (the Action) and the Result of the action. (This is commonly called the STAR model of interviewing). 

Don't *ever* ask questions like "What would you like to change about yourself?" or "What is your biggest weakness". I asked questions like this in my early days of interviewing and never once got a useful answer. In fact, I think 9 out of 10 applicants would say something like "I'm too much of a perfectionist", or "I sometimes work too hard".  Whatever.

Basically, avoid questions that start with the word "What", because they invite your candidate to provide an unqualified list. For example the question, "What accounting software have you used?" will invite the applicant to list off all the software he or she has used. It will tell you nothing about how your applicant solves problems and get things done. By contrast, a question like "Tell me about a time when you were given a problem you didn't know how to solve in QuickBooks?" should give you some insight into the way your candidate thinks.

The "technical" portion of the interview doesn't necessarily need to follow this format, but I would caution interviewers against "trick" questions or questions about some obscure aspect of the tools you are using. In real life, your employees don't have to memorize these things because we have the Internet. In my GreenPoint days, we asked the trick question, "Why are manhole covers round?” which was allegedly common when applying at Microsoft. It always made for an interesting discussion, but I don't think we ever hired anyone based on his or her answer to this question. (The answer we were expecting was "So the cover can't fall through the hole", but there are some other good answers like "so they are easier to move.") 

Sometimes I'll change the format for a few moments when faced with an overly nervous applicant. In this situation, a few questions about hobbies or other interests can help your prospect feel more comfortable so they can give better answers to the difficult questions.

Prospects should get a chance to ask questions.

Somewhere in middle of the interview, ask something like "Do you have any questions for me?". This will give you a bit of a break, and its another great way to learn more about your applicant.  Don't leave this question until the end, when your candidate may feel rushed or tired. Ask it earlier, because it will provide insights into what motivates your applicant.

For instance, too many questions about the benefits plan or the salary may signal an applicant that's all about the money. Conversely, questions about your product plans or long term opportunities in the business show more interest in a long term relationship.

In the software business, questions like "Tell me about your agile process?" or "Tell me about your build system?" are a great signal that this applicant is interested in making a contribution, and not just collecting a paycheque. And if your candidate asks questions that start with "Tell me about", that's another sign that he or she can adopt well to new environments!