until ssh -t -o "ServerAliveInterval 1" -o "ServerAliveCountMax 5" -D 1080 -R 8000:localhost:8000 some.host screen -x;  do
    echo "Server 'ssh' crashed with exit code $?. Respawning.." >&2
    sleep 1
done

Basically, that little shell script makes sure SSH stays nailed up and connected to my VPS, port forwards port 8000 to localhost for my web development stuff I play around with, and today I added -D 1080 which creates a SOCKS5 Proxy on port 1080 that will tunnel through SSH turning

Into

I use FoxyProxy in Firefox to automatically tunnel Netflix through the SOCKS proxy.  I could set it up in Chrome which is my primary browser, but this way I don’t have to turn the Proxy on and off between home and work.  I rarely have time to watch anything at work, but at least now I can.

The world of IT and custom software is complicated, and it’s very different from the world you read about in the blogs.  It’s not nearly as sexy as a startup, a social network, or a product company that’s releasing other product “killer”s, but billions are spent a year writing custom software for use in enterprises.  Unfortunately, unlike a product company that sells to its end users, custom software is often designed by a committee of people who never use or have any plans of using the actual software.

We just completed one of our quarterly major software releases this week.  From many people’s perspective it’s been an unqualified disaster.  I sit somewhere in the middle, as my expectations for acceptable quality are somewhat lower than a lot of the business side management.  I can’t blame them for their high standards, but I’m a bit more of a realist.  However, there were undoubtably major problems released into production which should have never passed testing, and we caused very real impacts to our sales and care reps as well as to our actual customers, largely because we have cut ourselves off from listening to our real users and instead spend our time listening to corporate “Program Management” type functionaries who represent our actual users’ interests.

Anything designed by committee is likely to suffer in quality, and custom software only amplifies the affect of design by committee.  Product Manager, Business Analyst, Enterprise Program Manager, these are all titles of people hired to, among other things, write requirements for IT to deliver software against.  Their job is to design enterprise “products” that companies sell, however, in many companies, these people never actually use the output of their work.  Sales and care representatives are then forced to follow processes concocted by their corporate counterparts which make customers jump through hoops, follow inane business rules, and in some cases turn paying customers away.

Also, when this software goes to production with issues, as all software does to some extent, there becomes a prioritization exercise that’s required in order to get the biggest issues triaged and fixed as fast as possible.  It’s these same corporate managers who are prioritizing what needs to be fixed first, even though they’re likely not seeing any of the issues for themselves.  For example, from this last release there was a defect where the customer’s account number was not printing on a receipt when taking a service payment.  This sounds critical until you realize that we don’t take service payments in the locations this defect was said to be affecting.  The people prioritizing the defects didn’t even understand the business well enough to know the basics of how payments are taken in various channels.  On the flip side, coming out of user acceptance testing, an issue was detected where migrating a customer from one pricing plan to another caused their error to order out which requires a 24 hour manual process to correct the customer’s account, but this wasn’t considered critical so it went to production affecting hundreds of users a day.

These types of corporate structures feed upon themselves.  Productive departments servicing customers end up hiring their own analysts and “managers”, often with no actual management responsibilities, to combat the work being thrown at them from their corporate counterparts.  Corporate product managers and business analysts design business rules which do nothing to help the actual customers, but they do help to drive revenue to their particular corporate general ledger line which keeps their management hiring more.  This does nothing to grow your business.  Yet, it’s common in many large organizations.  Instead, eliminate these corporate departments.  They’re completely unnecessary, and you’ll find yourself not only saving money by avoiding having to pay their salaries, you’ll also be more productive. At the very least listen to your real users.  If you need business analysts, bring in field sales representatives, bring in help desk and customer care representatives in 6 month rotating cycles.  These people actually know what they need, because they just lived the life of your end user.  You shouldn’t be writing software, custom or not, that’s being designed by people who have had no interaction with the users of the products.  This is worst in custom software, but the same wisdom should hold for anyone writing software.  No one should ever be more than 6 months from having worked directly with end users.

There are some complications to this that bear explanation so you can understand why this is more difficult than it would otherwise seem.  First, 75% or more of the user base sits outside the corporate firewall and does not have access inside the network.  This means that without a significant engineering effort to authenticate them, we cannot easily provide content and messaging to them from a simple to use platform like WordPress.com or any other public SaaS.  We decided that the best way to reach them would be from within the software they use every day, a web based CRM system, but since we didn’t want to make it into a blogging platform too, we decided it would be easiest to post the messaging in a blog on WordPress.com.  It was publicly accessible and everyone would be able to read the same content.

However, as it turns out, it’s better to ask forgiveness than permission.  We decided to check with the public relations and legal teams to ensure they’d be ok with this.  As it turns out, not surprisingly, they want to control the messaging around our brand.  Now, to communicate with our users, unless we can put it behind an authentication system to ensure only reps can see it and not the whole Internet, we need to get Legal and PR to approve every posting.  The best case scenario from writing a blog post to send to our users and approve the posting is 2 days.  Obviously, this process is burdensome and defeats the purpose of blogging, which is to get out quick, concise, and often time sensitive messaging.

I first started blogging 7 years ago.  I started video blogging 6 years ago.  I was on Facebook 4 years ago.  Corporate America has had 7 plus years of customers, suppliers, employees, and everyone else having self-publishing capabilities where they will write about your brand.  These people writing on platforms not controlled by the company need no approval! 

One thing is certain about your employees when they want to communicate.  They will write good things about you.  With a minimal amount of training from Legal and PR they can be instructed not to give any forward looking statements, reveal sensitive information, or communicate anything that might be damaging to your brand.  You must trust your employees.  Creating undue process and centralized communication systems will ensure your employees are given a strong disincentive to try to communicate good things on your behalf!  Being part of the conversation on Facebook, Twitter, Google Plus, WordPress.com, Tumblr, etc, requires brand ambassadors who are enthusiastic about your brand and are willing to be your advocates.  The conversation certainly is going to happen whether your company approves of it or not, and if your company does not learn to trust its brand ambassadors, I am quite certain they will regret it.

1. Wechooselunch.com
   • Go to lunch every day?  Got a group of people you like to go with, but picking the place is always a huge hassle?  Easy, we pick a random location within a small radius of where you work every day.
   • Lots of other ideas to go with it, add a scheduling feature to upload free/busy data so you can schedule around meetings, randomly rotate your lunch crew, do matching with interested parties, etc.
2. Point of Sale – Webified
   • Shocking, I used to work for a POS software vendor and the idea of reinventing it from scratch with modern technology is appealing
   • This is a crowded space, with 25 years of entrenched players who have kept it pretty well out of the web era, somehow.  A SaaS model is ripe to be successful.
   • Unfortunately, horizontal software targeting brick and mortar is difficult to market, and getting people through the virtual “front-door” will be difficult, especially as conservative as retailers generally are (“What happens if the Internet goes down?”).  Unlike most web startups, distribution will be key.  A strong VAR channel will be a must here.
3. Enterprise Search, Email Seeded
   • Enterprise search isn’t something you read about much anymore.  It’s not because Google has owned the market and it’s a no-brainer, it’s because it doesn’t really work and nobody wants to really be in the space anymore.  Relevancy is a huge issue internal to Enterprises because unlike the Internet where relevancy is based at a fundamental level based on how many references (links) there are to a given set of content, Enterprises don’t generally generate links, so tagging, keywords, other meta data becomes required to determine relevancy.  Given my earlier comments, obviously, I think there’s huge opportunity in the space.
   • The best place to farm link data in an Enterprise?  Exchange or another email repository.  I get dozens of links a day, they’re just not publically indexable.  Even better, by examining organizational structure, a link from a CEO is likely to be more relevant than a link from an individual contributor, unless that individual contributor is close to you in the organization (say not more than two degrees removed).
4. IRC for the Web
   • Some would say Twitter has it locked up here, but I think there’s still a market for topical chat on the Web that’s real time, isn’t limited to 140 characters per message, is easy to get into, and is totally anonymous if you want it to be.  ChatRoulette shows it’s possible for it to be successful, but I think there’s a market for something that isn’t largely filled with video of people showing their penii [sic].
   • I haven’t fully baked the idea, but I think mixing membership with some type of game mechanic to allow for leveling up is a great mix, I just don’t know how to do it because I haven’t thought about it enough.
5. Game Sharing/Scheduling Network
   • This one I just had the other day.  I’m kind of exited about it, it definitely warrants more thought.  You just bought Game X, and so did all your friends.  When are they online?  Your wife wants the TV until 9, will they still be playing?  Maybe they would be if you were online.  The ability to get a txt or a notification on an iPhone app that your friends were online playing, or maybe even text into them from your phone would be huge.  Twitter for gamers.
   • What about when you’re at the office and you’re talking about what to play that night.  Will they remember?  Maybe you can play from 8-10 and he can play from 9-11, would you stick around till 9 if you were playing by yourself?  Would you maybe not start till 9, win some wife bonus points etc, if you knew he was going to start a 9?  Game time matching for friends could be a big win.
   • Needs to be console independent.  Xfire and others probably play here for the PC, don’t know any that integrates with mobile though to notify you on the device in your pocket instead of your PC which you probably don’t have open while gaming.
6. Enterprise Distributed Filesystem
   • I put this one here to remind myself I didn’t do nothing in 10 years of thinking about it.  I had very basic deduplication engine completed in Java, when it became obvious that even storing the metadata about unique blocks across the entire enterprise was going at least 10-20% of the total space allocated across all the distributed nodes.  Network latency was going to be a killer.  It’s a great idea, but the reason it’s never made it out of academia is the real world problems kill the idea.  Even for doing backups only (i.e., could be a write-once filesystem), it was still infeasible for a lot of reasons.  Perhaps it’s worth a revisit with things like Redis and Cassandra out there now which weren’t there a year and a half ago when I looked into it heavily.

If virtual ink were a tradable commodity, I’m sure there’d be a playboy trading magnate who made his entire family fortune from selling virtual ink to people who like to write about what makes one company’s culture so successful versus another.  So I suppose I’m walking a well-trodden path, but after spending much time analyzing why culture can go from good to bad, there are a couple of things to do (or not do):

Hire Slow, but Make Hiring Easy

Choose your employees very carefully.  Only hire the best.  One bad apple will certainly spoil the bunch, but make it easy to bring people on.  If it takes more than 2 weeks from when the candidate is identified, after the interviews have completed, from the day they could start, your HR processes are broken and you need to fix HR.

Fire quickly

As I said earlier, a bad apple spoils the bunch.  If you have a low performer, move them out quickly.  Give them a month or two of severance, apologize for it not working out, and wish them the best with a good reference.  Just because they’re not working out at your company does not mean they will add no value anywhere.

Accountability, Accountability, Accountability

Failure happens, even at the best companies.  Adjust, move on, but hold people accountable.  This doesn’t mean your first or second failure means you should exit, it simply means if you’re not being successful then it needs to be analyzed and corrected.  Maybe you’re in the wrong seat on the bus.  Nothing frustrates your workforce more than seeing someone still in the same position, still failing, a year after it was obvious to everyone they weren’t succeeding.  Your personal relationships can be difficult to untangle from your business relationships, but the phrase “it’s not personal, it’s just business” exists for a reason.  Everyone should be measured, graded, and held responsible for the areas for which they are accountable.

“What Gets Measured Gets Managed”

Everyone’s heard this quote.  S.M.A.R.T. goals, etc.  You would think it’s a bunch of management school bullshit that they teach to people to allow them to become HR Generalists.  You would think that, until you’ve been somewhere it isn’t practiced.  The fish out of water analogy is a very graphic way of describing an organization where the rank and file are flapping around without direction.  Goals should be clear, everyone should be marching in lock step towards achieving them.  Giving your teams freedom to work on their own is awesome, in moderation.  An organization where everyone is marching to their own drum breeds frustration and malicious compliance.

Sticktuitiveness

Ok, so it’s not a word.  It conveys the point though.  Being wishy-washy, constant direction changing, and a lack of consistency will drive morale through the floor.  Nothing will demoralize a group of people who have been working hard towards a goal than to see that goal shift, morph, move, etc, on a regular basis.  This doesn’t mean you shouldn’t guide the ship, correct course, and be proactive about hitting a moving target.  However, large scale about-faces will certainly destroy productivity.

Conclusion

Most of the points were written on what not to do, but these points are very binary.  Each has two sides of the coin, the right way to function and the wrong way.  Avoid the wrong way, the right way finds itself.  Create a performance culture, foster a meritocracy, and smart and motivated people will find their way to you.  An organization full of the best people has no option but to succeed.

Amy Hoy (@amyhoy) is an old friend of mine.  We have spent untold amounts of hours on IRC together.  We’ve just recently reconnected on Twitter, where yesterday she tweeted a link to “Don’t bite the shit sandwich” on her blog.   I’m an avid follower of the startup scene, especially a number of the bullshit peddlers she warns against in her post.  I’ve been actively involved in that scene as well, and while I don’t have a huge success story, I think the mistakes I’ve made lead me to some slightly different conclusions.

First of all, I’d get a lot more viewers here if I came out and said “Eric Ries is wrong, lean startups are bad,” or even “Amy Hoy doesn’t know what the hell she’s talking about,” but I won’t.  The truth is in the middle.  I agree with several of Amy’s conclusions.  Startup methodology, like development methodology, exhibits a lot of characteristics of a religion.  I believe a lot of great startup companies fail because of the idea that they need to give a product away for free to attract eyeballs, gain critical mass, create shareholder value and then exit.  I was part of one of those.

I’ve started an Internet company and I’ve started more traditional small businesses.  What I’ll say about the small business world is that it’s steeped in reality.  No income coming through the door?  Speculative business idea?  Don’t look for help from your local bank.  More money flowing out than flowing in?  Hope you have a rich uncle to fund your “speculation.”  Most of the country works this way, real businesses, exchanging dollars for good and services from each other.

If you want freedom from Corporate America (which I’m seriously considering at this point), you can follow Amy’s model (and Charlie’s too).  I’d like to see a lot more like those two as well.  However, it’s not really new.  Plenty of people have been out selling themselves for years.  Artists, writers, consultants, you name it, there’s plenty of small businesses built around a person.

The problem with her concept of not taking funding or building to sell is if you have an idea, a great idea, a world changing idea, or maybe even just an idea that’ll fill a great niche, you’re going to get pushed into a service based model.  Maybe you have to do consulting on the side to start your business, which is a major distraction.  If you don’t, your business is going to be pushed into compromise to make revenue early and be forced early on to attempt to extract value from your idea when it may not be the time to charge for it yet.  You’re going to be forced into overhead for customer service, general administration, and key early stage development cycles are going to be consumed attempting to respond to customer demands when your vision for where the product should be going should be the right one.

There is no right answer.  If you’re looking to create a product business and not a service business though, don’t discount financing your business with an early stage capital investment.  Google doesn’t get built if Sergey and Larry turn their search engine into a product for a specific customer and lose their long term vision of indexing the world’s information.  Facebook, recent controversy aside, does not get to be a household product if Zuckerberg spends his days trying to find a way to bring in revenue three months after Facebook’s founding.  Nothing wrong with creating a brand around yourself and spending your days free from bullshit either, though.

Preparation

When we created the offshore IT Operations Center, I spent a very significant amount of money with the offshoring company’s consultants to document every application, database, and server they would be supporting.  I had them develop all of our procedures for how to escalate problems, and they put together all of our SLA documentation.  In short, they wrote the manual on how to do business for our IT Operations Center.  Then, and only then, did we consider hiring them to do actual operations work.

Augment, Not Replace

When we brought on the offshore team for the ITOC, it was an augment of existing personnel, which gave us two advantages.  One, we didn’t have a fixed limit on transition period which occurs in a lot of offshoring initiatives.  Second, we were not letting the onshore team go, so we did not end up with personnel training their replacements or the onshore teams attempting to poison the project.  In fact, the offshore team has been a huge success with the onshore team they were augmenting because rather than having a 5 person team covering 24 hours a day, 7 days a week via pagers, cell phones, and a brutal on call schedule, the onshore team now has the ability to leave their house over the weekend rather than be tied to a laptop and Internet connection with confidence we have coverage for the business.

Process, Process, and did I mention Process?

If you cannot onboard a new resource in 30 days, no matter how complicated the position, you’re not ready to offshore.  You cannot take a position that requires intense collaboration with onshore resources, high-end skill set, and a strong requirement on independent thinking and problem solving skills and send it offshore without also outsourcing the entire function.  Creating a new team in a far-flung location requires them to be able to execute their job without ambiguity, without constant reliance on onshore resources, and with a clear direction as to their responsibilities and deliverables.  This means that the processes they are facilitating need to be documented, defined, and well understood.

Offshoring is not Outsourcing

These two terms are used interchangeably, but they are anything but interchangeable.  Offshoring means augmenting your resources with lower cost alternatives in another location (usually another country but I hear people are doing the same thing with lower cost US locations).  Outsourcing means you are moving a function or a department to be handled by another company.  Offshoring is often outsourcing but not always.  If you are expecting a high-end skill set, you will only be successful with outsourcing the entire function.  The team will need the flexibility to define the process, determine service levels, and generally be in control of the deliverables if you want them to do something other than providing a very specific well defined function.

Summary

Offshoring can be awesome for everyone involved, or it can end up in failure.  How it’s approached has everything to do with how it’s received by the co-workers on-shore.  Deciding whether to augment or outsource is critical, and if it’s an augment ensuring there are well defined processes that govern the output of the augmented resources is critical.  Unclear expectations, lack of preparation, and malicious compliance by existing personnel will doom any offshoring effort to failure.

Being customer centric, as implemented most places I’ve seen, means an unwavering attitude internal to the organization that it will bend over backwards to provide the customer what they want.  This leads to some unintended consequences, such as a continual focus on the functional, rather than the non-functional.  For those who spend little time doing software project, program or requirements management, functional requirements are things like “I want the software to be able to calculate taxes and print it on a receipt” and non-functional requirements are things like “I want the software to always work.”  Bending towards customer centricity and an attitude that the customer is always right, especially in the world of custom software, means the resources of the organization are constantly allocated to the functional while often largely ignoring the non-functional until it’s too late.

This attitude can and will, at a point in the future, leave you in a situation where the scale and complexity of the software the IT organization is producing will begin to collapse under it’s own weight.  The nimbleness once felt, perhaps as recently as a few years back, will begin to become cumbersome, as the continuous iterations of “get it done, the customer needs it right away” lead to band-aids, patchwork engineering, and layer upon layer of complexity.  Time for things like application rationalization activities, engineering and architecture improvements, and infrastructure improvements are pushed aside in order to make room in the capacity model for more functional requirements for the customers.

The solution? Simple IT.  A culture of simplicity is required around every change to drive compromise between what the customer wants and needs and what IT can deliver and maintain.  Constant reinforcement of a simplicity culture is required to drive every individual contributor to ask “is this additional complexity required?”  It will drive from monolithic to modular, from n to n-1 layers of abstraction, from customization to off the shelf.  No organization is perfect, so additional iterations of software should continually simplify and rationalize the changes that in hindsight complicated rather than simplified.  The customer demands the non-functional without asking for it, and the customer gets what they’re asking for through simplicity.

What does the mean for the customer?  The customer asks for 30 reports, they get 5, because it’s actually in their benefit.  The customer wants business rules that force the agents to jump through hoops to try to upsell customers, which leads to ridiculous clicking and wasted agent time, IT pushes towards training, because the complexity hurts the users’ perception of the software and impacts their behavior negatively.  The customer asks for a different pricing model for every market, IT refuses to support it.  Will this lead to significant conflict and accusations that IT is not responsive to the business?  Absolutely!  Perhaps the company will go through three IT leadership teams before they realize their behavior is the problem, or perhaps they will never realize.  Either way, IT has a duty to the business to draw a line between what is possible and what is right, and IT is in the unique position to provide the business with facts and data about how decisions driving complexity hurt not only IT, but hurt the business.  Simplicity is, in reality, complicated to deliver; it’s the enemy of mediocrity; it’s the champion of forethought and design; and a culture centered around simplicity is the only way to deliver years of iterative software without being forced to spend significant amounts of money, time, and resources, redoing what customer centricity, and by extension, complexity, create.