Clint Sharp

Startup ideas are a dime a dozen.  I guess I’ve generated about a nickel in value in over 10 years of thinking about it.  I’m still thinking about implementing one of these, but I’ve been having a hard time getting really behind one of them enough to start detailed work on one.  The reason I publish these publicly, is because I believe the idea is worth nothing without the execution. If you can take the idea and do well with it, then more power to you.  I obviously can’t implement all 6 of these, but if you’re reading this and you’re interested in working on them, contact me.

1. Wechooselunch.com
   • Go to lunch every day?  Got a group of people you like to go with, but picking the place is always a huge hassle?  Easy, we pick a random location within a small radius of where you work every day.
   • Lots of other ideas to go with it, add a scheduling feature to upload free/busy data so you can schedule around meetings, randomly rotate your lunch crew, do matching with interested parties, etc.
2. Point of Sale – Webified
   • Shocking, I used to work for a POS software vendor and the idea of reinventing it from scratch with modern technology is appealing
   • This is a crowded space, with 25 years of entrenched players who have kept it pretty well out of the web era, somehow.  A SaaS model is ripe to be successful.
   • Unfortunately, horizontal software targeting brick and mortar is difficult to market, and getting people through the virtual “front-door” will be difficult, especially as conservative as retailers generally are (“What happens if the Internet goes down?”).  Unlike most web startups, distribution will be key.  A strong VAR channel will be a must here.
3. Enterprise Search, Email Seeded
   • Enterprise search isn’t something you read about much anymore.  It’s not because Google has owned the market and it’s a no-brainer, it’s because it doesn’t really work and nobody wants to really be in the space anymore.  Relevancy is a huge issue internal to Enterprises because unlike the Internet where relevancy is based at a fundamental level based on how many references (links) there are to a given set of content, Enterprises don’t generally generate links, so tagging, keywords, other meta data becomes required to determine relevancy.  Given my earlier comments, obviously, I think there’s huge opportunity in the space.
   • The best place to farm link data in an Enterprise?  Exchange or another email repository.  I get dozens of links a day, they’re just not publically indexable.  Even better, by examining organizational structure, a link from a CEO is likely to be more relevant than a link from an individual contributor, unless that individual contributor is close to you in the organization (say not more than two degrees removed).
4. IRC for the Web
   • Some would say Twitter has it locked up here, but I think there’s still a market for topical chat on the Web that’s real time, isn’t limited to 140 characters per message, is easy to get into, and is totally anonymous if you want it to be.  ChatRoulette shows it’s possible for it to be successful, but I think there’s a market for something that isn’t largely filled with video of people showing their penii [sic].
   • I haven’t fully baked the idea, but I think mixing membership with some type of game mechanic to allow for leveling up is a great mix, I just don’t know how to do it because I haven’t thought about it enough.
5. Game Sharing/Scheduling Network
   • This one I just had the other day.  I’m kind of exited about it, it definitely warrants more thought.  You just bought Game X, and so did all your friends.  When are they online?  Your wife wants the TV until 9, will they still be playing?  Maybe they would be if you were online.  The ability to get a txt or a notification on an iPhone app that your friends were online playing, or maybe even text into them from your phone would be huge.  Twitter for gamers.
   • What about when you’re at the office and you’re talking about what to play that night.  Will they remember?  Maybe you can play from 8-10 and he can play from 9-11, would you stick around till 9 if you were playing by yourself?  Would you maybe not start till 9, win some wife bonus points etc, if you knew he was going to start a 9?  Game time matching for friends could be a big win.
   • Needs to be console independent.  Xfire and others probably play here for the PC, don’t know any that integrates with mobile though to notify you on the device in your pocket instead of your PC which you probably don’t have open while gaming.
6. Enterprise Distributed Filesystem
   • I put this one here to remind myself I didn’t do nothing in 10 years of thinking about it.  I had very basic deduplication engine completed in Java, when it became obvious that even storing the metadata about unique blocks across the entire enterprise was going at least 10-20% of the total space allocated across all the distributed nodes.  Network latency was going to be a killer.  It’s a great idea, but the reason it’s never made it out of academia is the real world problems kill the idea.  Even for doing backups only (i.e., could be a write-once filesystem), it was still infeasible for a lot of reasons.  Perhaps it’s worth a revisit with things like Redis and Cassandra out there now which weren’t there a year and a half ago when I looked into it heavily.

Three years with barely an update, now I’m at least cranking out content on a once a week basis.  I only know of a couple of people who still read my rantings, but it feels good to express opinions after 4 years of attempting to claw my way to middle management by biting my tongue and playing nicely.  What a perfect segue into my thoughts on culture.

If virtual ink were a tradable commodity, I’m sure there’d be a playboy trading magnate who made his entire family fortune from selling virtual ink to people who like to write about what makes one company’s culture so successful versus another.  So I suppose I’m walking a well-trodden path, but after spending much time analyzing why culture can go from good to bad, there are a couple of things to do (or not do):

Hire Slow, but Make Hiring Easy

Choose your employees very carefully.  Only hire the best.  One bad apple will certainly spoil the bunch, but make it easy to bring people on.  If it takes more than 2 weeks from when the candidate is identified, after the interviews have completed, from the day they could start, your HR processes are broken and you need to fix HR.

Fire quickly

As I said earlier, a bad apple spoils the bunch.  If you have a low performer, move them out quickly.  Give them a month or two of severance, apologize for it not working out, and wish them the best with a good reference.  Just because they’re not working out at your company does not mean they will add no value anywhere.

Accountability, Accountability, Accountability

Failure happens, even at the best companies.  Adjust, move on, but hold people accountable.  This doesn’t mean your first or second failure means you should exit, it simply means if you’re not being successful then it needs to be analyzed and corrected.  Maybe you’re in the wrong seat on the bus.  Nothing frustrates your workforce more than seeing someone still in the same position, still failing, a year after it was obvious to everyone they weren’t succeeding.  Your personal relationships can be difficult to untangle from your business relationships, but the phrase “it’s not personal, it’s just business” exists for a reason.  Everyone should be measured, graded, and held responsible for the areas for which they are accountable.

“What Gets Measured Gets Managed”

Everyone’s heard this quote.  S.M.A.R.T. goals, etc.  You would think it’s a bunch of management school bullshit that they teach to people to allow them to become HR Generalists.  You would think that, until you’ve been somewhere it isn’t practiced.  The fish out of water analogy is a very graphic way of describing an organization where the rank and file are flapping around without direction.  Goals should be clear, everyone should be marching in lock step towards achieving them.  Giving your teams freedom to work on their own is awesome, in moderation.  An organization where everyone is marching to their own drum breeds frustration and malicious compliance.

Sticktuitiveness

Ok, so it’s not a word.  It conveys the point though.  Being wishy-washy, constant direction changing, and a lack of consistency will drive morale through the floor.  Nothing will demoralize a group of people who have been working hard towards a goal than to see that goal shift, morph, move, etc, on a regular basis.  This doesn’t mean you shouldn’t guide the ship, correct course, and be proactive about hitting a moving target.  However, large scale about-faces will certainly destroy productivity.

Conclusion

Most of the points were written on what not to do, but these points are very binary.  Each has two sides of the coin, the right way to function and the wrong way.  Avoid the wrong way, the right way finds itself.  Create a performance culture, foster a meritocracy, and smart and motivated people will find their way to you.  An organization full of the best people has no option but to succeed.

Jason,

I’m a member of the Jason Nation, and I read your email today that you’re looking for developers for Mahalo.  I don’t know if I’m your new Mahalo developer, but you want to hire me.

It’s cocky, but I want to grab your attention.  I’m also taking a big risk writing a public cover letter, which will easily be found by any person looking for me, since I’m pretty easy to find on Google.  Any BDC HR rep reading in this in the future will certainly file my resume in the circle file.

You probably don’t remember, but we met in 2005 at Gnomedex.  At the time, I was working for a BDC (I’ve since spent the last 4 years at one as well), however I was an active member of the burgeoning videoblogging community.  I was a moderator on the Yahoo Videoblogging group, where the formations of today’s video podcasting started.  There were some awesome people on that group; Jay Dedman and Josh Kinberg, Steve Garfield, Robert Scoble, Mike Hudack from blip.tv, are all names you might recognize among the dozens of participants.  We started a trend that helped lead to the early success of YouTube among other things.

When my attempt to get a job working with Robert and Jeff Sandquist on Channel 9 at Microsoft fell through (lost out to pt’s wife who was an insider), I worked with Josh and Jay from the Videoblogging group to start a company called Mycelia Networks, which created FireAnt (if you remember it, early coverage on TechCrunch here).  I was the web developer and built a directory for videoblogs and video podcasts.  We were not a success.  It eventually sold to Sonic Mountain, who also acquired Odeo (my postmortem here).  Feeling the need to make some kind of a living to support my newborn, I went back to corporate life.

I’m an entrepreneur stuck in a corporate middle manager’s body.  I currently run IT operations for a multi-billion dollar company.  I direct teams that run all of the critical applications and a 24/7 off-shore operations center.  I manage millions of dollars and 3 dozen or more people.  I am well respected, influential, well-paid, but I live constantly with various shades of mediocrity and I’m absolutely sick of sucking.  I work with some excellent people and have an excellent team, but I yearn to do something awesome.  I want to go back to where there’s no place to hide; you’re either a success or a failure.  I want to work somewhere results matter.

I can be your new hacker.  I’ve just recently built an internal tool using Django, learning Python from scratch to build it, to try to keep my skills current.  I can run a development organization, or I can run operations.  I’m confident I can be successful in any role you see fit for me.  My resume and contact information are here on my website.  Thanks for your time.

Today we fired one of our offshore teams.  This post isn’t to criticize that initiative or the manager, they definitely weren’t working out, but it does bring to mind several lessons learned, at least from my perspective.  This wasn’t my offshore team, but I do have an offshoring effort that’s been very effective thus far.  Some advice for when you may be considering your next offshoring initiative:

Preparation

When we created the offshore IT Operations Center, I spent a very significant amount of money with the offshoring company’s consultants to document every application, database, and server they would be supporting.  I had them develop all of our procedures for how to escalate problems, and they put together all of our SLA documentation.  In short, they wrote the manual on how to do business for our IT Operations Center.  Then, and only then, did we consider hiring them to do actual operations work.

Augment, Not Replace

When we brought on the offshore team for the ITOC, it was an augment of existing personnel, which gave us two advantages.  One, we didn’t have a fixed limit on transition period which occurs in a lot of offshoring initiatives.  Second, we were not letting the onshore team go, so we did not end up with personnel training their replacements or the onshore teams attempting to poison the project.  In fact, the offshore team has been a huge success with the onshore team they were augmenting because rather than having a 5 person team covering 24 hours a day, 7 days a week via pagers, cell phones, and a brutal on call schedule, the onshore team now has the ability to leave their house over the weekend rather than be tied to a laptop and Internet connection with confidence we have coverage for the business.

Process, Process, and did I mention Process?

If you cannot onboard a new resource in 30 days, no matter how complicated the position, you’re not ready to offshore.  You cannot take a position that requires intense collaboration with onshore resources, high-end skill set, and a strong requirement on independent thinking and problem solving skills and send it offshore without also outsourcing the entire function.  Creating a new team in a far-flung location requires them to be able to execute their job without ambiguity, without constant reliance on onshore resources, and with a clear direction as to their responsibilities and deliverables.  This means that the processes they are facilitating need to be documented, defined, and well understood.

Offshoring is not Outsourcing

These two terms are used interchangeably, but they are anything but interchangeable.  Offshoring means augmenting your resources with lower cost alternatives in another location (usually another country but I hear people are doing the same thing with lower cost US locations).  Outsourcing means you are moving a function or a department to be handled by another company.  Offshoring is often outsourcing but not always.  If you are expecting a high-end skill set, you will only be successful with outsourcing the entire function.  The team will need the flexibility to define the process, determine service levels, and generally be in control of the deliverables if you want them to do something other than providing a very specific well defined function.

Summary

Offshoring can be awesome for everyone involved, or it can end up in failure.  How it’s approached has everything to do with how it’s received by the co-workers on-shore.  Deciding whether to augment or outsource is critical, and if it’s an augment ensuring there are well defined processes that govern the output of the augmented resources is critical.  Unclear expectations, lack of preparation, and malicious compliance by existing personnel will doom any offshoring effort to failure.

I was listening to a video from Startup Lessons Learned, where a brief comment was made regarding customer centricity at IMVU, which reminded me of a nagging thought in my head in recent months.  Customer Centricity is evil, in IT.  That’s a bold statement, so let me explain.  There are a lot of mechanisms of providing the customer what they need, and perhaps even want, without placing them at the center of your activities.

Being customer centric, as implemented most places I’ve seen, means an unwavering attitude internal to the organization that it will bend over backwards to provide the customer what they want.  This leads to some unintended consequences, such as a continual focus on the functional, rather than the non-functional.  For those who spend little time doing software project, program or requirements management, functional requirements are things like “I want the software to be able to calculate taxes and print it on a receipt” and non-functional requirements are things like “I want the software to always work.”  Bending towards customer centricity and an attitude that the customer is always right, especially in the world of custom software, means the resources of the organization are constantly allocated to the functional while often largely ignoring the non-functional until it’s too late.

This attitude can and will, at a point in the future, leave you in a situation where the scale and complexity of the software the IT organization is producing will begin to collapse under it’s own weight.  The nimbleness once felt, perhaps as recently as a few years back, will begin to become cumbersome, as the continuous iterations of “get it done, the customer needs it right away” lead to band-aids, patchwork engineering, and layer upon layer of complexity.  Time for things like application rationalization activities, engineering and architecture improvements, and infrastructure improvements are pushed aside in order to make room in the capacity model for more functional requirements for the customers.

The solution? Simple IT.  A culture of simplicity is required around every change to drive compromise between what the customer wants and needs and what IT can deliver and maintain.  Constant reinforcement of a simplicity culture is required to drive every individual contributor to ask “is this additional complexity required?”  It will drive from monolithic to modular, from n to n-1 layers of abstraction, from customization to off the shelf.  No organization is perfect, so additional iterations of software should continually simplify and rationalize the changes that in hindsight complicated rather than simplified.  The customer demands the non-functional without asking for it, and the customer gets what they’re asking for through simplicity.

What does the mean for the customer?  The customer asks for 30 reports, they get 5, because it’s actually in their benefit.  The customer wants business rules that force the agents to jump through hoops to try to upsell customers, which leads to ridiculous clicking and wasted agent time, IT pushes towards training, because the complexity hurts the users’ perception of the software and impacts their behavior negatively.  The customer asks for a different pricing model for every market, IT refuses to support it.  Will this lead to significant conflict and accusations that IT is not responsive to the business?  Absolutely!  Perhaps the company will go through three IT leadership teams before they realize their behavior is the problem, or perhaps they will never realize.  Either way, IT has a duty to the business to draw a line between what is possible and what is right, and IT is in the unique position to provide the business with facts and data about how decisions driving complexity hurt not only IT, but hurt the business.  Simplicity is, in reality, complicated to deliver; it’s the enemy of mediocrity; it’s the champion of forethought and design; and a culture centered around simplicity is the only way to deliver years of iterative software without being forced to spend significant amounts of money, time, and resources, redoing what customer centricity, and by extension, complexity, create.

A couple of things have happened recently which has made it to where I believe it’s time to start writing thoughts, publicly, again.  First, I’m relatively certain people do not read this, at least regularly.  Secondly, even though I have advanced significantly in my career and I am now a very competent middle manager, I believe my entire happiness in my work life is suffering simply because we seem to defer to political correctness.  Making sure no one’s feelings are hurt rather than running an organization as a meritocracy where the best ideas win seems to be lay of the land, at least in American business these days.  I used to not be afraid of putting my thoughts out there, now I seem to be in constant fear that being liked is the most important aspect to career advancement.  I have decided that while I have come very very far in the last few years attempting to ensure that I am effective at change in a large organization due to being well liked and respected, I will no longer place being liked ahead of stating what I think will be in the best interest of our organization.  That does not mean I intended to go back to being a daft prick like I have been in former years, but I will be firm on my thoughts on the direction our organization should be taking.  I want these ideas to stand on their own, so this will be immediately followed by another post on being too customer centric.

I have just had the greatest random text conversation ever!  For those who have never been lucky enough, a random text conversation is the equivalent of a wrong number in texting land.  You’re sitting there, suddenly a text shows up from a random stranger.  Here’s the conversation:

“hey! its bri” – stranger
“Hey” – me
“are you starting rumors that im cheating on chris cuz im pissed right now” – stranger
“I am not but you are right to be pissed” – me
“who told you that tho” – stranger
“I can’t tell u.  I pinkie sweared.” – me
“I dont give a fuck” – stranger
“I promised them I wouldn’t tell.  What kind of person do u think I am?  I never thought u guyz were right for each other anyways.” – me
“its only cuz you like him but we are perfect together and i care about him a lot” – stranger
“When did u guy do it last?  U should get checked or did he get it from u?” – me
“what???” – stranger
“Talk to him” – me
“okay and why cant you tell me???!!” – stranger
“Just talk to him then let me know what he says.” – me
“about what??m” – stranger
“Whether he’s been checked out.” – me
“for what?”
“Don’t be a dumbass   What do people in relationships normally ask their partners if they’ve been checkd for?” – me
“i fucking hate you” – stranger
“one more thing HE’S MINE BACK OFF!!!!!” – stranger

Holy shit.  That’s crazy.  Wonder if anyone still reads this thing.  It appears to have been hax0r’d as there’s some links on my blogroll I didn’t add.  This WordPress version is crazy old now too.

So, things are going great at Cricket.  Since my last posting I’ve signifcantly advanced in my management track and have something like 20 people working for me (although that’s about to go down a bit).  I have lots of interesting commentary on IT Management, and I suppose I’ll probably refactor this blog to voice my opinions on IT Operations.  Dunno if anyone even cares about such things, but we have a number of challenges I don’t mind sharing wisdom with the Internets on.

 More to come in a week or two while I figure out what to do about this hosting situation and get this thing refactored.

Mr. Cuban has another post up about P2P.   I want to refine his model a bit and propose one that I think would work far better.   First, there’s something fundamental that most people don’t get about the Internet business.   While the Internet was designed to be a P2P medium, with end to end connectivity between all nodes, largely it’s become a publish and subscribe model, more like television and less like the phone system.   Since primarily people want the content that’s available “out there” and they’re not so interested in sending things “out there”, the technology and the service offerings have been designed to offer bandwidth asynchronously to the home user.   This means that instead of a something like a T1, which offers 1.544 megabits per second synchronously (meaning you can transfer and receive at the full rate, all the time), home internet usage is sold asynchronously (for example, I have 8 megabits downstream and 2 megabits upstream).   However, at the provider level, bandwidth is sold synchronously.   These providers are buying large pipes (OC48, 2.4 gigabits, OC192, 10 gigabits, etc), which provide for as much upstream as they do downstream, but since their customers buy asynchronously, they generally have large amounts of upstream capacity available.

The problem with the unlimited model is that people will use more on an unlimited plan than they would normally.   Think about the people that feel the need to gorge themselves at a buffet “to get their money’s worth.”   This isn’t necessarily a problem.   The company I work for, Leap Wireless, sells unlimited wireless.   We can do this because there is a significant amount of cost that can be removed as well as a significant amount of profit that’s embedded in the wireless business that we eschew in favor of servicing an underserved customer base.   It’s working well for us now.   However, we don’t work in a business where any given customer can use 100 or 1000 times more of what we’re selling than another.   This makes for an incredibly difficult problem to manage for ISPs.

Mr. Cuban posits that it would be best to start charging for upstream bandwidth, which would limit the amount of seeding done from P2P users.   However, it’s not the seeding that’s slowing down the network, it’s the downstream.   Most protocols are setup to allow more transfer for the more you seed.   So, while his model would work, I think there’s a far simpler model that would work for everyone, although it would surely piss off the net neutrality folks.   Basically, the idea would be to create two tiers of service.   One would be a metered model, which is what the providers would primarily be selling.   The metered model would offer something like 100 to 200 gigabytes of transfer per month, which is far more than the average customer users.   It’s enough to do some P2P transfers without blowing outside your bucket, but it limits the network abusers (the ones downloading terabytes a month) from falling into this plan.   This plan will be a premier plan.   For giving up your unlimited plan, you will be placed into a QoS bucket that has a higher drop priority than unlimited customers.   The second plan is the existing unlimited plan.   This plan could charge more than the rated plan or charge the same, either way has pluses and minuses, and it will offer truly unlimited service.   No letters from the ISP about abuse etc.   The customer is made aware that they are being offered the same max downstream and upstream rates, but they that are receiving a lower class of service.   They will be placed into the lowest QoS bucket.   Without a congestion scenario, no one notices any difference.   During peak times, when the unlimited users are filling up the pipes, the metered users are still receiving high quality always on Internet access, and the unmetered users still get to download to their heart’s content.

This will require the same shaping devices the ISPs are already using to control inbound bandwidth, but rather than shaping at the protocol level, they will shape at a subscriber level.   There is technology already in place to accommodate this (we have a couple of devices from Cisco which will do exactly that).   The primary problem to implementing this strategy for most ISPs will be on the billing and provisioning side, but the software to do this is readily available.

The freeloaders will still get pissed off.   They think if they’re paying for 10 megabits of downstream bandwidth, they should get it, all the time.   They don’t understand the technology problems with actually filling a pipe (TCP wasn’t designed for fat pipe, high latency networks), and they don’t understand the business model of trying to provide high bandwidth connections when there’s no business feasible way of selling the service to where everyone can light up at once and have it work.   Hell, not even the telephone network can accommodate it, which is why during emergencies people are asked to minimize their phone usage, since the phone system can run into capacity issues.   The average consumer might be upset as well, thinking they’re getting less for their money than they used to (“I used to have unlimited, now I’m metered”), but I think this can be solved by education and marketing (“For the same price you’ve always paid, you will now be a premium customer and always have access to all the bandwidth you want, so long as you’re willing to limit your monthly transfers.”)   However, both are being offered the alternative to choose the other plan should they think that the downsides of the plan they’ve chosen outweigh the benefits of the other plan.   Everyone has options.

This will piss off net neutrality folks who think that the network should always be best effort, but this is a pretty justifiable position.   The ISPs have a right to frame their service to their customers how they so choose, and it does not affect how services on the Internet are delivered on a per site basis, merely on per subscriber per plan basis.   This is a legitimate business case which does not affect the ability of customers to have equal access to Internet resources.

In the end, I think it’s a compromise everyone can live with.   The technology is already in place, and I think the missing pieces would be relatively inexpensive to implement given the upsides to the business.   What do you think, Mark?

Strangely, I find myself agreeing with Mark Cuban.   I’ve spent some time thinking back to what I’ve downloaded via P2P applications.   I’ve used BitTorrent and previous P2P technologies to download many things over the years, but I can only think of one legitimate application, and that’s Blizzard using BitTorrent for WoW client distribution.   The potential for this is immense, however, the only reason legitimately for Blizzard to use BitTorrent for distribution is to save on bandwidth costs on their end.   A company like Akamai could easily provide a similar or superior experience for most users, but it would cost Blizzard significantly more than their current distribution model.   I find it ironic that one of the most successful users of legitimate P2P is primarily using it to offload costs from them out to the ISPs when they are probably one of the most successful pay services on the Internet.   The only thing I’d miss about losing various P2P applications is the ability to download television seasons during the summer for viewing.   Mainly this is because there isn’t a suitable for-pay alternative.

Honestly, the striking fact is that 60% of Internet traffic is P2P, and that was from a report from last year.   It’s certainly not going down, if anything it’s increasing.   That means that every bit of traffic most normal users do (web browsing, email, etc) is fighting for bandwidth on networks that are largely congested simply because as soon as the ISPs provision more bandwidth, the P2P users fill up the pipes.   We can get into the oversubscription arguments, but frankly oversubscription is the only way the business model works.   If ISPs had to provision enough bandwidth for everyone to fully light up their last mile pipe to the home, they’d go out of business.   What this means, and what I’ve specifically been noticing more in the past few weeks as I’ve traveled, is that my service is starting to suffer.   Every time I get to a hotel, the damn pipe is filled and I can barely VPN into work to get email.   Even as I come back to Arkansas, I’m noticing that my mother-in-law’s Internet connection with Cox appears to be slow out in Greenwood.   It’s almost impossible without access to the various places I’ve been’s network management systems to fairly diagnose exactly why they’re slower than I expect, but a safe bet would certainly be on lack of bandwidth at the upstream (especially during peak hours) due to P2P users.

If I’m starting to get the feeling like my service is suffering, then shape all the damn P2P traffic down to 0.   Honestly, if I get better service, I’ll probably not lament the loss of my ability to make 250 TCP connections at once to pull down files in little increments at 10KB/sec per connection.   Maybe without the ability to go to the alternative and get the content for free, this will force consumers to start demanding acceptable for pay alternatives for the things they’re getting illegally currently.   I just don’t see anything getting much better in the current stalemate we’re in without some sort of drastic measures.   I just never thought I’d be siding with the providers on this particular issue.

So, if you hadn’t seen the news FireAnt was acquired by Sonic Mountain (Odeo).   You can read recaps of the news on two of my favorite blog networks, NewTeeVee (run by Om Malik), and Tech Crunch (by Mike Arrington).

 I came to be involved in FireAnt through my connections to Jay Dedman and Josh Kinberg.   We had some discussions at Vloggercon in July of 2005 which extended into the following months involving my helping them get FireAnt off the ground.   I had started a project I was calling MediaFeedr, which would poll RSS feeds, examine any links, and then develop a new RSS 2.0 feed with enclosures for downloading into FireAnt.   The theory was that you could put any feed into MediaFeedr and then come out with any linked content as enclosures.   In reality, it never really got out of testing, but the initial feedback was good and I was proud of the code and the idea.

 Jay and Josh were in need of a directory.   Josh had put together some rudimentary code to implement some server side components to tie in the Mac and PC versions of FireAnt, but  while Josh is an excellent visionary and a good leader, he is by his own admission a pretty poor coder.   I took the best of what I had and the best of what Josh had developed and we developed a videoblogging directory and some really innovative server side features to go along with with the video aggregation clients.   We spent months developing it, and we released it to the public on January 24th of 2006 (initial TechCrunch coverage can be found here).   We were ironically directly competing with Odeo at the time for one of the best directories available on the web.   It was developed with AJAX technology which at the time was still fairly new and required a lot of hand coding of JavaScript, etc.

 I was incredibly proud of the work I had done, but even by that point it was becoming obvious that the things we had thought were important weren’t what the market felt was important.   YouTube had in the course of a year become huge, and flash-based web video was where the traffic and the money was at.   The idea of aggregating different forms of video (of which Flash was incredibly hard to play on a PC based client and for the most part no sites supported RSS 2.0 with media enclosures) was falling by the way-side.   After a successful launch but a limit in the amount of video content to be obtained through podcasting, I left in March of 2006 shortly before Katie was born to pursue other opportunities and to limit my workschedule to spend time with my newborn child.

What went wrong then?   I’ve had over a year to reflect on this, and I think I can boil it down to a few choice areas where we wrong:

• Too much focus on the business and not enough focus on the technology
  • We brought in BizDev people very early in the process, in fact before I even officially joined the company.
  • Our BizDev people were unsuccessful at selling the technology.   Simple fact is, they were opportunists who were looking to make a quick buck and really didn’t believe in the company other than they thought they had a gravy-train to ride on.   The early stages of the startup should focus on the technology first and the business second.
• Poor initial design of the business and ownership structure
  • The initial design of the business was a 5 way partnership between two visionaries, two developers and one business development guy.   First of all, equal partnerships never work.   There was no clear leader and far too many chiefs without enough Indians.   When I was brought in, the initial founders were reticent to give up more of their ownership structure since it was already fairly deluted as it was.
• We bet wrong
  • We bet people wanted offline content and simple aggregation of feeds across many websites across the Internet.   Fact was, people wanted one destination in their web browser to view content.   YouTube won, we lost.

 There were great people involved in the founding of the company, but there were just too many.   The next startup I do will have a clear leader, a core set of technology people, and we’ll worry about making money last.   There just isn’t enough of a small company to split it 7 ways.   It should be split three ways and then a quarter left over for the rest to come.   The development people, the ones doing the work to get the technology off the ground should come first.   I’m slightly bitter over the fact that I worked hundreds of hours and at the end of the whole story I ended up with virtually none of the company.   The technology I developed for them was critical to the initial success of the company and I felt from the beginning that even thought my work was highly valued, the ownership percentage was never ponied up.   This is probably why I left early and didn’t stick with the project.   I think had I have stuck with it and not run out of personal funds we probably could have been much more successful.   There were also numerous problems with the client development founders who were also having to work day jobs.   I was the best suited financially at that time due to my severance with Cingular to work for no money, and I was rewarded the least.

 While this may seem harsh to the people who were involved with the company, I want to point out that I feel no ill-will towards the people who I worked with.   Mistakes were made all around, and I have the highest respect for Josh, Jay, Daniel and Erik who were involved in the project during my tenure.   They are all excellent people, and I’d work with all of them again.   I only note these things largely for my own reference, and I point them out so that if I were to ever team up with these people again we can have an open and honest discussion of our mistakes so we don’t repeat them again.   This was a learning experience for all of us, and I hope that some time in the future I can find a way to work with these people again.

 I’d especially like to point out Josh’s effort.   Josh stuck with FireAnt from the beginning to the end.   Josh sacrificed far more than any of the rest of us, even delaying his wedding so that he could see this through to the end.   I consider Josh a close personal friend, and I’d jump at the chance to work with him again.   Josh is an excellent person of the highest moral caliber.   Josh has endured personal threats, personal hardship, and he has endured and completed this project while the rest of us moved on.   I have the utmost respect for the sacrifices he made, and I tip my hat to the Sonic Mountain team who more than the technology we developed got the best part of FireAnt when they got Josh.

 You can still see the technology I developed for FireAnt at getfireant.com.   Some of our more unscrupulous shareholders stole fireant.tv as part of a petty personal squabble, but at least it’s still available there.   To those of you shareholders who were involved in that, shame on you.   Being involved in a small company with no revenue is about sacrifice, dedication and a pursuit of developing your vision, not about cashing out.   Stealing money, lieing, and personal threats are no way to end a failed startup, and I hope you feel ashamed of your behavior.   You know who you are.

 Jay’s thoughts can be viewed here.   Josh’s thoughts can be viewed here.

Usually, we have a hard time getting shoes on Katie.   She throws an incredibly huge fit.   Today, however, we decided to take her outside for a walk and she thoroughly enjoyed it, up until the very end of course.   We’ve uploaded some photos to the gallery and a video for your enjoyment.

Download Video

I just got back from spending three days at Cisco’s corporate headerquarters.   I’ve spent a good portion of my career dealing with vendors.   I’ve spent days in meetings, evaluating products, specing out hardware, leased lines, and other ancillary pieces for building out networks and datacenters, but rarely have I left a set of meetings so jazzed about the future.

The first day was spent in design meetings and demo sessions.   We opened the day by outlining where we’re currently at with our network.   Currently, it’s a fairly well designed network with quite a few well known warts, but for the most part it functions very well.   We’re not really hitting any capacity issues anywhere, but for the first time we’re really trying to plan out 2 to 5 years in advance.   We know our current backbone, which is OC12 at the core nationwide, won’t last us for probably more than the next 12 to 18 months.   We know this, because every time we improve our network, we find new ways to utilize it and we blow away all our traffic projections.   However, we know that while our current design is working really well, we don’t want to upgrade 10 core locations to OC48 since half the sites sitting on that core won’t need that kind of connectivity.   So we spent the first part of the morning drawing up a new core architecture with fewer sites and ideas for making it more highly available, including using DPR and/or MPLS fast reroute to provide sub-50 millisecond or sub-100 millisecond failover time in the event of a network failure.   I think we have a promising set of ideas to go into our meetings next month with Level 3, who we purchase the majority of our core connectivity from.

The second half of the day was spent in demos.   We saw some interesting new products on the Unified Communications front, as well a demo of Telepresence.   Cisco makes this point often, but it can’t be stressed enough.   Telepresence cannot be compared to video conferencing.   We have an existing video conference system, which Telepresence has inspired me to improve, but it’s clunky and awkward by comparison.   Telepresence really makes you feel like you’re sitting across the room from the other participants.   Our current video conference system is used maybe once a month nationwide, but I guarantee Telepresence would be booked 100% in our company.   I’d like to try to make our existing video conference system more like Telepresence, because a lot of the ideas Telepresence puts forward in terms of controlling the environment, using high quality video, and directional audio for the partipants can be adapted to our existing system.   It won’t be Telepresence, but it certainly would improve what we currently have.

We finished up the day with a demo of products Cisco is pushing for retail and a discussion about their Service Control Engine product.   The retail demo inspired a discussion about using EVDO as a backhaul for our retail stores, which is so obvious as to be a real slap in the face that we’re not already testing it out.   We already own that last mile connectivity, why are we continuing to pay our competitors for connectivity to our retail outlets?    Also, the Service Control Engine discussion was fascinating.   We went from discussing a product we didn’t even know we owned to talking about all the things it could do for  us.   If you hear about providers shaping traffic down for Peer to Peer traffic and dunning customers for high bandwidth utilization, this is the product that’s doing it.   It can shape traffic at wire speeds including multiple gigabit connections.   Really really impressive.

The next two days were spent in Cisco’s Customer Proof of Concept lab.   We went there to prove that the 7600 platform which we’re currently basing our core network architecture on will scale to meet our demands over the next 2 to 5 years.   As I expected it did.   I also had an excellent discussion with a Cisco executive over in the Unified Communications area about how we can leverage our investment in our IP Telephony infrastructure to really improve communications and productivity.

Cisco really went out of their way to get us access to people who would could answer our questions, give us good ideas for the future and give us access to equipment to test our their claims for our own assurance.   They’re really the gold standard that I’m holding everyone else up to in terms of how they should treat their customers.   We could have spent the same, dollar for dollar, with many other vendors and barely gotten a sales engineer to give us a call back, but Cisco seems to understand from the top down that people sell products, the products don’t sell themselves.   They understand that we’re looking for a vendor to partner with us to help us get to our next level.   Most often in this business there are two or more products that will do any given job, and while technically one might be superior, in the end both would work for a given task.   The decision comes down to more than what exactly the product does, and while having a superior product helps, having a superior organization that stands behind that product and offers it at competitive prices is in my book the best way to get your product into customers hands.   Cisco has a product for nearly anything you need in the networking world, and while we won’t ever be exclusively a Cisco shop, they’re much more likely to sell us a product on a given day due to the fact that they spend so much time focusing on helping their customers implement their solutions and spend so much time focusing on helping their customers support their existing installations.   At the end of the day, the most important question I have to ask when making a purchasing decision usually isn’t what is the best product, it’s which is the best company.   Right now, Cisco’s going a long way to convincing me they are.

We recently purchased a Canon SD1000 for my 27th birthday. It’s an awesome, incredibly compact camera, unlike our old Nikon which was a total brick and never got used because it was inconvenient to tote around. I’m hoping we’ll be taking a lot more pictures since the camera will be a lot more convenient to have with us. In the mean time, we’ve added a gallery from today’s testing to show off the new camera. Keep checking back, Katie will be featured often. Here’s a sample image of the most beautiful girl in the world:

Once again, open source proves its usefulness. Currently we’re implementing a rather large network monitoring setup from several large companies, mostly with TLAs as their company names. These cost millions of dollars to buy, hundreds of thousands of dollars to customize and equally as much to get the basics working in your environment. I used to use a tool called tkined back in the day to monitor our ISP network via bandwidth graphs overlayed on a network diagram. This was real-time and was the ultimate in network monitoring at the time. Strangely, the tools have all advanced, but I had yet to find something to do an equivalant graphical view of the network. That was until I found PHP Weathermap, which can be installed as a Cacti plugin. We already had a small Cacti installation that just needed to be expanded to include the majority of our core routers (it was mainly being used for ancillary routers and switches), and then I drew the map on top of an existing Visio I exported to PNG. The result:

This gives a near real time view of network bandwidth utilization. It was exactly what I was looking for. When I discussed this with the team doing the implementation from the TLA company, they said it might be possible, but I have a feeling it’s going to cost more than it would take a programmer to rewrite this from the ground up. Thus is enterprise software: the more expensive the package, the less you get out of the box and the more likely you are to end of spending hundreds of thousands if not millions only to replace it with something simpler.

I obviously watch way more television than I should. For some reason these days I find that when I get home from a hard day’s work that I prefer passive entertainment to sitting down and writing my thoughts here. This should seem obvious to anyone who has noticed my posting frequency since I took my new job.

This season, I avidly watched Jericho on CBS. Jericho is a show about a post-apocalyptic society in a small town, called Jericho, in Kansas. Terrorists detonate nuclear bombs in most major cities across the US, and the inhabitants of Jericho must live with the effects of having a broken economy, lack of electricity, food, running water, and a total lack of information from the outside world (although in theory parts of the Internet would have still survived, although that’s I guess addressed by the lack of electricity). There are shortages of nearly everything needed to survive, including food and water, and to make matters worse, the neighboring town of New Bern is coveting what Jericho has and has declared war on Jericho to take large portions of the town away. This is where the season finale cliffhanger left us.

That is, that’s what we thought was the season finale. Apparantly CBS, in it’s infinite wisdom, decided that more reality TV and more cop drama were the answer to their ratings woes. They stuck Jericho up against American Idol, the number one show on television, and then blame the viewers for not tuning in when they decided to cancel the show. The die-hard fans, including myself, are fighting back by sending thousands of pounds of nuts to CBS headquarters (an inside reference to an integral part of the plot of the season finale). If you’ve watched the show, or even if you just care about having quality produced dramas rather than more reality TV drivel and cop dramas on prime-time television, please donate to the cause at the Nuts for Jericho site.

Over on Bram Cohen’s blog (of BitTorrent fame), made a comment that reminded me of a pretty common misconception about the ISP business. Mark said, in reference to Bram and attempting to make money from BitTorrent:

Unfortunately for you,ISPs crack down on heavy bandwidth users, particularly uploaders and enforce their TOS.

By definition, seeders create upstream bandwidth. The ISPs dont want to see more upstream usage Bram, i know its a tough concept for you, but in the mind of the ISP, upstream use = bad. MOre upstream b/w use = more bad. Which in turn pushes them not to increase the bandwidth available to end users, but to evaluate where the upstream use is coming from and look at shutting it off and throttling it. Call me crazy, but that equates to a challenge for the BT universe.

This couldn’t be further from the truth. The upstream bandwidth is not a concern for most ISPs, especially your standard Cable or DSL provider. Broadband ISPs have to order bandwidth synchronously, in various speed increments, because all high capacity circuits come in synchronous form. This means if you order a Gigabit Ethernet connection to an upstream provider, you’re actually buying 2 Gigabits per second worth of bandwidth, one from your provider and another towards your provider. Broadband ISPs sell bandwidth asynchoronously (the A in ADSL), which means they’re selling you something like 6 megabits down and 784k or less upstream. Obviously, it’s well known that ISPs oversubscribe their upstream links which is why peak times can see serious difficulties achieving the full downstream. However, no matter how you do the math of an ISP which serves mostly bandwidth consumers, this still leaves large amounts of bandwidth on the upstream on the ISPs egress connections. In fact, most Broadband ISPs don’t provision large WANs to haul traffic back to central egress points, because it makes more sense to dump the traffic off at a peering point in the market, assuming you’re in a large metro where you can cheap bandwidth. Because ISPs providing broadband to homes and businesses also have such a one way usage pattern, it means that they cannot negotiate peering arrangements with other ISPs for an even trade of bandwidth, meaning they’re always going to be paying for that bandwidth.

In the business, this is generally called Tier 2 bandwidth. Broadband ISPs will sell large hicap circuits to businesses largely interested in serving content much cheaper than you can buy Tier 1 bandwidth from a major internet provider (something like UUNet, etc). This also becomes very evident if you attempt to buy bandwidth from a Tier 1 provider and a Tier 2 or Tier 3 provider, because it becomes very difficult to load balance your traffic over to the Tier 2 providers because of the way BGP routing works. There are methods for overcoming this (AS padding), but it still never works out quite the way you want.

So, in summary, ISPs which are selling bandwidth to users which are using BitTorrent don’t give a rats ass about the upstream. They’ve got loads to spare. What they’re concerned with is the users who are using large amounts of upstream bandwidth are generally also heavy consumers of downstream, and those people are causing oversubscription issues.

Apparantly I haven’t updated this blog in 3 months. My how things change. So, since my last post, I’ve accepted a new job in Denver and my daughter is three months older (crawling, teething, sitting up on her own, etc).

I’m not exactly sure what causes me to not update the blog, but I suppose it’s just life in general. It’s hard to want to sit down and write a blog post when I’m so busy with work and the home life. Katie keeps me busy when I’m at home (because when I’m at home, usually I’ve been gone for some time and it’s my turn to make up for time lost in childcare). When I’m not at home, I’m on the road and I’m busy working.

Lets give the work update because I’m sure that’s mostly what I’ll be writing about in the near future. Almost two months ago it became apparant it was time for a change. I needed to be making more money, and it became obvious that my goals and plans for doing that in Fort Smith, Arkansas were for not. It’s unfortunate, and I’m sure there were many execution failures on my part that could have resulted in a more positive outcome for me continuing to live in Fort Smith, but the reality of the situation is that, in the end, it didn’t work out for the second time. It’s unfortunate, and I’m very sad about it. However, what’s done is done, and while I can reflect upon it and determine what I’ve done wrong, in the end it’s just not going to work out. So, near the end of August I started contacting old friends in the Wireless world, people I used to work at back when I worked at AT&T Wireless/Cingular. Obviously, the most influential of these people was Dave Truzinski, who is the current CIO of Leap Wireless, which you might know as Cricket Communications. A couple of days later Dave came back to me wondering if I’d be interested in a contracting position, and things have progressed from there.

Firstly, let me tell you a bit about Cricket. Cricket is a unique play in the wireless world. In terms of wireless companies out there, Cricket/Leap is very small. Annual revenues were around $1 billion for 2005. Cingular, for example, when I left was around a $60 billion unit of SBC/BellSouth (soon to be the combined AT&T). So in terms of POPs covered and subscribers, Leap is barely on the radar of most industry analysts (I’ve rarely seen Om Malik writing about them, but I think they probably deserve more coverage than they get. They just spent nearly a billion dollars in the AWS spectrum auction for example). So, as I was saying, Cricket is a unique play in the wireless world. Cricket focuses mainly on lower income subscribers, competing with MVNOs like Virgin Mobile, TracFone, etc. However, Cricket is unique in this market being that it owns its own network. Secondly, Cricket is very unique is that it offers all you can eat plans, of which I believe only Metro PCS also offers aside from Cricket.

So, I accepted a contract position with Cricket working on their new nationwide network rollout. We’ve rolled out a killer new nationwide WAN which should provide a good amount of growth room. We’re current doing and will be doing a lot of very interesting things over this grand new network. My job for this contract was basically to help get the ball rolling in finishing the rollout. I flew to various cities including San Diego, Houston, San Antonio, Nashville, Phoenix, Spokane, Tulsa, and probably some others I’ve forgotten over 4 weeks to help them get the facilities provisioned and ready to complete the rollout.

After the contract was nearing an end, I became involved with the IPTel project. I’m now currently working on rolling out Cisco IP Telephony system. I’ve done this before with AT&T Wireless, when we rolled out this same solution to the Caribbean. I came onto a project with numerous issues and a long way to go to completion, so it should be a lot of fun. There’s nothing I like more than to be a fireman, so a project in trouble is my idea of a good time. There’s lots of work to be done and lots of room to roll out some really cool telephony features, so I’m pretty happy at the moment. I’ve written in the past and continue to feel that there’s a lot of issues with larger companies (and while Cricket may be small for the Wireless world, a $1 billion annual company is still a large company in my book), there are certain aspects that make it to where I can fit in quite well. Unfortunately, smaller companies can’t offer the range of opportunities to work on cool projects and cool equipment that larger companies can. Hopefully Cricket is the right size and I can be happy there. Who knows, that remains to be seen, but I’m very excited about the prospects now.

I leave on Monday for San Diego again to work on another project in trouble for a week, then hopefully back to IPTel. I’m going to try to keep this blog updated on the progress of this project, what I’ve seen and experienced with Cisco IP Telephony implementations (especially my experience with it compared to things I really like about Asterisk), and what’s going in general with my business life. Hopefully if I can get some free time at home, I’ll also update with a video of what’s going on with my kid, because she’s really the light of my life, and she’s really at the perfect video age.

Best of luck to the few readers I have left, and I hope I can find time to keep you all updated in the near future.

The one thing I like about writing, and blogging in particular, is that it gives one a chance to air out their thoughts, in public, with the potential for feedback. However, one of the most important aspects that is easily overlooked, and this obviously applies to all journaling and not just blogging, is that it gives one the chance to take partially formed ideas and simply by the act of writing them down, flesh them out into fully formed thoughts. This is what I am going to attempt today. Buckle your seatbelts.

In case you’ve fallen behind in following my life (wouldn’t be hard, I don’t think I’ve talked about it much here in the last few months), I’m currently working for Kirkham Systems and Fireball Media. Kirkham Systems is an established IT services company servicing the Fort Smith, Arkansas metro area offering IT services to small businesses. It’s been very successful, although obviously no business is as successful as it’s proprietors would hope. Fireball Media is an advertising agency, brand spanking new, less than a couple months old, so we really have no expectations or data to suggest whether that one is performing well or not. Time will tell.

Thus far, Kirkham Systems is excelling in its sweet spot of providing services to companies with about 5 to 25 PCs. More than that, most people have decided to hire full time IT help, less than that people generally don’t have the budget to afford our services or prefer to let their kids best friend come in and help them out with those ‘puters. Kirkham Systems also does a significant amount of web design and custom software. It’s a very well rounded company, distinguishing itself not only with its customer service but also in its ability to offer a full suite of services from networking, to custom software, to web design.

However, we haven’t as yet been able to crack the nut that I’d really like to crack. Unfortunately with a company like Kirkham Systems, companies who hire a full time IT staff have usually done so because they feel they’re going to get the best service from having those services in house. If I owned my own business, I can’t see wanting to outsource a full time staff member to waiting in queue at some other company. I know from experience you’d probably get better service than the average person staffing a one person IT shop, but it’s a hard sell. We don’t sell hardware or software solutions, so we really don’t have a lot to offer companies with an IT staff. Like I said, it’s a tough nut to crack.

It’s very difficult to make a service company grow exponentially like a product company can. Take any recent product phenomena like the iPod and you can see what I mean. Rarely can you come up with a service based idea that will catch on like a product can. You can’t hold a service in your hand, you can’t start up its motor, you can’t look at after the salesman has come and gone. You grow a service company one customer at a time, and while you can, and we are, building other existing channels outside of direct sales and standard advertising based marketing, it’s very difficult to find a niche that is wide, has limited competition, and has the potential for growth that a product based company could have.

I’m still looking for the magic solution, but I’m afraid there isn’t one. I think this is largely why service based businesses tend to not be chains, or at least not large ones. Chains require profit margins and scalability that isn’t present in service based businesses. Larger prodcut companies, chains (restraunts, retail, etc) included, require being able to gain effeciencies with the size of the company such that the potential revenue for a given employee grows with the success of the product (or in the case of retail, the mix of products I suppose). Service based businesses, since their revenues are largely based on labor, are limited to revenue per employee based on some multiple of billable hours. Administrative overhead merely cuts into the profit generated by a revenue generating employee, thusly, to grow to the size required to maintain a large administrative staff and bureaucracy as is typical of larger companies becomes nearly inpossible. I believe Brad Feld has written about this. I don’t know what the exact size would be, I think it would depend on the company, but I think every service based business would hit a point where it becomes unprofitable to add more billable employees because the overhead required to manage those employees becomes too expensive.

I think perhaps I’m pondering a problem that I would have learned about in Business School. Perhaps I should go back to college . I may be writing about a problem that’s been written about so extensively as to make my thoughts seem like a rehash of some famous business writer’s thesis. Maybe someone reading this can point me to it. Am I right or wrong? Am I missing the examples of people who have found the magic bullet and turned service based businesses into large enterprise?