Clint Sharp’s Blog an’ Vlog

Mr. Cuban has another post up about P2P.  I want to refine his model a bit and propose one that I think would work far better.  First, there’s something fundamental that most people don’t get about the Internet business.  While the Internet was designed to be a P2P medium, with end to end connectivity between all nodes, largely it’s become a publish and subscribe model, more like television and less like the phone system.  Since primarily people want the content that’s available “out there” and they’re not so interested in sending things “out there”, the technology and the service offerings have been designed to offer bandwidth asynchronously to the home user.  This means that instead of a something like a T1, which offers 1.544 megabits per second synchronously (meaning you can transfer and receive at the full rate, all the time), home internet usage is sold asynchronously (for example, I have 8 megabits downstream and 2 megabits upstream).  However, at the provider level, bandwidth is sold synchronously.  These providers are buying large pipes (OC48, 2.4 gigabits, OC192, 10 gigabits, etc), which provide for as much upstream as they do downstream, but since their customers buy asynchronously, they generally have large amounts of upstream capacity available.

The problem with the unlimited model is that people will use more on an unlimited plan than they would normally.  Think about the people that feel the need to gorge themselves at a buffet “to get their money’s worth.”  This isn’t necessarily a problem.  The company I work for, Leap Wireless, sells unlimited wireless.  We can do this because there is a significant amount of cost that can be removed as well as a significant amount of profit that’s embedded in the wireless business that we eschew in favor of servicing an underserved customer base.  It’s working well for us now.  However, we don’t work in a business where any given customer can use 100 or 1000 times more of what we’re selling than another.  This makes for an incredibly difficult problem to manage for ISPs.

Mr. Cuban posits that it would be best to start charging for upstream bandwidth, which would limit the amount of seeding done from P2P users.  However, it’s not the seeding that’s slowing down the network, it’s the downstream.  Most protocols are setup to allow more transfer for the more you seed.  So, while his model would work, I think there’s a far simpler model that would work for everyone, although it would surely piss off the net neutrality folks.  Basically, the idea would be to create two tiers of service.  One would be a metered model, which is what the providers would primarily be selling.  The metered model would offer something like 100 to 200 gigabytes of transfer per month, which is far more than the average customer users.  It’s enough to do some P2P transfers without blowing outside your bucket, but it limits the network abusers (the ones downloading terabytes a month) from falling into this plan.  This plan will be a premier plan.  For giving up your unlimited plan, you will be placed into a QoS bucket that has a higher drop priority than unlimited customers.  The second plan is the existing unlimited plan.  This plan could charge more than the rated plan or charge the same, either way has pluses and minuses, and it will offer truly unlimited service.  No letters from the ISP about abuse etc.  The customer is made aware that they are being offered the same max downstream and upstream rates, but they that are receiving a lower class of service.  They will be placed into the lowest QoS bucket.  Without a congestion scenario, no one notices any difference.  During peak times, when the unlimited users are filling up the pipes, the metered users are still receiving high quality always on Internet access, and the unmetered users still get to download to their heart’s content.

This will require the same shaping devices the ISPs are already using to control inbound bandwidth, but rather than shaping at the protocol level, they will shape at a subscriber level.   There is technology already in place to accommodate this (we have a couple of devices from Cisco which will do exactly that).  The primary problem to implementing this strategy for most ISPs will be on the billing and provisioning side, but the software to do this is readily available.

The freeloaders will still get pissed off.  They think if they’re paying for 10 megabits of downstream bandwidth, they should get it, all the time.  They don’t understand the technology problems with actually filling a pipe (TCP wasn’t designed for fat pipe, high latency networks), and they don’t understand the business model of trying to provide high bandwidth connections when there’s no business feasible way of selling the service to where everyone can light up at once and have it work.  Hell, not even the telephone network can accommodate it, which is why during emergencies people are asked to minimize their phone usage, since the phone system can run into capacity issues.  The average consumer might be upset as well, thinking they’re getting less for their money than they used to (”I used to have unlimited, now I’m metered”), but I think this can be solved by education and marketing (”For the same price you’ve always paid, you will now be a premium customer and always have access to all the bandwidth you want, so long as you’re willing to limit your monthly transfers.”)  However, both are being offered the alternative to choose the other plan should they think that the downsides of the plan they’ve chosen outweigh the benefits of the other plan.  Everyone has options.

This will piss off net neutrality folks who think that the network should always be best effort, but this is a pretty justifiable position.  The ISPs have a right to frame their service to their customers how they so choose, and it does not affect how services on the Internet are delivered on a per site basis, merely on per subscriber per plan basis.  This is a legitimate business case which does not affect the ability of customers to have equal access to Internet resources.

In the end, I think it’s a compromise everyone can live with.  The technology is already in place, and I think the missing pieces would be relatively inexpensive to implement given the upsides to the business.  What do you think, Mark?

Strangely, I find myself agreeing with Mark Cuban.  I’ve spent some time thinking back to what I’ve downloaded via P2P applications.  I’ve used BitTorrent and previous P2P technologies to download many things over the years, but I can only think of one legitimate application, and that’s Blizzard using BitTorrent for WoW client distribution.  The potential for this is immense, however, the only reason legitimately for Blizzard to use BitTorrent for distribution is to save on bandwidth costs on their end.  A company like Akamai could easily provide a similar or superior experience for most users, but it would cost Blizzard significantly more than their current distribution model.  I find it ironic that one of the most successful users of legitimate P2P is primarily using it to offload costs from them out to the ISPs when they are probably one of the most successful pay services on the Internet.  The only thing I’d miss about losing various P2P applications is the ability to download television seasons during the summer for viewing.  Mainly this is because there isn’t a suitable for-pay alternative.

Honestly, the striking fact is that 60% of Internet traffic is P2P, and that was from a report from last year.  It’s certainly not going down, if anything it’s increasing.  That means that every bit of traffic most normal users do (web browsing, email, etc) is fighting for bandwidth on networks that are largely congested simply because as soon as the ISPs provision more bandwidth, the P2P users fill up the pipes.  We can get into the oversubscription arguments, but frankly oversubscription is the only way the business model works.  If ISPs had to provision enough bandwidth for everyone to fully light up their last mile pipe to the home, they’d go out of business.  What this means, and what I’ve specifically been noticing more in the past few weeks as I’ve traveled, is that my service is starting to suffer.  Every time I get to a hotel, the damn pipe is filled and I can barely VPN into work to get email.  Even as I come back to Arkansas, I’m noticing that my mother-in-law’s Internet connection with Cox appears to be slow out in Greenwood.  It’s almost impossible without access to the various places I’ve been’s network management systems to fairly diagnose exactly why they’re slower than I expect, but a safe bet would certainly be on lack of bandwidth at the upstream (especially during peak hours) due to P2P users.

If I’m starting to get the feeling like my service is suffering, then shape all the damn P2P traffic down to 0.  Honestly, if I get better service, I’ll probably not lament the loss of my ability to make 250 TCP connections at once to pull down files in little increments at 10KB/sec per connection.  Maybe without the ability to go to the alternative and get the content for free, this will force consumers to start demanding acceptable for pay alternatives for the things they’re getting illegally currently.  I just don’t see anything getting much better in the current stalemate we’re in without some sort of drastic measures.  I just never thought I’d be siding with the providers on this particular issue.

So, if you hadn’t seen the news FireAnt was acquired by Sonic Mountain (Odeo).  You can read recaps of the news on two of my favorite blog networks, NewTeeVee (run by Om Malik), and Tech Crunch (by Mike Arrington).

 I came to be involved in FireAnt through my connections to Jay Dedman and Josh Kinberg.  We had some discussions at Vloggercon in July of 2005 which extended into the following months involving my helping them get FireAnt off the ground.  I had started a project I was calling MediaFeedr, which would poll RSS feeds, examine any links, and then develop a new RSS 2.0 feed with enclosures for downloading into FireAnt.  The theory was that you could put any feed into MediaFeedr and then come out with any linked content as enclosures.  In reality, it never really got out of testing, but the initial feedback was good and I was proud of the code and the idea.

 Jay and Josh were in need of a directory.  Josh had put together some rudimentary code to implement some server side components to tie in the Mac and PC versions of FireAnt, but while Josh is an excellent visionary and a good leader, he is by his own admission a pretty poor coder.  I took the best of what I had and the best of what Josh had developed and we developed a videoblogging directory and some really innovative server side features to go along with with the video aggregation clients.  We spent months developing it, and we released it to the public on January 24th of 2006 (initial TechCrunch coverage can be found here).  We were ironically directly competing with Odeo at the time for one of the best directories available on the web.  It was developed with AJAX technology which at the time was still fairly new and required a lot of hand coding of JavaScript, etc.

 I was incredibly proud of the work I had done, but even by that point it was becoming obvious that the things we had thought were important weren’t what the market felt was important.  YouTube had in the course of a year become huge, and flash-based web video was where the traffic and the money was at.  The idea of aggregating different forms of video (of which Flash was incredibly hard to play on a PC based client and for the most part no sites supported RSS 2.0 with media enclosures) was falling by the way-side.  After a successful launch but a limit in the amount of video content to be obtained through podcasting, I left in March of 2006 shortly before Katie was born to pursue other opportunities and to limit my workschedule to spend time with my newborn child.

What went wrong then?  I’ve had over a year to reflect on this, and I think I can boil it down to a few choice areas where we wrong:

• Too much focus on the business and not enough focus on the technology
  • We brought in BizDev people very early in the process, in fact before I even officially joined the company.
  • Our BizDev people were unsuccessful at selling the technology.  Simple fact is, they were opportunists who were looking to make a quick buck and really didn’t believe in the company other than they thought they had a gravy-train to ride on.  The early stages of the startup should focus on the technology first and the business second.
• Poor initial design of the business and ownership structure
  • The initial design of the business was a 5 way partnership between two visionaries, two developers and one business development guy.  First of all, equal partnerships never work.  There was no clear leader and far too many chiefs without enough Indians.  When I was brought in, the initial founders were reticent to give up more of their ownership structure since it was already fairly deluted as it was.
• We bet wrong
  • We bet people wanted offline content and simple aggregation of feeds across many websites across the Internet.  Fact was, people wanted one destination in their web browser to view content.  YouTube won, we lost.

 There were great people involved in the founding of the company, but there were just too many.  The next startup I do will have a clear leader, a core set of technology people, and we’ll worry about making money last.  There just isn’t enough of a small company to split it 7 ways.  It should be split three ways and then a quarter left over for the rest to come.  The development people, the ones doing the work to get the technology off the ground should come first.  I’m slightly bitter over the fact that I worked hundreds of hours and at the end of the whole story I ended up with virtually none of the company.  The technology I developed for them was critical to the initial success of the company and I felt from the beginning that even thought my work was highly valued, the ownership percentage was never ponied up.  This is probably why I left early and didn’t stick with the project.  I think had I have stuck with it and not run out of personal funds we probably could have been much more successful.  There were also numerous problems with the client development founders who were also having to work day jobs.  I was the best suited financially at that time due to my severance with Cingular to work for no money, and I was rewarded the least.

 While this may seem harsh to the people who were involved with the company, I want to point out that I feel no ill-will towards the people who I worked with.  Mistakes were made all around, and I have the highest respect for Josh, Jay, Daniel and Erik who were involved in the project during my tenure.  They are all excellent people, and I’d work with all of them again.  I only note these things largely for my own reference, and I point them out so that if I were to ever team up with these people again we can have an open and honest discussion of our mistakes so we don’t repeat them again.  This was a learning experience for all of us, and I hope that some time in the future I can find a way to work with these people again.

 I’d especially like to point out Josh’s effort.  Josh stuck with FireAnt from the beginning to the end.  Josh sacrificed far more than any of the rest of us, even delaying his wedding so that he could see this through to the end.  I consider Josh a close personal friend, and I’d jump at the chance to work with him again.  Josh is an excellent person of the highest moral caliber.  Josh has endured personal threats, personal hardship, and he has endured and completed this project while the rest of us moved on.  I have the utmost respect for the sacrifices he made, and I tip my hat to the Sonic Mountain team who more than the technology we developed got the best part of FireAnt when they got Josh.

 You can still see the technology I developed for FireAnt at getfireant.com.  Some of our more unscrupulous shareholders stole fireant.tv as part of a petty personal squabble, but at least it’s still available there.  To those of you shareholders who were involved in that, shame on you.  Being involved in a small company with no revenue is about sacrifice, dedication and a pursuit of developing your vision, not about cashing out.  Stealing money, lieing, and personal threats are no way to end a failed startup, and I hope you feel ashamed of your behavior.  You know who you are.

 Jay’s thoughts can be viewed here.  Josh’s thoughts can be viewed here.

I just got back from spending three days at Cisco’s corporate headerquarters.  I’ve spent a good portion of my career dealing with vendors.  I’ve spent days in meetings, evaluating products, specing out hardware, leased lines, and other ancillary pieces for building out networks and datacenters, but rarely have I left a set of meetings so jazzed about the future.

The first day was spent in design meetings and demo sessions.  We opened the day by outlining where we’re currently at with our network.  Currently, it’s a fairly well designed network with quite a few well known warts, but for the most part it functions very well.  We’re not really hitting any capacity issues anywhere, but for the first time we’re really trying to plan out 2 to 5 years in advance.  We know our current backbone, which is OC12 at the core nationwide, won’t last us for probably more than the next 12 to 18 months.  We know this, because every time we improve our network, we find new ways to utilize it and we blow away all our traffic projections.  However, we know that while our current design is working really well, we don’t want to upgrade 10 core locations to OC48 since half the sites sitting on that core won’t need that kind of connectivity.  So we spent the first part of the morning drawing up a new core architecture with fewer sites and ideas for making it more highly available, including using DPR and/or MPLS fast reroute to provide sub-50 millisecond or sub-100 millisecond failover time in the event of a network failure.  I think we have a promising set of ideas to go into our meetings next month with Level 3, who we purchase the majority of our core connectivity from.

The second half of the day was spent in demos.  We saw some interesting new products on the Unified Communications front, as well a demo of Telepresence.  Cisco makes this point often, but it can’t be stressed enough.  Telepresence cannot be compared to video conferencing.  We have an existing video conference system, which Telepresence has inspired me to improve, but it’s clunky and awkward by comparison.  Telepresence really makes you feel like you’re sitting across the room from the other participants.  Our current video conference system is used maybe once a month nationwide, but I guarantee Telepresence would be booked 100% in our company.  I’d like to try to make our existing video conference system more like Telepresence, because a lot of the ideas Telepresence puts forward in terms of controlling the environment, using high quality video, and directional audio for the partipants can be adapted to our existing system.  It won’t be Telepresence, but it certainly would improve what we currently have.

We finished up the day with a demo of products Cisco is pushing for retail and a discussion about their Service Control Engine product.  The retail demo inspired a discussion about using EVDO as a backhaul for our retail stores, which is so obvious as to be a real slap in the face that we’re not already testing it out.  We already own that last mile connectivity, why are we continuing to pay our competitors for connectivity to our retail outlets?  Also, the Service Control Engine discussion was fascinating.  We went from discussing a product we didn’t even know we owned to talking about all the things it could do for us.  If you hear about providers shaping traffic down for Peer to Peer traffic and dunning customers for high bandwidth utilization, this is the product that’s doing it.  It can shape traffic at wire speeds including multiple gigabit connections.  Really really impressive.

The next two days were spent in Cisco’s Customer Proof of Concept lab.  We went there to prove that the 7600 platform which we’re currently basing our core network architecture on will scale to meet our demands over the next 2 to 5 years.  As I expected it did.  I also had an excellent discussion with a Cisco executive over in the Unified Communications area about how we can leverage our investment in our IP Telephony infrastructure to really improve communications and productivity.

Cisco really went out of their way to get us access to people who would could answer our questions, give us good ideas for the future and give us access to equipment to test our their claims for our own assurance.  They’re really the gold standard that I’m holding everyone else up to in terms of how they should treat their customers.  We could have spent the same, dollar for dollar, with many other vendors and barely gotten a sales engineer to give us a call back, but Cisco seems to understand from the top down that people sell products, the products don’t sell themselves.  They understand that we’re looking for a vendor to partner with us to help us get to our next level.  Most often in this business there are two or more products that will do any given job, and while technically one might be superior, in the end both would work for a given task.  The decision comes down to more than what exactly the product does, and while having a superior product helps, having a superior organization that stands behind that product and offers it at competitive prices is in my book the best way to get your product into customers hands.  Cisco has a product for nearly anything you need in the networking world, and while we won’t ever be exclusively a Cisco shop, they’re much more likely to sell us a product on a given day due to the fact that they spend so much time focusing on helping their customers implement their solutions and spend so much time focusing on helping their customers support their existing installations.  At the end of the day, the most important question I have to ask when making a purchasing decision usually isn’t what is the best product, it’s which is the best company.  Right now, Cisco’s going a long way to convincing me they are.

We recently purchased a Canon SD1000 for my 27th birthday. It’s an awesome, incredibly compact camera, unlike our old Nikon which was a total brick and never got used because it was inconvenient to tote around. I’m hoping we’ll be taking a lot more pictures since the camera will be a lot more convenient to have with us. In the mean time, we’ve added a gallery from today’s testing to show off the new camera. Keep checking back, Katie will be featured often. Here’s a sample image of the most beautiful girl in the world:

Once again, open source proves its usefulness. Currently we’re implementing a rather large network monitoring setup from several large companies, mostly with TLAs as their company names. These cost millions of dollars to buy, hundreds of thousands of dollars to customize and equally as much to get the basics working in your environment. I used to use a tool called tkined back in the day to monitor our ISP network via bandwidth graphs overlayed on a network diagram. This was real-time and was the ultimate in network monitoring at the time. Strangely, the tools have all advanced, but I had yet to find something to do an equivalant graphical view of the network. That was until I found PHP Weathermap, which can be installed as a Cacti plugin. We already had a small Cacti installation that just needed to be expanded to include the majority of our core routers (it was mainly being used for ancillary routers and switches), and then I drew the map on top of an existing Visio I exported to PNG. The result:

This gives a near real time view of network bandwidth utilization. It was exactly what I was looking for. When I discussed this with the team doing the implementation from the TLA company, they said it might be possible, but I have a feeling it’s going to cost more than it would take a programmer to rewrite this from the ground up. Thus is enterprise software: the more expensive the package, the less you get out of the box and the more likely you are to end of spending hundreds of thousands if not millions only to replace it with something simpler.

Over on Bram Cohen’s blog (of BitTorrent fame), made a comment that reminded me of a pretty common misconception about the ISP business. Mark said, in reference to Bram and attempting to make money from BitTorrent:

Unfortunately for you,ISPs crack down on heavy bandwidth users, particularly uploaders and enforce their TOS.

By definition, seeders create upstream bandwidth. The ISPs dont want to see more upstream usage Bram, i know its a tough concept for you, but in the mind of the ISP, upstream use = bad. MOre upstream b/w use = more bad. Which in turn pushes them not to increase the bandwidth available to end users, but to evaluate where the upstream use is coming from and look at shutting it off and throttling it. Call me crazy, but that equates to a challenge for the BT universe.

This couldn’t be further from the truth. The upstream bandwidth is not a concern for most ISPs, especially your standard Cable or DSL provider. Broadband ISPs have to order bandwidth synchronously, in various speed increments, because all high capacity circuits come in synchronous form. This means if you order a Gigabit Ethernet connection to an upstream provider, you’re actually buying 2 Gigabits per second worth of bandwidth, one from your provider and another towards your provider. Broadband ISPs sell bandwidth asynchoronously (the A in ADSL), which means they’re selling you something like 6 megabits down and 784k or less upstream. Obviously, it’s well known that ISPs oversubscribe their upstream links which is why peak times can see serious difficulties achieving the full downstream. However, no matter how you do the math of an ISP which serves mostly bandwidth consumers, this still leaves large amounts of bandwidth on the upstream on the ISPs egress connections. In fact, most Broadband ISPs don’t provision large WANs to haul traffic back to central egress points, because it makes more sense to dump the traffic off at a peering point in the market, assuming you’re in a large metro where you can cheap bandwidth. Because ISPs providing broadband to homes and businesses also have such a one way usage pattern, it means that they cannot negotiate peering arrangements with other ISPs for an even trade of bandwidth, meaning they’re always going to be paying for that bandwidth.

In the business, this is generally called Tier 2 bandwidth. Broadband ISPs will sell large hicap circuits to businesses largely interested in serving content much cheaper than you can buy Tier 1 bandwidth from a major internet provider (something like UUNet, etc). This also becomes very evident if you attempt to buy bandwidth from a Tier 1 provider and a Tier 2 or Tier 3 provider, because it becomes very difficult to load balance your traffic over to the Tier 2 providers because of the way BGP routing works. There are methods for overcoming this (AS padding), but it still never works out quite the way you want.

So, in summary, ISPs which are selling bandwidth to users which are using BitTorrent don’t give a rats ass about the upstream. They’ve got loads to spare. What they’re concerned with is the users who are using large amounts of upstream bandwidth are generally also heavy consumers of downstream, and those people are causing oversubscription issues.

Apparantly I haven’t updated this blog in 3 months. My how things change. So, since my last post, I’ve accepted a new job in Denver and my daughter is three months older (crawling, teething, sitting up on her own, etc).

I’m not exactly sure what causes me to not update the blog, but I suppose it’s just life in general. It’s hard to want to sit down and write a blog post when I’m so busy with work and the home life. Katie keeps me busy when I’m at home (because when I’m at home, usually I’ve been gone for some time and it’s my turn to make up for time lost in childcare). When I’m not at home, I’m on the road and I’m busy working.

Lets give the work update because I’m sure that’s mostly what I’ll be writing about in the near future. Almost two months ago it became apparant it was time for a change. I needed to be making more money, and it became obvious that my goals and plans for doing that in Fort Smith, Arkansas were for not. It’s unfortunate, and I’m sure there were many execution failures on my part that could have resulted in a more positive outcome for me continuing to live in Fort Smith, but the reality of the situation is that, in the end, it didn’t work out for the second time. It’s unfortunate, and I’m very sad about it. However, what’s done is done, and while I can reflect upon it and determine what I’ve done wrong, in the end it’s just not going to work out. So, near the end of August I started contacting old friends in the Wireless world, people I used to work at back when I worked at AT&T Wireless/Cingular. Obviously, the most influential of these people was Dave Truzinski, who is the current CIO of Leap Wireless, which you might know as Cricket Communications. A couple of days later Dave came back to me wondering if I’d be interested in a contracting position, and things have progressed from there.

Firstly, let me tell you a bit about Cricket. Cricket is a unique play in the wireless world. In terms of wireless companies out there, Cricket/Leap is very small. Annual revenues were around $1 billion for 2005. Cingular, for example, when I left was around a $60 billion unit of SBC/BellSouth (soon to be the combined AT&T). So in terms of POPs covered and subscribers, Leap is barely on the radar of most industry analysts (I’ve rarely seen Om Malik writing about them, but I think they probably deserve more coverage than they get. They just spent nearly a billion dollars in the AWS spectrum auction for example). So, as I was saying, Cricket is a unique play in the wireless world. Cricket focuses mainly on lower income subscribers, competing with MVNOs like Virgin Mobile, TracFone, etc. However, Cricket is unique in this market being that it owns its own network. Secondly, Cricket is very unique is that it offers all you can eat plans, of which I believe only Metro PCS also offers aside from Cricket.

So, I accepted a contract position with Cricket working on their new nationwide network rollout. We’ve rolled out a killer new nationwide WAN which should provide a good amount of growth room. We’re current doing and will be doing a lot of very interesting things over this grand new network. My job for this contract was basically to help get the ball rolling in finishing the rollout. I flew to various cities including San Diego, Houston, San Antonio, Nashville, Phoenix, Spokane, Tulsa, and probably some others I’ve forgotten over 4 weeks to help them get the facilities provisioned and ready to complete the rollout.

After the contract was nearing an end, I became involved with the IPTel project. I’m now currently working on rolling out Cisco IP Telephony system. I’ve done this before with AT&T Wireless, when we rolled out this same solution to the Caribbean. I came onto a project with numerous issues and a long way to go to completion, so it should be a lot of fun. There’s nothing I like more than to be a fireman, so a project in trouble is my idea of a good time. There’s lots of work to be done and lots of room to roll out some really cool telephony features, so I’m pretty happy at the moment. I’ve written in the past and continue to feel that there’s a lot of issues with larger companies (and while Cricket may be small for the Wireless world, a $1 billion annual company is still a large company in my book), there are certain aspects that make it to where I can fit in quite well. Unfortunately, smaller companies can’t offer the range of opportunities to work on cool projects and cool equipment that larger companies can. Hopefully Cricket is the right size and I can be happy there. Who knows, that remains to be seen, but I’m very excited about the prospects now.

I leave on Monday for San Diego again to work on another project in trouble for a week, then hopefully back to IPTel. I’m going to try to keep this blog updated on the progress of this project, what I’ve seen and experienced with Cisco IP Telephony implementations (especially my experience with it compared to things I really like about Asterisk), and what’s going in general with my business life. Hopefully if I can get some free time at home, I’ll also update with a video of what’s going on with my kid, because she’s really the light of my life, and she’s really at the perfect video age.

Best of luck to the few readers I have left, and I hope I can find time to keep you all updated in the near future.

Joel Spolsky points me to a post by Mike Gunderloy about VMWare releasing their GSX Server product for free. This is outstanding news, and I plan to test it out at the office this week. I guess VMWare listened to my advice to Microsoft, which I wrote in May of last year. Virtualization technology is already free with Xen and Linux systems, it’s about time one of the commercial vendors released one of their lower-end products for people to use for free. Way to go VMWare!

I like Steve Rubel. I like Robert Scoble too. Robert’s dead wrong on this one though. For some reason, bloggers seem to think that just because there’s more of us and anyone can contribute the conversation, that somehow everything has to change. Not so.

Perfect example, we just did a major release about 3 weeks ago of FireAnt. We spent a lot of time on the product. The directory was over 4 months in development. There were test sites available to the public about a month prior to release. We seeded the release out to trusted videobloggers and our users groups for the product to get feedback, but we asked all of them to remain quiet. They did. The reason? We wanted to give someone who had traction the exclusive to write about the new release such that we’d get a bit of a bang with our release instead of a gradual dull thud. That exclusive fell to Mike Arrington of TechCruch, and we were not disappointed. He got the exclusive, he was happy, his readers got the scoop the day it was released, and we got extended coverage in the blogosphere echo chamber because we gave a high-profile blogger the exclusive.

Steve groks it. I’m not sure why Chris and Robert seem to think everything has changed. I could have had the exclusive or given it to someone like my good friend Steve Garfield (whose readership/viewership is nothing to sneeze at), but why would I want to release something to my 200 readers and wait for it to maybe disseminate throughout the blogosphere when I can seed it to someone with a much larger and more influential readership? If we had given it to everyone all at once, we would have ended up with that dull thud I was talking about earlier. Somebody has to help control the noise, and a little bit of PR and marketing savvy can go a long way to doing that.

This came across an email list I’m on recently:

Ning co-founder Marc Andreessen recently said…

Ideally we’ll never meet any of our customers. We actually had to take the sign down from our front door because one of our customers actually stopped in, uninvited, and said, “Hi, I love your service.� And we’re like, “why are you here?� And so down came the sign.

Drop-bys like that should only happen in sitcoms as far as I’m concerned… The consumer internet businesses in a sense are ideal businesses from the standpoint of never meeting your customers.

Only in the technology business would anyone be caught dead uttering such an utterly stupid statement, and even then it doesn’t make it any less of a moronic comment. Your customers are your bread and butter. You should jump up and down if someone takes the time to stop by your office just to tell you how much they like you, and you should be just as excited if someone takes the time to tell you what you’re doing poorly, because it’s a chance to save a customer and make an advocate. This is something I’d expect to see on Rick Segal’s blog, in one of his infamous (at least to me) overheard dumb business conversations. I can’t honestly see anyone with this kind of attitude being successful in any business in the long run, technology or not.

A while back, I wrote about what I’m calling the Local Web. The Local Web, in my mind, is a group (an infinite number of groups are possible) which arrange their interconnectedness by sharing a geographical point of reference, traditionally Metropolitican Statistical Areas, or MSAs. The Local Web is already built in many of the larger cities, with directories and vertical search engines to allow you to search for stuff in major metropolitan areas, but a good percentage if not the majority of Americans live outside of a major metropolitan area. The connected netizens from those areas are being largely overlooked by current major initiatives to create localized web experiences.

I’m starting an experiment in a town that should be the perfect size. My hometown is Fort Smith, Arkansas, a town of about 80,000 with about a quarter million in the MSA. There are billions of dollars of business done every year here, and many companies here ship worldwide. However, for doing business in town, most people still reach for the phone book. The reason for this, of course, is because you can spend days Googling around for information about Fort Smith businesses without finding much but spam sites. No one in this town has made a concerted effort to make sure things are easily found on the web about businesses they’d like to do business with.

So, I’m starting an experiment. I’m going to organize a blogger meetup to start. I’ve already found several local bloggers and I’m going to find or create more. I’m going to organize them and attempt to get them to write about business and other activities (softball, church, whatever) they that they do locally and where they do them at. I’m going to try to incent people to create links from site to site across town and try to make information more easily indexable by the search engines so that when you search for something in the area you don’t end up at a spam site. We will be holding the meetings at Kirkham Systems of Fort Smith.

Once this is going strongly, I, along with the staff of Kirkham Systems are going to start showing the results to local businesses and convince them they should have a website with a blog and incent them to link to the people they’re doing business with and write about their experiences with it. The goal is to create an interconnected web of links focused on this geographical area, so that if you end up at Kirkham Systems website you’ll find annotated links about the people we do business with, and when you end up there you can find the people they do business with.

If I’m right, by the time I’m done, Google will be a far more interesting resource to find information about businesses, things and places in Fort Smith, Arkansas than any other resource, anywhere. This may seem boring to people who live on the coasts and can find a well designed and well organized website for even local businesses, but for the large portions of the country that have been ignored by businesses attempting to organize information for them on the web, I think this will be a large step forward. No one understands or cares about this because they haven’t been educated as to what it can mean for both their businesses, themselves and their community. My goal is to educate everyone here.

The Local Web is long overdue.

There’s some discussion going in on in the comments of my last post. Check it out, I think we might be in for an interesting discussion.

My friend Raymond is getting some attention for his love of OPML over on Dave Winer’s Scripting.com here and here. As is typical of format geeks, there’s a debate over on Raymond’s blog comments about why you’d use OPML over XHTML ordered and unordered lists. When are people going to realize that 99% of people don’t care? I’ve been involved in more format discussions than I care to remember, and in the end the reason RSS and OPML will become popular is because Dave Winer goes to the effort to develop tools rather than writing specifications and hoping someone will write tools for them. The format in the end doesn’t really matter much, it’s just a way to format data. There have been thousands over the years, and as long as everyone can read it, the rest is just syntax and semantics.

Josh and I were having a debate the other day as to whether using a pseudo-protocol like fireant:// was an acceptable solution for one-click subscribe in our aggregator. Most of the other aggregators are fighting over feed:// or some other specific file format (like iTunes pcast files). Why should we worry about all this when all we want is to enable easy one-click subscribe for people who already have our software? Josh’s concern is that the geeks will be upset over our use of a protocol that’s not really a protocol (of course, no need to remind people that feed:// isn’t a valid protocol either) instead of doing it through a file or some other method that’s more robust. Sorry, it works. The facilities are already in the OS and the browser to facilitate it, why not use it? Is it a hack? Yeah, so what? It works!

The same people who would be upset about us using fireant:// as a protocol are the ones who’d be upset that people are using OPML rather than XHTML formatted unordered and ordered lists. Hello? Who fucking cares. The user cares that it works! We spend far too much time debating the merits of one format over another and lot less time than we should making sure that software works for the end user. This is why Dave Winer continues to be a success in getting formats adopted, because unlike the Atom folks who have spent years making a format that’s the most robust and most well-documented, there isn’t a refrence implementation. Why is Microsoft Word the default format for exchanging documents and not OASIS? Because of the software people use. Why is RSS the preferred format for exchanging feed information? Because there was software that worked when the format was introduced that everyone could use as a reference implementation.

There something also to be said for simplicity. OPML and RSS are simple. Perhaps the specs are not complete and don’t cover all the use cases, but I can also code something up to work with them in a matter of hours. I investigated the Atom publishing protocol, and it would take me a couple days to do a pull implementation. By contrast, I have done a full Metaweblog implementation in a couple of hours.

Dave Winer can be an ass, but I give him credit where credit is due. The people who spend so much time complaining about him are excellent at complaining and not so good at getting things done. For that, I look to Dave.

Seems the pronoucements (including my own) of Solaris’ death were a bit premature, according to David Berlind. I agree. In terms of performance and stability, Solaris is definitely the 800 pound gorilla over Linux, and now with it being essentially free on x86 (it has been for a while, although making it open source definitely makes this much more clear) I see no reason not to deploy it over Linux. I’ll definitely be exploring this as an option for some of my new deployements, especially with ZFS out in the wild now.

I love reading Paul Graham. He’s a must read for anyone in the technology business, especially anyone who is either involved with or considering a startup. His latest essay, “How to Fund a Startup” is an excellent read and very timely for me. Howevever I do take exception to his third note, which states:

[3] If “near you” doesn’t mean the Bay Area, Boston, or Seattle, consider moving. It’s not a coincidence you haven’t heard of many startups from Philadelphia.

If the Internet has done anything, it’s changed the dynamics of the workplace such that it doesn’t really matter where you’re located anymore. I’m still highly in favor of having a physical office and a place where people can collaborate, but as long as you can find your core team where you’re located, you can fill in the rest from wherever in the world you so choose. I lived in Arkansas for 23 years. I’ve lived in Seattle for 2. I’ve met a lot of technology people out here, and lot of them are very smart, but I also know a lot of incredibly smart technology people back home in Arkansas as well. I’d take anyone’s bet that I could start a software company and develop software on par with any California company from my home state (perhaps with some work filled in from overseas or out of state, but that’s what this whole Internet thing is about). If we’re still thinking that all successful startup technology companies need to be located in one of those three places, things really haven’t changed. I’m very disappointed in Paul, because I would have thought of all the people I respected and read on a regular basis he would have thought differently. I guess old prejudices never die.

I’ve been searching for this for a year and a half.  My laptop runs like shit.  I’ve always meant to come back around and find out exactly what’s wrong with it.  It’s not incredibly high CPU usage, it’s not swapping, it’s not disk I/O.  But, I went and grabbed Process Explorer from SysInternals today and I see that when I’m docked, and I have a USB mouse plugged in, my Hardware Interrupts go through the roof and the DPCs (deferred procedure calls) take up 30-40% of my CPU.  This is obviously not right.  I’ve updated everything I can think of from Dell’s website and still I’m having the same problem.  I’m now installing Service Pack 2 to see if that helps, but I’m going to be fucking ecstatic when I can finally dump this piece of shit Dell laptop for one I buy and setup myself.  I will have no problem saying goodbye and good riddance to Cingular’s corporate IT and their insistance on buying poor hardware and loading it up with crappy software that kills the performance on my PC.

Update: For anyone who happens to come across this later via a search engine, Windows XP SP2 fixed the problems I was having with a USB mouse plugged into the Dock USB controller causing 30-40% processor usage via the “Deferred Procedure Calls” or DPCs.

I had to read the whole article to really believe to what extent Sony would go to protect one damn CD, but this software they install, with no prompting, no EULA acceptance, nothing, uses the same technology rootkits use to hide their existance on your system.  Amazing.  Read the whole thing, from Sysinternals.

Russell Beattie has discovered video podcasts, and wants to start one of his own. I’m looking forward to it, Russell, but just a word of caution: it’s very time-consuming it’s not particularly easy to make it look good.

The part that’s annoying me these days about the Video iPod is that people seem to have missed that we’ve been doing this “video podcasting” thing for quite some time, only we called it vlogging. Check out the Yahoo! Videoblogging Group, where we’ve been discussing all things Internet video for over a year now. There’s a lot of really good content out there, and my job is to help you find it. Hopefully in the next few weeks I can show everyone our new directory and we can show everyone what “video podcasting” is really all about.

Redwood Virtual (the host I use for my email virtual server) is down again. These guys suck. I hear they live in Seattle. If I was ever able to contact a person there, I’d find out their names so I could personally drive over there and beat the shit out of them. Absolutely no excuse to run a business this way. I hope they die and rot in hell. If you’ve tried to email me today, sorry. In the mean time I’m checking my gmail account (coccyx@gmail.com… I don’t do spam obfuscation, if they want my email they probably already have it). I’ve gotta get my shit moved somewhere else.