Dealing with hypergrowth

Many people will think that handling hyperfast growth is a nice problem to have in their business. Yes and no. While it is great to have a business that is growing in revenue and customers, in these days of Software as a Service and virtualized servers keeping up can be a challenge. It isn’t always obvious how to build an infrastructure that can scale up quickly. A recent article about the career of Aditya Agarwal in First Round is worth taking a closer look. Agarwal went to Oracle right out of college, and has been with Facebook when it had just a few employees, and has worked at Dropbox and other Internet companies.

If you are trying to build an engineering team that can handle growth, here are some of the highlights from the article that resonated with me.

1. Be flexible. You might have to move to a different team, or take a different approach, because of the increased scale. Or realize that your code was built for a smaller customer base. Don’t dwell on it.
2. See the bigger picture. They wanted people “who can think about the product all the way down to the infrastructure.” That means understanding how your code fits in to the overall project and how what you work on can influence someone else’s code too.
3. Be a polyglot programmer. Don’t use a hammer for every job, and learn new programming languages when it is appropriate for a particular project. Spend some time to develop your skills so you can work on multiple platforms, OSs etc.
4. Fight complacency or depression. Try to work each day on a more even keel. This is going to be hard work, and don’t get discouraged. Accept failure as part of the learning process. Be on the outlook for what needs to be done and set your sights accordingly.
5. Develop your leadership skills. Know when you can’t fix something by yourself and how to find someone who can help you. Leading a team isn’t just about telling someone to do something, but also about understanding what your team’s strengths and weaknesses are and how to code around these problems. It is also about providing the right motivation, resolving conflicts quickly, and making sure communication and collaboration is flowing in a positive direction. Be transparent about your decision process and communicate often and clearly about what you are doing and why.
6. Know the trade offs between producing quality or quantity. There are times to focus on one or the other, and just be able to deal with that. There is no right or wrong or absolute way. Agrawal mentioned that when he went to Oracle for his first job, he wasn’t expected to deliver any code for the first month. Facebook was another story: there he was expected to contribute code on his first day.
7. How much chaos can you tolerate? How much can your customers tolerate? These are important questions to ask and get answers to that define your daily work habits.
8. Know the corporate values and cherish them. Learn from your new hires that have worked at established software companies and how they have built their cultures over the years.

Network World: You are viewing Insider content. Browse other Insider articles Secure browsers offer alternatives to Chrome, IE and Firefox

The Web browser has been a major infection vector for years, allowing malware to be transported to millions of computers through phishing, man-in-the-middle, SQL injection and countless other attacks. But what if there was a way to stop this madness and secure the browsing channel itself?

spoon consoleI talk about ways to replace existing Chrome and Firefox browsers with a new breed of products, such as this screenshot of Spoon at left. You can read my article posted on Network World today here.

Why New Jersey needs to slow down its traffic

It is ironic that the same thing that got Chris Christy in trouble – delaying traffic into Manhattan — is being used by others to build a multi-million business. I am talking about network traffic for stock traders. Perhaps you have seen the stories about them based on Michael Lewis’ latest book Flash Boys.

In the celebrated George Washington “bridgegate” fracas, it was delays introduced by closing various on-ramps to the bridge. In the case of the stock market, it was delays of 350 microseconds that made high-frequency trades more equitable across exchanges.

Wait a minute, come again? Adding in latency is a good thing? Yes, that is another irony of the situation. A company called IEX developed a technique where they slow down the trades so that the exchanges can get the trading requests at the same time. This means that exchanges closer (in terms of connection time) to the Big Apple trading desks can’t trade a few microseconds ahead of the others. The technique IEX developed is basically a big spool of fiber optic cable that is 35 miles long (pictured above), the length it takes a beam of light to traverse 350 microseconds. The problem is that the biggest trading exchanges are located at different peering points in New Jersey, with some being up to 35 miles apart from others, at least in terms of how their packets are routed. Another irony: these locations were chosen so that the exchanges would have as little latency as possible to gain access to the trading data streams (see the map from the New York Times below).

It is probably the first time that anyone was deliberately introducing more network latency to improve their business that I have heard of. Many of us have spent a good chunk of our careers trying to cut down on latency issues: I can remember when I rolled out the first PC local area network application at MegaLith Insurance back in the middle 1980s. The file transfer app that normally took a few seconds to get from one point to another across our mainframe network now took tens of minutes. That wasn’t a good thing.

Programming professor Donald Knuth wrote in one of his seminal books how “programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs.” (Thanks to Dave Methvin of jQuery Foundation for uncovering this quote.) And in the world of high-frequency trading, a company called Spread Networks invested hundreds of millions of dollars building a fiber connection from Chicago to NYC a few years ago, only to find that a microwave network could shave even more time off their latency figures.

So we have the Internet to thank for introducing all kinds of unpredictable latencies into our apps and drive us nuts trying to track down the culprit. Now lower latency has been productized, thanks to the smart guys at IEX. Maybe the next step will be for the traders to actually colocate their offices on top of the major peering points themselves: that could actually work for cutting down on the times of some New Jersey commuters, too.

Why Bitcoin isn’t just for criminals

bitcoinMost of us have lived and worked in a single country and don’t think anything of being paid in anything other than that country’s currency. But we now exist in a global village, and that means that we have choices for our money supply. This is the area that Bitcoin and its ilk are now operating in.

The brave new world of independent money is upon us, and this time I think it is going to take hold. We tried this before with all the digital cash variants in the mid-1990s: a combination of the Internet boom and the rise of internationally accepted credit cards nipped those efforts in the bud. One good thing to come from all of that work was Paypal, which while not new money is the default way that many online transactions happen.

But what is different about today’s money that will make it stick? For one thing, there is already a lot of it in circulation: Bitcoin alone has already topped a billion or so US$. This has motivated some criminals to follow the money: witness the high-tech bank heist of Mt. Gox not too long ago. But they aren’t just trying to steal the digital coins; criminals are also making a big effort to use these new currencies for their own transactions. This is always a great endorsement for the legitimate use of any currency. (Which is why a US$100 bill is worth a bit more than 100 George Washingtons in Russia.)

You have people asking to be paid in Bitcoin for all sorts of things now, including over-priced New York City apartments (see the photo above), vending machines, restaurants, overpriced electronics and other mundane activities. It is somewhat counter-intuitive, but the same thing happened in the early Paypal days too.

Finally, these new efforts have important payment gateway applications, and that is what I want to really focus on. While Bitcoin et al. are getting a lot of attention for their seamier side, what is really happening is that they are attacking the overpriced payment transfer business. Look at how much it costs you for an international money order, or a wire transfer? What if we could drive these costs to near zero, and make it easier for anyone to pay anyone? That is the ultimate benefit of Bitcoin.

When I get paid by Paypal, it is in US$ and the funds don’t stay in my account for very long. I look at that account as a mechanism to collect money that is ultimately destined for my ordinary business checking account, which is where I pay bills and pay my ultimate salary. In the past, when I have worked for non-US clients, there has always been a dance about the process of being paid: sometimes the client has a dollar bank account that they can draw a check on, other times I can get sent funds via Paypal. And sometimes people ask me to bill them via their credit cards. Wouldn’t it be easier if I could have a service that handled these transfers easily and cheaply and reliably?

Bitcoin might not be the ultimate place that I will do my banking. Given its wide swings in exchange rates, I am not sure that I would trust having much of my funds in Bitcoin at the moment. Ideally, I would want a system that could accept transfers from a wide collection of currencies, including dollars and Euros, just to make sure that I cover as much of my potential audience as I could. I started looking into Ripple, which has a more inclusive philosophy, but its universe is also more complex to evaluate.

Unlike many of my columns, I am still doing research here. I don’t have any definitive answers, and these new forms of money might go the way of their older virtual cousins in another year or two. But it is interesting to watch what they are doing nonetheless.

ITworld: Hackathon gold: How to win a job offer in a coding competition

aaaAre software competitions bad? Some participants say hackathons can stifle innovation and chill the vibe of camaraderie because they offer such large prizes. But that doesn’t have to always be the case.

A lot depends on what happens during the contest itself, and who is participating and who is running the competition. Take the case of the January 2014 GlobalHack contest in St. Louis that was initially attended by several hundred programmers. The story of the contest isn’t who took away the top $50,000 prize but about the other participants who didn’t finish in the money but came away with something else that is arguably more important.

I interviewed a couple of the participants at GlobalHack and explain what happened to them in my post over on ITworld here.

Intuit blog: Time to end spreadsheet abuse!

Spreadsheets are great – when you’re creating a spreadsheet. But software like Excel isn’t the right tool for a lot of tasks, and vast sins have been committed in the name of using this hammer for things that emphatically are not nails.The versatile spreadsheet has been used for formatting a resume, running a presentation, a rudimentary word processor, drawing pictures of a network’s infrastructure connections, a simple database, and even drawing a map

The next supercomputer may be your cellphone

What if you could have access to a cheap supercomputer in the cloud, and one that automatically upgrades itself every couple of years? One that taps into existing unused processing power that doesn’t require a new ginormous datacenter to be constructed? This is the idea behind Devin Elliot’s startup called Unoceros.com.

I was skeptical when I first heard him talking about it. This is because he borrows processing time on millions of cellphones at night. Think this through for a moment: these phones are charging, often connected to your home Wifi network, and they are sitting completely idle next to your bed. Why not put them to a good purpose? Think of SETI@Home only instead of searching for intelligent life in space, it is being used for running intelligent apps here on planet Earth.

I mean, the puny cellphone? Can’t we find a better collection of processors? Turns out that while we were sleeping, all that CPU power can add up to quite a few petaflops of processing. If you have a couple million cellphones, you can construct a distributed supercomputer that can rival some of those that are on the top500.org list. Today’s modern phone has the processing equivalent of a medium Amazon Web Services instance. That is far from puny.

I have been fascinated with this topic for some time ever since I participated in a rather unique “flash mob” computing experiment about ten years ago in San Francisco. This was the idea behind a course offered at University of San Francisco and taught by scientist Pat Miller, who works full-time at the Lawrence Livermore Labs. Call it Bring Your Own Laptop. One of the participants was Gordon Bell, who was the father of the VAX while he worked at DEC and now at Microsoft. I was one of hundreds of volunteers and left two laptops of my own for the weekend while the class tried to knit them all together to run the usual benchmarks to prove we had created a supercomputer.

While this flash mob failed at assembling a top supercomputer, they were able to get several hundred machines to work together. But that was ten years ago. Now we have the cloud and efforts like CycleComputing,com to build more powerful distributed processors.

Anyway, back to Unoceros. They have developed some software that can be included inside a regular cellphone app that, with your permission, makes use of your idle time to become a distributed compute engine for those developers that are looking for spare cycles. They are working out the kinks now, figuring out how to distribute the load and make sure that bad actors don’t harness their network for evil purposes.There is also the not-so-small issue about who pays whom and how that aren’t trivial either.

Could it work? Perhaps. It isn’t as crazy as having hundreds of people carrying their gear into a university gym one weekend.

St. Louis Trains Hundreds of Coders

jimck“Have you ever had the opportunity to work with someone who is the best in the world?’ That question got at the heart of a presentation from Jim McKelvey last night at a rather unusual event that I attended at our newly renovated central library downtown. I’ll get to Jim in a moment, but first I want to tell you the context of the event.

Here in St. Louis, like many areas of the world, we have a coding shortage. There are dozens of companies, some big and some just getting started, that can’t hire good programmers. It isn’t from lack of trying, or resources: they have the money, the open positions, and the need. The problem in the past has been explained that either they can’t find them or don’t know where to look. But there is a third possibility: the coders exist, they just need some training to get started. That is where an effort called LaunchCode comes into play.

For the past several weeks, hundreds of folks have been taking the beginning computer science programming class, CS50 that Harvard offers over the EdX online platform. The class started with more than a thousand participants and is now down to about 300 or so hardy souls who spend anywhere from 20 hours per week or more learning how to code. Each week they gather in our library to listen to the lectures and work together on the various programming problem sets.

David Malan, who went to Harvard himself and is a rockstar teacher, teaches the course. I watched a couple of his lectures and found them interesting and engaging, even when he covers some basic concepts that I have long known. If I had him teaching me programming back in the day, I might have stuck with it and become a coder myself.

The CS50.tv collection online is pretty amazingly complete: there are scans of the handouts, quizzes, problem tests, additional readings, supplemental lectures and so forth. The courseware is very solidly organized and designed and very impressive, from my short time spend looking around.

But here is the problem: while the online class is fantastic, only one percent of the people who take the class complete it satisfactorily. That is almost a mirror image of the completion rate for those attending in-person at the Harvard campus, where 99% of the students finish. I was surprised at those numbers, because Malan goes quickly through his lectures. You have to stop and rewind them frequently to catch what he is doing.

This is where LaunchCode comes into play. The operation, which is an all-volunteer effort, is trying to short-circuit the coder hiring process by pairing the students who complete the course with experienced programmers in one of more than a 100 target tech companies who are looking for talent. They think of what they are doing as going around the traditional HR process and building a solid local talent pool. It is a great idea. I spoke to a few students, many of who come from technical backgrounds but who don’t have current coding experience. They are finding the class challenging but doable.

LaunchCode is also supplementing the CS50 lectures and online courseware with meatspace assistance. They have space reserved downtown for the students to get together and help each other. Some students have actually moved to St. Louis so they could take the class here: that was pretty amazing! LaunchCode has created mailing lists and Reddit forums where students can share ideas. But that isn’t enough, and last night we learned that Malan is coming to town in a few weeks, bring a dozen of his teaching assistants with him for a special evening hackathon for the class participants. Wow. Will that help get more students to finish the class? I hope so, because I want Malan & Co. to make a regular trip here to see the next class, and the next.

The problem with teaching programming is that you have to just do it to become good at it. No amount of academic study is going to help you understand how to parse algorithms, debug your code, figure out what pieces of the puzzle you need and how to organize them in such a way to make more efficient code. You just have to go do “build something” as McKelvey told us all last night.

Back to his question posed at the top of my post. Obviously, he thinks Malan is the best programming teacher in the world. He challenged everyone in the auditorium to think about what questions they would ask Malan when he comes into town, and how they can leverage their time with the master. He used the analogy of when he built his glassblowing studio he was able to spend time with Lino Talgiapietra, a master Venetian glassblower. Last night he once again told the story of how humbling an experience that was and how he was allowed to only ask a single question of the “maestro.” Wow.

McKelvey was very gracious with his time, and answered lots of questions from the LaunchCode students. Many of the questions last night were how the students were going to position themselves to get a coding job once the class was over in a few weeks. McKelvey kept emphasizing that they need to just “rock the class” and not worry about whether they were going to be programming in php or Ruby. “That isn’t important,” he kept saying: just demonstrate to Malan that they could write the best possible code when he comes here in a few weeks.

I have heard McKelvey speak before and last night he was in fine form. Will LaunchCode succeed at seeding lots of beginning coders? Only time will tell. But my hat is off to them for trying an very unconventional approach, and I hope it works.

Comments always welcome here:

FedTech: 5 tips for minimizing network latency

As agencies’ networks widen their reach through the adoption of cloud and mobile technologies, one of their biggest problems is in keeping overall network latencies low. Sure, overall bandwidth is important, but high latencies can destroy application usability, make streaming videos unwatchable, and can dramatically increase user frustrations. For example, every drop of 20ms of network latency will result in a 7-15% decrease in page load times.

Here are some tips to manage latency and keep it under control from FedTech Magazine‘s winter edition.

Network World: Virtual machine security still a work in progress

Trying to protect your expanding virtual machine (VM) empire will require a security product that can enforce policies, prevent VMs from being terminated or infected, and deliver the virtual equivalents of firewalls, IPS and anti-virus solutions.

CaptureWe last looked at this product category nearly three years ago, testing five products. At that time, we said that no single product delivered all the features we desired. That’s still true today even though the market matured some. This time around we tested three vendors who were in our previous test — Catbird, Hytrust and Trend Micro – plus a newcomer, Dome9. All represent solid approaches to improving your VM security, but coming from different places.

Sadly, I wasn’t able to test lots of other VM security technologies, which I have listed here.

You can read my review of these VM security products for Network World here. And you can view a series of screenshots of the four products here.