New Relic blog: JQuery Foundation’s Dave Methvin Shares his Rules of the Road for Speeding Up Your Website

It’s practically a universal truth: just about every website is too slow. There are many, many reasons for this unfortunate fact, of course, but Dave Methvin, president of the jQuery Foundation, has some practical advice on how to get the biggest speed improvements with the least amount of effort. This is based on an in-person meeting as well as some of his talk that he gave at a programming conference earlier this year.

You can find my article on the New Relic blog here.

Why New Jersey needs to slow down its traffic

It is ironic that the same thing that got Chris Christy in trouble – delaying traffic into Manhattan — is being used by others to build a multi-million business. I am talking about network traffic for stock traders. Perhaps you have seen the stories about them based on Michael Lewis’ latest book Flash Boys.

In the celebrated George Washington “bridgegate” fracas, it was delays introduced by closing various on-ramps to the bridge. In the case of the stock market, it was delays of 350 microseconds that made high-frequency trades more equitable across exchanges.

Wait a minute, come again? Adding in latency is a good thing? Yes, that is another irony of the situation. A company called IEX developed a technique where they slow down the trades so that the exchanges can get the trading requests at the same time. This means that exchanges closer (in terms of connection time) to the Big Apple trading desks can’t trade a few microseconds ahead of the others. The technique IEX developed is basically a big spool of fiber optic cable that is 35 miles long (pictured above), the length it takes a beam of light to traverse 350 microseconds. The problem is that the biggest trading exchanges are located at different peering points in New Jersey, with some being up to 35 miles apart from others, at least in terms of how their packets are routed. Another irony: these locations were chosen so that the exchanges would have as little latency as possible to gain access to the trading data streams (see the map from the New York Times below).

It is probably the first time that anyone was deliberately introducing more network latency to improve their business that I have heard of. Many of us have spent a good chunk of our careers trying to cut down on latency issues: I can remember when I rolled out the first PC local area network application at MegaLith Insurance back in the middle 1980s. The file transfer app that normally took a few seconds to get from one point to another across our mainframe network now took tens of minutes. That wasn’t a good thing.

Programming professor Donald Knuth wrote in one of his seminal books how “programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs.” (Thanks to Dave Methvin of jQuery Foundation for uncovering this quote.) And in the world of high-frequency trading, a company called Spread Networks invested hundreds of millions of dollars building a fiber connection from Chicago to NYC a few years ago, only to find that a microwave network could shave even more time off their latency figures.

So we have the Internet to thank for introducing all kinds of unpredictable latencies into our apps and drive us nuts trying to track down the culprit. Now lower latency has been productized, thanks to the smart guys at IEX. Maybe the next step will be for the traders to actually colocate their offices on top of the major peering points themselves: that could actually work for cutting down on the times of some New Jersey commuters, too.

St. Louis Trains Hundreds of Coders

jimck“Have you ever had the opportunity to work with someone who is the best in the world?’ That question got at the heart of a presentation from Jim McKelvey last night at a rather unusual event that I attended at our newly renovated central library downtown. I’ll get to Jim in a moment, but first I want to tell you the context of the event.

Here in St. Louis, like many areas of the world, we have a coding shortage. There are dozens of companies, some big and some just getting started, that can’t hire good programmers. It isn’t from lack of trying, or resources: they have the money, the open positions, and the need. The problem in the past has been explained that either they can’t find them or don’t know where to look. But there is a third possibility: the coders exist, they just need some training to get started. That is where an effort called LaunchCode comes into play.

For the past several weeks, hundreds of folks have been taking the beginning computer science programming class, CS50 that Harvard offers over the EdX online platform. The class started with more than a thousand participants and is now down to about 300 or so hardy souls who spend anywhere from 20 hours per week or more learning how to code. Each week they gather in our library to listen to the lectures and work together on the various programming problem sets.

David Malan, who went to Harvard himself and is a rockstar teacher, teaches the course. I watched a couple of his lectures and found them interesting and engaging, even when he covers some basic concepts that I have long known. If I had him teaching me programming back in the day, I might have stuck with it and become a coder myself.

The CS50.tv collection online is pretty amazingly complete: there are scans of the handouts, quizzes, problem tests, additional readings, supplemental lectures and so forth. The courseware is very solidly organized and designed and very impressive, from my short time spend looking around.

But here is the problem: while the online class is fantastic, only one percent of the people who take the class complete it satisfactorily. That is almost a mirror image of the completion rate for those attending in-person at the Harvard campus, where 99% of the students finish. I was surprised at those numbers, because Malan goes quickly through his lectures. You have to stop and rewind them frequently to catch what he is doing.

This is where LaunchCode comes into play. The operation, which is an all-volunteer effort, is trying to short-circuit the coder hiring process by pairing the students who complete the course with experienced programmers in one of more than a 100 target tech companies who are looking for talent. They think of what they are doing as going around the traditional HR process and building a solid local talent pool. It is a great idea. I spoke to a few students, many of who come from technical backgrounds but who don’t have current coding experience. They are finding the class challenging but doable.

LaunchCode is also supplementing the CS50 lectures and online courseware with meatspace assistance. They have space reserved downtown for the students to get together and help each other. Some students have actually moved to St. Louis so they could take the class here: that was pretty amazing! LaunchCode has created mailing lists and Reddit forums where students can share ideas. But that isn’t enough, and last night we learned that Malan is coming to town in a few weeks, bring a dozen of his teaching assistants with him for a special evening hackathon for the class participants. Wow. Will that help get more students to finish the class? I hope so, because I want Malan & Co. to make a regular trip here to see the next class, and the next.

The problem with teaching programming is that you have to just do it to become good at it. No amount of academic study is going to help you understand how to parse algorithms, debug your code, figure out what pieces of the puzzle you need and how to organize them in such a way to make more efficient code. You just have to go do “build something” as McKelvey told us all last night.

Back to his question posed at the top of my post. Obviously, he thinks Malan is the best programming teacher in the world. He challenged everyone in the auditorium to think about what questions they would ask Malan when he comes into town, and how they can leverage their time with the master. He used the analogy of when he built his glassblowing studio he was able to spend time with Lino Talgiapietra, a master Venetian glassblower. Last night he once again told the story of how humbling an experience that was and how he was allowed to only ask a single question of the “maestro.” Wow.

McKelvey was very gracious with his time, and answered lots of questions from the LaunchCode students. Many of the questions last night were how the students were going to position themselves to get a coding job once the class was over in a few weeks. McKelvey kept emphasizing that they need to just “rock the class” and not worry about whether they were going to be programming in php or Ruby. “That isn’t important,” he kept saying: just demonstrate to Malan that they could write the best possible code when he comes here in a few weeks.

I have heard McKelvey speak before and last night he was in fine form. Will LaunchCode succeed at seeding lots of beginning coders? Only time will tell. But my hat is off to them for trying an very unconventional approach, and I hope it works.

Comments always welcome here:

Stop Web Scraping With ScrapeDefender

Copying content from the Web can be both a good and bad thing. There are companies that make it easy to scrape public data archives such as ScraperWiki.org that are used by data sciences and journalists to track trends and uncover government abuses. And Google and other search engines use various kinds of scraping algorithms to index and categorize your site, and to ensure that your content is ranked appropriately.

But for the most part scraping is bad news. Chances are good that someone has copied your Web content and is hosting it as their own elsewhere online. This happened with LinkedIn not too long agoscrape dashboard2, where someone picked up thousands of personal profiles to use for their own recruiting purposes. That is a scary thought, indeed.

And lest you think this is difficult to do, there are numerous automated scraping tools that make it easy for anyone to collect content from anywhere, including Mozenda and Scapebox. I won’t get into whether it is ethical to use these on a site that you don’t own the content. Some of these attack sites are very clever in how they go about their scraping, with massive numbers of ever-changing IP addresses blocks to obtain their content.

So what can you do to prevent the bad kind of scraping? There are several companies that try to protect your site from being scraped by a bad actor, including Distill Networks and CloudFlare’s ScrapeShield.

But today’s post is to tell you about another one that goes even further than these two tools called ScrapeDefender. You can watch a screencast video that I just produced here that shows its features.

Scrape Defender is easy to get started with: you just plug in your site’s URL and it will take about a day to look at your site and see where you are vulnerable. When I tried it with my own domain strom.com I was surprised to see it listed 150 different exploits. Some of them have pretty oddball names, such as dripping water or shotgun that show where anyone can come in and grab your content. The service provides a piece of Javascript tracking code that you add to each of your site’s page headers. Once this is in place you can monitor what is going in in near-real time and protect your site against these abusers.

For example, you can view how many pages a potential abusive IP address has visited, any geolocation information, which risk metrics were tripped, what alarms were generated because of this activity and other IP addresses that are owned by the same organization. All that information can help you figure out if your site was suddenly very popular or was being targeted by one of your competitors or someone that wants to steal your content. Their service is Web-based; you bring up your browser and can view these metrics and reports, along with suggestions on best security practice to defend your content too.

The hard part about defending and hardening your site against potential scrapers is that it is difficult to distinguish between a legitimate visitor and an automated bot that is collecting your content. That is the secret sauce of ScrapeDefender: they have looked at thousands of websites to figure out when a bad actor is present, and have code these various behaviors into their system.

You can try Scrape Defender for free, the paid service starts at $79 per month to keep track of a single domain, with more expensive and extensive plans available. It is well worth a look.

A Letter from Budapest

IMG_0963I spent this last week in Budapest, thanks to Balabit, a Hungarian security software company, and fell in love with the city and its people. (Here are more of the photos that I took of the town.)

Budapest has an interesting mix of old world charm and new age coding: a vibrant startup scene that is just taking hold. While I estimate that it is four years or so behind where I see things in St. Louis. That isn’t a knock on their innovation or spirit, just more a comment on their available support infrastructure. Hungary isn’t completely in tune with the notion of startups as economic development: there are crazy laws on the books that don’t encourage new businesses and seem to date back to the Soviet era when full lifetime employment was the goal. But that just means their startups are more determined to succeed. Call it Silicon Goulash, perhaps?

Hungary has a fascinating and long history of innovation. Many of its citizens were prominent mathematicians and physicists, including the Intel founder Andy Grove and the grandfather of computing himself John Von Neumann. The carburetor, the transformer, parts of the first telephone exchange, the first synthetic vitamin, the modern CRT, Rubik’s cube and the ballpoint pen all came from the minds of Hungarians.

IMG_1018Speaking of the Soviet era, Hungary was also ahead of its time in offering long-term asylum to political refugees. Cardinal József Mindszenty was the leader of the Catholic Church who was imprisioned and then lived in the US Embassy in Hungary for 15 years before being allowed to flee in 1971. Does this sound familiar?

I found Budapest a very walkable and livable city. It has hundreds of outdoor cafes packing dozens of pedestrian streets; something that we so desperately lack in the States. It has a terrific riverfront on the Danube with bikeways galore and an island park in the middle of the river that has my favorite water park ever. And speaking of water, there are dozens if not hundreds of thermal baths that span centuries. You can see why I enjoyed my time there so much.

Ostensibly, I was there on behalf of Balabit to meet with their executives and see their products and story. The company is behind the open source log aggregator syslog-NG and other security products, and has been in business for more than a decade. But while I was there I also met with other IT firms, including Electool.com, Graphisoft and Metta.io, and also visited with Prezi.com.

Prezi, the alternative SaaS-based presentation delivery tool is one of three software companies to have become an international success, along with LogMeIn and UStream. The three of them originated in Budapest and still have decent-sized development offices there.

Of the three, LogMeIn is a public company, while Prezi and UStream are private. All three have attracted tens of millions of dollars in A-list Silicon Valley venture funding. UStream and LogMeIn both have several offices around the globe in addition to the ones in Budapest and America.

The three firms got together to recently produce the well-attended RAMP conference about how to scale up your software architecture, bringing in experts from around the globe to Budapest.

And founders of the three firms have also put together their own nonprofit called Bridge Budapest to offer fellowships and internships at major software companies to Hungarians, and for the rest of us to come to Budapest and intern at one of their companies.

IMG_1057While I have tried Prezi several times over the years it has been around I never have been able to gain much traction with the tool. Maybe it is just the way I work or my speaking style. But their offices are an interesting twist on the typical tech startup playground: they have recently moved into a Beaux-Arts building with the top floor a former telephone equipment room. It is a fitting place for a modern tech startup.

Tech isn’t the only thing happening in Budapest. I met an agribusiness manager one night. He just moved his family from the Midwest to Budapest. His company does a lot of business in Eastern Europe and Russia, and wanted a stable place to have an office. “To Western Europe, we don’t look all that stable a country,” said another entrepreneur to me. “But to the east, we look rock solid. It is a nice position to be in.” Hungary is in the EU but doesn’t use the Euro: its currency is the Forint which sounds very Princess Bride-like charming to me.

Budapest is also home to the Soros-backed Central European University. It was the first on the continent to offer an American-style MBA several years ago and now has a budding entrepreneurship program, a business incubator and courses that seem quite current with promoting startups and what one could find in the States.

Speaking of schools, if you know someone who is a CS/EE student and is looking to study abroad, take a gander at this program that is offered by the Aquincum Institute of Technology (and where Rubik himself teaches).

I asked several people about the tax situation in Hungary and got a range of responses. One source told me, “our tax and legal system is so wonky,” and complained that half of one’s salary is taken in taxes. Another source said while this is true, the tax rate is less than it has been in recent years, so things are improving.

If you have a chance to visit, I think you will be as excited as I was. And if you would like an introduction to the companies that I mentioned, let me know.

ITWorld: How to choose a social media management service

How do you know you are fully engaged with all of your social networks? This turns out to be a difficult question to answer. And as we try to resolve complaints from customers on Twiter and Facebook, we also need to track mentions across other networks and develop consistent workflows and processes to respond and measure these involvements.

Luckily, there are tools available for these tasks, and you can read my article in ITWorld here that reviews many of the issues involved before purchasing one.

Solution Providers for Retail: Gaming Replacements for Captchas

You are probably just as annoyed as I am when you encounter those cryptic blocks of text called “Captchas” (the acronym stands more or less for Completely Automated Public Turing Test to Tell Computers and Humans Apart). There must be a better mousetrap, particularly for online retailers that are looking to distinguish themselves and cut down on their shoppers’ frustrations as they navigate their sites. A company called PlayThru has one: they embed a small Flash or HTML5-based game that a human plays with a mouse to prove you are really are a carbon-based life form.

Part of the problem is that the bad guys are escalating their own solutions to defeat the Captchas. They pay actual humans a very low wage to enter the text in massive boiler rooms, run optical character recognition software to figure out the codes, or some other machine-based algorithm. All of these approaches have made Captchas more painful and less usable. (Here is a great collection of the worst of them.)There is research that suggests it takes the average person several attempts to successfully complete a Captcha request, and close to 25% of Captchas are solved by bots today.

There must be a better mousetrap, particularly for online retailers that are looking to distinguish themselves and cut down on their shoppers’ frustrations as they navigate their sites. Scientific American covered some of these alternatives here last year.

One that is catching on and could be useful for VARs looking to expand their practice areas is from a company called PlayThru. They embed a small Flash or HTML5-based game that a human plays with a mouse to prove your really are a carbon-based life form. It is intriguing, and has captured (if you will excuse the pun) more than 4,500 supporters in the past nine months already.

The Play Thru concept is pretty clever: you have to interact and identify objects with your mouse and keyboard, or drag and drop particular objects such as pizza toppings or food from a fridge. As you do so, the algorithms monitor your actions and find the tell that you aren’t a bot.

The company serves up 20 million miniature games each month and “the algorithm hasn’t been defeated yet,” said co-founder Reid Tatoris when I spoke to him in late February. “We are constantly looking at how people are interacting with our games and we write our own bots to test them too,” he told me. What made Play Thru’s games work is that the developers tackled the issue as a usability problem first and foremost, and then made sure the security was ironclad. Most of the Captcha deployments were steeped in security and thus the miserable and virtually unusable result that we are saddled with today.

The proof is in the pancakes, so to speak. They have seen conversion rates improve by 40% over the traditional text-based Captchas, which means fewer abandoned shopping carts and more real shoppers who can conclude the shopping process.

PlayThru offers a free plan along with two paid plans, including a white label plan at $79 a month that includes phone support and would make the most sense for VARs looking to implement this technology. There are plug-ins available for WordPress, Jooma and Drupal and code libraries for Php, Perl, Ruby, Python and Java to make installation easier.

Why eWallets still are bad news

I had a chance conversation with one of my neighbors recently where one of them casually mentioned that they have never bought anything online. Ever. That gave both my wife and I some pause. Not that we are big shoppers, on or offline: but we both think of online shopping as a natural extension of what we do, like breathing. We even know people who buy furniture online, which I think a bit risky.

Against this backdrop, there are yet another round of eWallet innovations that are destined to make the mistakes of the past. There are multiple solutions, usually involving your smartphone and a payment provider, such as the recent announcement from Visa and Samsung. Then there was the deal between Starbucks and Square so you could pay for your lattes by tapping your phone near the register at checkout.

A lot of this activity is motivated by having more near field communications on our phones, meaning we don’t need physical contact to conduct a transaction. While that is very sci-fi, it isn’t going to motivate people like my neighbor to start conducting ecommerce. Let’s go to the video tape of the past botched plays.

I dredged up an op/ed piece that I wrote 14 years ago for Computerworld where I concluded, “If you have a Web storefront, steer clear of e-wallets for now. Let your customers pay you as easily and as quickly as possible.” That advice still holds true today.

To set the context for this piece, you need to know that Microsoft had its eWallet software as part of Windows 98, and during the latter part of the 1990s there were probably a dozen or so vendors who were developing Internet payment schemes of one sort or another. Only Paypal survives from this era, and ironically they had their origins as a piece of software that was used on Palm Pilots to beam payment information using their built-in infrared technology. Many of us consider the contact manager on the Palm still better than anything we have today, but that is a fight that I will leave for another day.

So what happened to all those payment companies? They made several mistakes.

First was the chicken-and-egg of non-universal coverage and too many “standards.” I wrote back then: “Imagine going shopping at a physical mall store and getting ready to pay, only to find out that the store accepts one obscure credit card issued by a single bank in Tuvalu. How long do you think that store would stay in business?” Exactly. All these vendors need to get around one solution, and do it quickly. Imagine how long credit cards would have lasted if you needed separate machines to scan your Visa, Mastercard, and Amex at the checkout line.

Second is that credit/debit cards just work too well. We all have them, we all carry them with us at all times, and we all know how to shop with them. Trying to compete with this universal solution is madness. Indeed, they have largely replaced the need for actual cash. I remember when my dad wouldn’t leave home without several hundred dollars in his wallet. Even when I travel, I rarely have more than $20 or $40 in mine, and usually a lot less. Everyone takes plastic nowadays.

Next, I don’t want to manage yet another cache of cash. It is bad enough that Paypal exists, and that I have to track what is in my account and how quickly I can get any dollars in or out of it when I am buying or selling something. Why do I need yet another account to manage?

The last straw is that I usually need a specific piece of software, browser version, or phone. Check out what you need for the Google Wallet: “an NFC enabled Android device with a Secure Element chip running the most recent Android operating system.” That isn’t a very long list of phones, none of which I currently own. We tried this before and the number of variations means that almost always you don’t have the right mix of things to access your eWallet some of the time. See my remark about Tuvalu above. And note the roll call of failed Internet payments companies of the past too.

So our phones may have gotten smarter with all sorts of new protocols and wireless radios, but ultimately the real gating factor in having the carbon life forms suffer through using them, same today as back in 1999. It is not too late to learn from the past.And maybe sometime soon my neighbor will feel confident enough to buy something online.

Social media companies need to practice what they preach

Sometimes, it is those of us in the tech industry who are our own worse examples of actually using the technologies that we have created. Take the example of tools that variously go under the headings of sentiment analysis, social CRM, engagement measurement, social media management, enterprise listening platforms or social media marketing. These things help you figure out when you should Tweet or post, who is most influential among your social networks, and what conversations you should pay attention to. They offer pretty dashboards and real-time data feeds so you can control the social conversations around your brand.

I am starting a project for Network World reviewing these tools. So far, I have found nearly 100 of them, but I can only review 8. But that isn’t the problem. My issue is that I would expect that these vendors would be sterling examples of how to engage their own audiences. Not true, no way, sorry to say.

Example #1. By now, it should be obvious that a software vendor should make it easier for their potential customers if they actually want to purchase their product. So how about putting a phone number on the home page, just in case someone wants to call? Less than half of the vendors do this, or make it so hard to find their contact information. Almost all of them use Web forms that you have to fill out, which is less than satisfying because you have no recourse if you don’t get any follow up. One vendor takes you to a form on their Facebook page, which is interesting but not very helpful.

Example #2. The same should be true for displaying a press contact. Again, less than half of the vendors have this information, or make it so hard to find. Others, such as Google, ignored my emails entirely. Written on one vendor’s press page, I had to laugh: “Hi! We love you, you dashing citizen of the fourth estate. Even though we don’t know you personally yet, I can tell we’re going to get on famously. Can I get you a drink?” Now we are talking! While you don’t have to buy me a beer, it would be nice if the press contact was in plain sight.

Example #3. How easy is it to find these vendors on Twitter? You would think that placing a little bird icon at the top of their home page linking you to their Twitter accounts would be easy. And indeed, most of them (but not all) do include this information somewhere on their sites. One vendor had a broken link that didn’t take them to their Twitter account but someplace else entirely.

But let’s go beyond actually having a link to the ID, and see how engaged they are with their accounts. It is a spotty record, to be sure.

Some vendors have thousands of Tweets and followers, which is what you would expect from people in this space. Hootsuite is the Justin Bieber of social media tracking tools with more than four million followers, and dozens of daily tweets. (He is at 31 million, BTW.) The major vendors in this space, including Google, Salesforce, Oracle and Adobe, also have big followings and lots of tweets.

But when you get beyond the big guns and look around, it is disappointing. Very few of these vendors actually use their own products to track engagement and mentions. I started posting tweets with the vendor Twitter IDs (once I found them), asking them to get in touch with me. A very small number of vendors responded at all. An even smaller number started following me or sent me messages saying they wanted to help my project. How do these vendors expect anyone to use their products if they don’t track their own brands? Hmm.

Many of these are software efforts from marketing companies, or ad agencies, or others who should know better. Or so I thought.

Example #4: Pricing. I have written before about those vendors that don’t want to put pricing information online, but the social media tools that I am looking at really try to obfuscate their pricing. Perhaps because every deal is a custom negotiation, perhaps because they just don’t want you, the customer, to know. In this particular and chaotic market, prices vary all over the place. Some tools are designed for single users while others are geared for large teams. Some have freemium models, others have one-time fees like traditional packaged software.

socialvolt pricing pageThere was one site that had an explicit “Pricing” tab at the top of their home page: I thought, at last! When I clicked on it, I came to a page that had all sorts of details about the various plans they offered, but no dollar signs anywhere to be found (See above).

Gremln is an exception: they actually put their prices right at the bottom of their home page. Kudos to them. (No phone number, though.)

As I said, I am just starting out on this project for Network World. If you have any experience with these products, send me a tweet or an email. And if you want to see my collection of vendors, I have put together a list here.