Category Archives: Web software

Modern Infrastructure: The promise of SDN

Posted on by
Reply

Software defined networks are seemingly everywhere these days, offering the promise of having a virtual network infrastructure that can be provisioned as easily as spinning up a new virtual server or storage network. But SDNs are also hard to find outside of a few marquee customers who have dedicated lots of operational resources to set them up and manage them.

In my story for Techtarget’s Modern Infrastructure ezine, I look at the history of SDN, where things stand today, some of the bigger obstacles and how you can begin to plan for them in your own data center.

Slashdot: Segregate your data owners by personae

Posted on by
Reply

 

Positing particular personae (say that slowly) isn’t something new when it comes to website design: The FutureNow guys have been doing it for more than five years, and there are a number of other content engagement “experts” that have their own ways at better segmenting and understanding your ultimate audience. The process of using particular personae can be a way to develop websites that can deliver higher click-through rates and improved customer experience. All well and good, but what about improving the internal data access experience too?

That was the subject of a session at the Teradata Users Conference in Washington DC in October. I heard about how you can use personae to segregate and better target your data owners and data users. It is an intriguing concept, and one worth more exploration.

(An example of virtual data marts at eBay, more explanation below.)

slash

The session was led by Gayatri Patel, who works in the Analytics Platform Delivery team at eBay and has been around the tech industry for many years. There aren’t too many places that have as much data as eBay has: each day they create 50 TB’s worth and they have more than 100 PB per day that is streamed back and forth from their servers. That is a lot of collectibles being traded at any given point. And something that I didn’t really understand before: eBay is a lot more than a marketplace. They have developed a large collection of their own mobile apps that are specific for buying cars, or fashion items, or concert tickets for their specific audiences. In the past they have had difficulties in trusting their data, because two different metrics would come up with different numbers for the same process, so that often meetings would be consumed with different groups presenting conflicting views on what was actually going on across their network.

Patel has come up with mechanisms to focus her team’s energies on particular use cases to better understand how they consume data, and to supply her end users with the right tools for their particular jobs. To get there, she has worked hard to develop a data-driven culture at eBay, to identify the data decision-makers and how to help them become more productive with the right kinds of data delivered at the right time to the right person.

Let’s look at how she partitions her company of data heavyweights:

  1. First are the business executives who are looking at top-line health and metrics of their particular units and have relatively simple needs. They want to drill down deeper to particular areas or create operational metrics and get more narrow and focused areas of particular data sets. Let’s say they want to see how weather-caused shipping delays from sellers are impacting their business. These folks need dashboards and portals that are one-stop shops where you can see everything at a glance, post your comments and share your thoughts quickly with your business unit team. Patel and her group created personal pages with a “DataHub” portal called Harmony, that makes sure all of their metrics are current and correct, and where the executives can bookmark particular graphs and share them with others.
  1. Second are product managers who are looking to learn more about their customers, and want to do more modeling and find the right algorithms to improve their marketplace experience. “We followed some of our managers around, attended their meetings and tried to understand how they use and don’t use data,” Patel said. Her team came up with what they call the “happy path” or what others have called the “golden path” – the walk that someone takes during their daily job to find the particular dataset and report that will help them do their job and make the best decisions. “Each product team has a slightly different path in how they interact with their data,” she said. “Our search development teams are more technical and data-savvy than the teams who work on eBay Motors, for example.” Her team has to constantly refine their algorithms to make the happy paths more evident and useful and well, happier for this group of users.
  1. Third are data researchers and data scientists. These folks want to go deep and understand how everything fits together, and are looking to make new discoveries about particular eBay data patterns. They want more analysis and are constantly creating ad hoc reports. Patel wanted to make this group more self-sufficient so they can concentrate on finding these new data relationships. Her team created better testing strategies, what she called “Test and Learn,” which has a collection of short behavioral tests that can be quickly deployed, as well as more longitudinal tests that can take place over the many days or weeks of a particular auction item on eBay. “We want to fail fast and early,” she said, which is in vogue now but still is something to consider when building the right data access programs. Patel and her team have developed a centralized testing platform to make it easier to track company-wide testing activities and implement best practices.
  1. Next is your product and engineering teams. They do prototypes of new services and want to measure their results. These teams are creating their own analytics and constantly changing their metrics using methods that aren’t yet in production. For this group, Patel made it easier for anyone to create a “virtual data mart” which can be setup within a few minutes, so that each engineer can build their own apps and create specific views pertinent to their own needs. (A sample screen is shown above.)

eBay has three different enterprise data efforts to help support all of these different kinds of data users. They have a traditional data warehouse on Teradata, three of them in fact. They have a fourth warehouse which is semi-structured and called “singularity” that has more behavioral data for example. Finally, they use Hadoop for unstructured Java and C programs to access. The sizes of these things is staggering: Each of the traditional data warehouses is 8 TB and the other two are 42 and 50 PB respectively.

As you can see, the eBay data landscape is a rich and complex one with a lot of different moving parts and specific large-scale implementations that meet a wide variety of needs. I liked the way that Patel is viewing her data universe, and having these different personae is a great way to set her team’s focus on what kinds of data products they need to deliver for each particular group of users. You may want to try her exercise and see if it works for you, too.

Network World: Web-based conferencing comes of age

Posted on by
Reply

As more people telecommute, having a reliable way to connect via desktop video conferencing takes on greater importance. And for employees working in the office, Web-based meetings are a less expensive and less time consuming alternative to business travel.

Web-based conferencing services aren’t new, but they have been getting better, easier to use and less expensive. The options range from one-on-one desktop screen sharing to group video chats to large-scale presentations such as Webinars or “virtual conferences.”

We looked at eight desktop conferencing services, a mix of market leaders and newcomers, including Adobe Connect, Cisco Webex, Citrix GotoMeeting, InterCall Unifed Meeting (in beta with v5), LogMeIn Join.Me Pro, Microsoft Lync 2013 (in beta, and part of Office 365), Skype Premium (now owned by Microsoft), and Vyew.com Professional. Connect and Webex come out on top.

You can read my complete review in Network World here.

How Liberty Mutual built their first mobile app with Mendix

Posted on by
Reply

One of the largest insurers in the US was looking to roll out a new mobile app for its group insurance customers. Chris Woodman, an IT manager at the firm, described at Mendix World the process they went through and how Mendix was a key element to their success.

“In 2011, we wanted to develop a mobile app, but we didn’t know what we were getting into, and we had no previous mobile development experience,” he said. “Two months later we had our app deployed.” Mendix awarded the project as the outstanding effort of the year at the conference.

You can read more of my report on Liberty Mutual’s efforts from the Mendix blog here.

There are other entries that I authored during the show, and here are their links. Mendix definitely has an interesting story to tell. Here are the original stories that I filed and since then taken off their blog.

  • How fast can you deploy your apps?
  • John Rymer from Forrester describes his favorite mobile apps
  • Wrap of the first day at the confrence
  • Ron Tolido of Cap Gemini Europe spoke about whether your company has a business prevention department
  • The student programming competition
  • Wrap of the second day of the conference

Hanging with the kids at the Microsoft Imagine Cup

Posted on by
4

For the past several days I have been in Sydney as one of the judges in Microsoft’s annual student software contest called the Imagine Cup. This is the tenth year of the event that brings together several hundred mostly college students from around the world. I got to see dozens of presentations and talk to dozens of other geeky kids. To say that I was in my element is an understatement. It gives me lots of hope for our youth.

The kids had to write apps that were around a theme of “imagine a world where technology solves the world’s toughest problems” and they didn’t shy away from tackling many of them head-on. There were more than five different apps to try to help blind people navigate their neighborhoods, and other apps dealt with poverty, hunger, health, and the environment. Each team had to prepare a pitch video for the initial judging round and then do an in-person presentation and demo of their technology.

This year’s competition was a fifth female contestants, and I got to see several all-women teams from Oman (pictured here), Qatar and Ecuador. That gives me a lot of hope: back when I was in engineering school, the women could be counted on one hand.

Some of the projects were very elaborate, using Kinect sensor data being fed to some cloud-based service and being controlled from a smartphone. Others were fairly traditional software projects. Of course, Microsoft encourages the teams to use the broadest possible selection of its own software tools, and there are different contests for general software development and gaming and phone-based projects. The gaming judges had a tough assignment: they actually had to spend time playing the games. I had to settle for Powerpoint and demos with my teams.

Part of my job as a judge was to make sure that the demos actually were working code. For those of you that have ever demo’ed something to me, you know I like to kick the tires and pull wires to make sure that the stuff is real. I was very pleased with one team, when I noticed they were running it from a local Web server, brought up their code to show me that they had done the work, they just didn’t trust the local Internet connection to give them the bandwidth they needed. That was delicious.

Many of the teams didn’t really understand the business context judging requirement: talking to old hands who have been to prior contests I found out that the Imagine Cup rules see-saw back and forth between technical and business achievements. But then others got some of the business flavor almost instinctively, as you see in this photo of the Omani all-girls team. The red head scarves are coordinated with the logo on their shirts and their logos on their slide deck: all had to do with their blood bank software. I daresay there are few established corporations that could match that level of polish.

And given that kids came from all over the world, we got to listen to some very heavy accented English. I could tell that the best teams had their techie lead speak English. Those that relied on having a marketing person as the “face” (or better yet, voice) of the team had problems when it came time to answer the judges’ technical questions and had to lose time in translating them into the native language and then explaining the answers to us.

You could also see that the countries that have a longer history of educating their kids in English were doing better: Singapore, the Middle East, and parts of Asia. Given that the contest is held in English, this is no surprise. What I found interesting is that almost all of the engineering schools around the world have English classes. Even in China and Germany: one professor told me that “we have to be competitive and English is the universal language of software.”

Here are some pictures from Sydney.

When does your workday end?

Posted on by
Reply


An interesting study for Mozy, the online backup folks, caught my attention today. They asked 500 employers and employees in various countries around the world a series of questions about their working day. The study and results can be found here (there are tabs along the left hand side that exposes more content).

For example, on average an employee can show up at work about 30 minutes late and not be “written up” as we used to say back in the day. Mozy concludes that is because people are working from home on average about 45 minutes before they come into the office. Not surprisingly given their Teutonic heritage, German bosses were most likely to demand absolute punctuality with 40 percent of the respondents in the survey requiring on-the-dot attendance. Yet Germans and Americans both were close to an hour of early morning work time before they even hit their offices.

Interesting, 60 percent of workers check their email before 8:30 am in the morning and 60 percent check it after 6:30pm in the evening. About a fifth of the respondents check their emails before 7am. Not surprisingly, the French check theirs the latest compared to other nationalities, and the Americans are online nearly 12 hours each workday.

Certainly, email has been extending the workday almost since it was invented decades ago. What is interesting about the Mozy study is how the Internet has permeated the first world fabric of worker bees and bosses alike. And while there were some differences between the various countries that participated in the survey, there was surprising agreement between employee and employer about work styles. One thing that was interesting was workers underestimated the amount of flexibility employers are willing to give in terms of start times. Employers are also underestimating the amount of time that workers spend completing tasks in their own time.

I’ve seen cases where people who wanted to work from home were penalized as slackers, or outright prohibited from doing so. For the most part, thankfully that is a thing of the past as bosses have realized that workers are being more productive when they are out of the office. But it looks like flex hours are here to stay and growing faster and becoming more accepted in more places. And that is certainly a good thing.

Tell your children to learn Hadoop

Posted on by
Reply

I spent some time last week with several vendors and users of Hadoop, the formless data repository that is the current favorite of many dot coms and the darling of the data nerds. It was instructive. Moms and Dads, tell your kids to start learning this technology now. The younger the better.

I still know relatively little about the Hadoop ecosystem, but it is a big tent and getting bigger. To grok it, you have to cast aside several long-held tech assumptions. First, that you know what you are looking for when you build your databases: Hadoop encourages pack rats to store every log entry, every Tweet, every Web transaction, and other Internet flotsam and jetsam. The hope is that one day some user will come with a question that can’t be answered in any way other than to comb through this morass. Who needs to spend months on requirements documents and data dictionaries when we can just shovel our data into a hard drive somewhere? Turns out, a lot of folks.

Think of Hadoop as the ultimate in agile software development: we don’t even know what we are developing at the start of the project, just that we are going to find that proverbial needle in all those zettabytes.

Hadoop also casts aside the notion that we in IT have even the slightest smidgen of control over our “mission critical” infrastructure. It also casts aside that we turn to open source code when we have reached a commodity product class that can support a rich collection of developers. That we need solid n.1 versions after the n.0 release has been debugged and straightened out. Versions which are offered by largish vendors who have inked deals with thousands of customers.

No, no, no and no. The IT crowd isn’t necessarily leading the Hadooping of our networks. Departmental analysts can get their own datasets up and running, although you really need skilled folks who have a handle on the dozen or so helper technologies to really make Hadoop truly useful. And Hadoop is anything but a commodity: there are at least eight different distributions with varying degrees of support and add-ons, including ones from its originators at Yahoo. And the current version? Try something like 0.2. Maybe this is an artifact of the open source movement which loves those decimal points in their release versions. Another company has released its 1.0 version last week, and they have been at it for several years.

And customers? Some of the major Hadoop purveyors have dozens, in some cases close to triple digits. Not exactly impressive, until you run down the list. Yahoo (which began the whole shebang as a way to help its now forlorn search engine) has the largest Hadoop cluster around at more than 42,000 nodes. And I met someone else who has a mere 30-node cluster: he was confident by this time next year he would be storing a petabyte on several hundred nodes. That’s a thousand terabytes, for those that aren’t used to thinking of that part of the metric system. Netflix already has a petabyte of data on their Hadoop cluster, which they run on Amazon’s Web Services. And Twitter, Facebook, eBay and other titans and dot com darlings have similarly large Hadoop installations.

Three years ago I would have told you to teach your kids WordPress, but that seems passé, even quaint now. Now even grade schoollers can set up their own blogs and websites without knowing much code at all, and those who are sufficiently motivated can learn Perl and PHP online. But Hadoop clearly has captured the zeitgeist, or at least a lot of our data, and it poised to gather more of it as time goes on. Lots of firms are hiring too, and the demand is only growing. (James Kobielus, now with IBM, goes into more detail here.)

Cloudera has some great resources to get you started from knowing nothing about it: they claim 12,000 people have watched or participated in their training sessions. You can start your engines here.

What ever happened to Intranets?

Posted on by
Reply

Back in the mid-1990s when the Web was young, we had corporate Intranets popping up all over the place. These were typically internal projects that were used to disseminate information to employees about projects, products, and customers. They were quick and dirty efforts that often involved off-the-shelf parts and little (if any) programming. The idea was to produce a corporate Web portal that was just for internal use, to enable staff to share documents, best practices, customer information and the like.

But they are mostly historical artifacts now. What happened? Well, for one thing, TCP/IP happened. Back in the mid-90s, corporate networks were a hodge-podge of protocols, including SNA and Netware. No one talks about these anymore. Having an all-IP network made it easier to adopt more Internet-native technologies. Remember when sending emails from one company to another was a chore and not always successful? Now we take it for granted that we can communicate with anyone.

Secondly, the tool sets got better. Many companies migrated their Intranets to Wikis or WordPress when it became clear that these products were easier to maintain and use. And then a whole class of products now called enterprise social networks arrived which have ready-made discussion groups, microblogs, news streams, and social media. For example, you can share files with comments attached to them, such as if a team is collaborating on a presentation slide deck. Or use them for customer support actions. Or tracking competitors. All the things that we once used Intranets for.

Then Twitter took off, and many of these products modeled their user interface on the simple 140-character “what are you doing now” dialog box. That made it dirt simple to add content and for a work team to collaborate together.

The final nail in the Intranet coffin may be the announcement this week from Socialcast. They are offering a fully-featured version for free and forever for up to 50 seats of its software. Expect that others will jump on board this model.

These enterprise social networking tools mean more than a “Like” button on a particular page of content: it is a way to curate and disseminate that content quickly and easily. Let’s look at a few of the distinguishing features for this class of products.
• Team workspace. You can segregate your work teams by project and have all the materials for that project in a single place for easy access. These spaces can be persistent to serve as an archival record for completed projects, too.
• Activity stream. The Twitter-like stream is useful to keep track of what your colleagues are doing in any given day.
• Presence detection. Like corporate Instant Message tools, you can keep track of when your co-workers are in the office or ask them quick questions via text or video chats.
• Document collaboration. You can edit documents in real-time to shape a particular deliverable for a client without having to do serial emails.
• External services connections. Many of these products can search and interact with CRM systems, SharePoint servers, Salesforce, emails, and other external services.
• Mobile clients. Most products have specialized clients that have been optimized for iOS and Android phones.
• Public or private deployments. You can start with a public cloud deployment of the product to try out, and then move your system to your own server behind a firewall for the ultimate security.

So say goodbye to Intranets. It was nice to know them. Certainly, the new breed of social network products makes it easier to communicate and collaborate. Now we just have to use them.

HP Input/Output: 6 Ways to (Discreetly) Share Your Photos Online

Posted on by
Reply

To be sure, we live in an over-sharing world. On Facebook, millions of questionable party photos are shared daily, documenting things that many of us later regret. But what if you want some discretion, such as when you have just returned from a corporate retreat or some other event? You want to take the output from your group of several amateur photographers and just share this among the group. The challenge is that you want to keep them private to the participants and not plaster them all over the Internets. What to do?

Consider that your requirements are to satisfy the ultra-paranoid in the group and also something that is dirt simple to use. You don’t want everyone to join a new social network: most of us have too many logins already. This means most of the microblogging sites are out. And you don’t want to have to worry that someone will click on the wrong button and share the entire photo collection with the universe inadvertently.

Facebook is probably the first site that comes to mind for sharing photos. But trying to stay on top of its ever-shifting privacy controls is vexing, and besides, it is almost too easy for one of your group members to inadvertently share a photo that you would rather not have in general circulation. So that’s out.

What about LinkedIn? It is the go-to social network for business purposes. But alas, you can’t really upload any content that isn’t already out on the Internet somewhere. Too bad, because it has some very solid privacy controls in place. The other downside of LinkedIn is that getting one of its groups setup is an exercise in patience. I call it “triple opt-in.” For one of my groups of about 60 people, it has taken the better part of six months to get everyone to become a member.

And some of the same goes for other social networking sites, including Google+. While you can set up specific groups of users called “circles” in Google+ that will limit who gets to view the pictures, one mistake and your work is on display for the whole world or inside Google’s indexing maw. You want something that can set up discrete privacy controls for your group and not have to worry about it if you don’t check all the right boxes. The photo sharing part comes from Picasa, which once was a great photo sharing site, but now has been tricked out with all sorts of Google+ tagging baggage.

You could start a private mailing list group and upload your photos there. For example, Yahoo Groups is a free service and has the ability to support uploaded photos. But starting up a mailing list can be cumbersome, and might be overkill for your purposes.

Then there are dozens of file sharing sites such as Box.net and Evernote that make it easy to share general files in the cloud. These will keep your information private, to be sure. But you really want something that is designed around uploading and sharing images.

Fortunately, there are many photo sharing services out there designed for this purpose. We looked at six of them:

  • Shutterfly.com
  • Photobucket.com
  • Flickr.com (now part of Yahoo)
  • Zangzing.com
  • SmugMug.com
  • Posterous Spaces (now part of Twitter)

None of these services is perfect. They fall into two broad categories: those that have better privacy controls or those that are easier to use. Let’s look at our requirements in more detail. First, you want a service that can create a private space that doesn’t appear on search engines or that any random user can find. Photobucket and Shutterfly both do this, by setting up a special URL (Photobucket.com/groupname or Groupname.shutterfly.com) for your group. For Photobucket, for example, you have three choices for each album’s privacy controls: everyone can see them, no one else can see them, or you can password protect them by invitation only. The latter is what we want to use and you can set up an album password so that only those folks who know the password can see and download the photos. (See the screenshot below.) Shutterfly has similar options with its share sites option.

The problem for both Photobucket and Shutterfly is that you need to become a member to upload any photos: that is fine if you just have a few shutterbugs in your group, but if you have lots of sources of images, it can become cumbersome.

SmugMug has lots of granular security controls for its service. You can set up your photos on a special site username.smugmug.com for example. But there is a lot more: You can prevent Google from finding your photos with one mouse click, and add watermarks with another. You can add password-protection to your albums, and even provide another password for guests to upload photos to a particular album, so they don’t have to join the service to share their work. Here is a screenshot of its many security controls:

Flickr has URLs for groups, such as http://www.flickr.com/groups/groupname. But Yahoo really wants you to sign up to its service, and you will need to do so if you want to post any photos. Flickr has a guest pass option but it is somewhat clunky. Also, if you are using Flickr, make sure you have turned off its autoposting/notification features if you want to keep your photos from showing up in your Facebook timeline or other social places.

Zangzing.com is better at ease of use but it comes at a cost. You can set up individual albums that have their own URLs, such as http://www.zangzing.com/username/albumname. There is no password required so anyone who knows the URL can access the entire album. And if you want to upload pictures, you will need to join. You can also email pictures to albumname@zangzing.com and they will be automatically posted to the album, which is a nice feature.

Finally, Posterous is more of a blogging site than a photo collection, but it can be used for sharing photos, as well. Indeed, if you want to mix your photos with other business content, Posterous could be a good choice and could serve as the base for a simple low-end Web presence. Groups of photos can have their own URLs, but you do need to become a member to post content. You can also email your photos and have them posted to your site, like what Zangzing does.

There are lots of other photo sharing services, including Instagram.com, Klip.com, Twitpic.com and Pixable.com. Most of them aren’t focused on privacy but in making it easier to share photos across the universe.

Recommendations: Start with SmugMug

We recommend you start with SmugMug, especially if you require the simplicity of a shareable URL and don’t want to mess with having each person sign up for the service. If you need the additional security that a membership site offers, then look at Photobucket. It has more granularity for the security options than Shutterfly. If all the bells and controls of SmugMug are daunting, then take a closer look at ZangZing. Steer clear of Flickr: Its interface is somewhat long in the tooth, and it is too easy to click on the wrong button and end up sharing your entire photo collection to Facebook or Twitter. If you have more confidence in your users’ abilities, you can set up private groups in Facebook or Google+. Finally, if you have yet to join the blogosphere and want something simple to set up that will include a lot of photos and other illustrations, then Posterous is where you should start.