A new way to do big data with entity resolution

I have this hope that most of you reading this post aren’t criminals, or terrorists. So this might be interesting to you, if you want to know how they think and carry out their business. Their number one technique is called channel separation, the ability to use multiple identities to prevent them from being caught.

Let’s say you want to rob a bank, or blow something up. You use one identity to rent the getaway car. Another to open an account at the bank. And other identities to hire your thugs or whatnot. You get the idea. But in the process of creating all these identities, you aren’t that clever: you leave some bread crumbs or clues that connect them together, as is shown in the diagram below.

This is the idea behind a startup that has just come out of stealth called Senzing. It is the brainchild of Jeff Jonas. The market category for these types of tools is called entity resolution. Jonas told me, “Anytime you can catch criminals is kind of fun. Their primary tradecraft holds true for anyone, from bank robbers up to organized crime groups. No one uses the same name, address, phone when they are on a known list.” But they leave traces that can be correlated together.

Jonas started working on this many years ago at IBM. He is trying to disrupt the entity resolution market and eventually spun out Senzing with his tool. The goal is that you have all this data and you want to link it together, eliminate or find duplicates, or near-duplicates. Take our criminal, who is going to rent a truck, buy fuel oil and fertilizer, and so forth. He does so using the sample identities shown at the bottom of the graphic. Senzing’s software can parse all this data and within a matter of a few minutes, figure out who Bob Smith really is. In effect, they merge all the different channels of information into a single, coherent whole, so you can make better decisions.

Entity resolution is big business. There are more than 50 firms that sell some kind of service based on this, but they offer more of a custom consulting tool that requires a great deal of care and feeding and specialized knowledge. Many companies end up with million-dollar engagements by the time they are done. Jonas is trying to change all that and make it much cheaper to do it. You can run his software on any Mac or Windows desktop, rather than have to put a lot of firepower behind the complex models that many of these consulting firms use.

Who could benefit from his product? Lots of companies. For example, a supply chain risk management vendor can use to scrape data from the web and determine who is making trouble for a global brand. Or environmentalists looking to find frequent corporate polluters. A finservices firm that is trying to find the relationship between employees and suspected insider threats or fraudulent activities. Or child labor lawyers trying to track down frequent miscreants. You get the idea. You know the data is out there in some form, but it isn’t readily or easily parsed. “We had one firm that was investigating Chinese firms that had poor reputations. They got our software and two days later were getting useful results, and a month later could create some actionable reports.” The ideal client? “Someone who has a firm that may be well respected, but no one actually calls” with an engagement, he told me.

Jonas started developing his tool when he was working at IBM several years ago. I interviewed him for ReadWrite and found him fascinating. An early version of his software played an important role in figuring out the young card sharks behind the movie 21 were taking advantage of card counting in several Vegas casinos, and was able to match up their winnings all over town and get the team banned.  Another example is from a Colombia university who saved $80M after finding 250,000 fake students being enrolled.

IBM gets a revenue share from Senzing’s sales, which makes sense. The free downloads are limited in terms of how much data you can parse (10,000 records), and they also sell monthly subscriptions that start at up to $500 for the simplest cases. It will be interesting to see how widely his tool will be used: my guess is that there will be lots of interesting stories to come.

Fixing Facebook’s flaws

Facebook has been under fire for the past several months as Zuck does his World Apology Tour, both in DC and in Belgium giving testimony to the EU Parliament. That link takes you to a YouTube video from The Verge which shows him not answering very pointed questions from the body’s members. The EU format was very different from his US Congressional testimony in April: In Europe, the session was just an hour and a half, with much of that time taken up by Members’ speeches. In the States, he was there for a total of ten hours.  Business Insider called the EU appearance “a wash out.” That difference between the two geographies was noted by lawmakers quoted in Vox. “We are here in terms of regulation,” said Claude Moraes of the British Labour Party, gesturing upward with one hand, “And the United States is here,” gesturing downward with the other.

Sadly, the social media giant has paid lip service in protecting users’ privacy. There is this story in the NY Times about how it cooperated with the major cellphone vendors to give them access to vast amounts of private user data.

And the company hasn’t done very well towards policing its content for terrorist and hate speech. This recent post in the UK’s Independent talks about the effort that the vendor is going to try to block hate speech in Germany. The reporter takes us inside a 1200-person cubicle farm where analysts try to screen content in real time.

But to get a more complete picture, you should read this report last month from the Counter Extremism Project called Spiders of the Caliphate. It lays out a chilling analysis of how poorly Facebook has been in policing pro-ISIS propaganda. It documents how their supporters operate on that network and even leverage its features. ISIS’ online networks are growing and are used to plan and direct various terror attacks as well as to mobilize foreign supporters to fight in various places around the world. ISIS’ Facebook presence is pervasive and well organized. According to the authors, ISIS “has developed a structured and deliberate strategy of using Facebook to radicalize, recruit, support, and terrorize individuals around the world.” They found from careful path analysis that ISIS’ “Facebook networks are strong, extensive, and growing.”

The authors selected a thousand Facebook accounts that they claim are ISIS supporters, using positive language and geolocation to specific areas, usernames with pro-ISIS meaning, accounts from people that claimed they worked at ISIS or are from place names that are under ISIS control. You would expect many of these accounts to originate from the Middle East, but there also were accounts from Nepal, South Korea and South America too: ISIS has truly gone global. There were even American accounts.

They examined each account’s timeline and pattern of liking and sharing posts and then recorded the number of their friends or followers and other data. They then visualized this data using the open source network path analysis tool Gephi. While I am not an expert here, it seems their methodology is sound.

They found many disturbing things. There were 28 accounts that were used exclusively to post pro-ISIS propaganda, with some posts that have remained online for more than a year and racked up thousands of views. Also, “a group of American ISIS supporters holds weekly meetings on Facebook Live to discuss topics ranging from ISIS ideology to how to avoid detection from the FBI.” ISIS supporters live in more than 80 different countries. Most supporters had publicly visible posts, too.

Facebook’s misleading efforts to counteract terrorism

Facebook says they have worked hard to try to stem this pro-ISIS tide, but the CEP report documents how they have mislead the public and been largely ineffective. The report says that Facebook has been unable to do anything “in a manner that is comprehensive, consistent, and transparent.” Rather, it has enabled ISIS supporters to flourish and grow their social networks. Of the 1,000 accounts analyzed, less than half of them had been removed by Facebook by March 2018, and many accounts were reinstated multiple times after removal. “Perhaps most concerning is that Facebook’s suggested friends algorithm reveals how the company’s tools have aided in connecting extremist profiles and help expand ISIS networks.” The report goes further and says that Facebook executives have purposely misled policymakers and the public in terms of their cleansing of their network from pro-ISIS activities.

The post in New Europe was quite disparaging and called Zuck’s non-answers before the EU evasive and a disaster. It mentions his claim that Facebook “can flag 99 percent of the ISIS and al-Qaeda related content that we end up taking down before any person in our community flags that for us.” Clearly, that number (apart from being meaningless) is at odds with the CEP report.

One final personal note about Facebook’s inadequacies.  Two months ago, I tried to download information from Facebook and other Internet sites that they have collected about my usage, and documented the experience in my blog here. It wasn’t an easy exercise, but it was sobering to see how many advertisers had my name in their sights, and in their sites as well. None of the Internet properties make this easy for you to do, but the effort is worthwhile and another eye-opener.

The New Europe post says, “It’s not like Facebook doesn’t have the resources to do better. Facebook’s market capitalization is more than the GDP of Belgium. Until Facebook finally tells the truth, it will be difficult for lawmakers and the public to hold it, and other tech companies, accountable for the level of disturbing and harmful content that proliferates online today.” Finally, I speak to this issue of corporate and leadership integrity on Shel Holtz’ For Immediate Release podcast this week. (Skip to 12:15 if you don’t want to listen to the entire hour.)

CSOonline: The state of the CASB market

In just a few years,a lot has happened in the Cloud Access Security Broker (CASB) market.

Most of the main-line security vendors have purchased CASB solutions: Oracle (Palerra), IBM (Gravitant), Microsoft (Adallom), Forcepoint (Skyfence), Proofpoint (FireLayers), Symantec (Skycure) and McAfee (Skyhigh Networks). The three independent vendors still standing include CipherCloud, Netskope, and Bitglass. The market has matured, although this is a matter of degree since even the longest-running vendors have only been selling products for a few years. It has also evolved to the point where many analysts feel CASB will be just as important in the near future just as firewalls once were back in the day when PCs were being bought by the truckloads. Gartner predicts that by 2020, more enterprises will use CASBs than not, which represents a big jump from the 10% that used them at the end of 2017.

Four things also helped the CASB cause: First was its quick learning curve by security personnel. Second was that they became more inclusive in terms of applications support. Third was the beginnings of a managed service provider business, and finally, multimode operation has become more prevalent. 

In this story for CSOonline, I talk about what are these products, why enterprises are motivated to purchase and deploy them,  what features you should look for that are appropriate for your network. what are your decision points in the purchase process, and links to many of the major CASB vendors.


Why isn’t marketing attracting more college grads? That’s the topic Paul Gillin and I explore this week, starting with the results of a study commissioned by Marketing Week earlier this year which  found that just 3% of undergraduates think marketing offers them the best career opportunities.

The publication held a seminar to try to explore ways to better engage Gen Z, and we have several thoughts on the matter too. Colleges need to have more focused marketing programs, and businesses need to show that a wide range of skills and talents can be put to best use with marketing programs. Certainly there are obstacles, such as CEOs who think they are good marketers when they aren’t, or conflicts between sales and marketing staffs. But with big data becoming an essential part of the marketing discipline, there’s more opportunity for marketing to impact a company’s future than we’ve seen since the dawn on TV advertising.

Listen to our 14 min. podcast here:

Security Intelligence (IBM) blog: Space Rogue, A Security Rebel Turned Pen Tester

Cris Thomas, who also goes by the pseudonym Space Rogue, is the global strategy lead at IBM X-Force Red. I recently spoke with him to discuss his work as a penetration testing specialist, his role as a cybersecurity activist in the late 1990s. In 1998, Thomas and other members of attacker think tank L0pht Heavy Industries testified to Congress. L0pht is infamous for developing a series of hacking tools, such as Windows NT password crackers and a website called Hacker News Network. The white-hat hacking group also took on numerous consulting projects over the years and was recently back in DC to talk about what has changed, and what hasn’t, in terms of infosec. My interview with Thomas can be found in IBM’s Security Intelligence blog.

Having better risk-based analysis for your banks and credit cards

When someone tries to steal money from your bank or credit card accounts, these days it is a lot harder, thanks to a number of technologies. I recently personally had this situation. Someone tried to use my credit card on the other side of Missouri on a Sunday afternoon. Within moments, I got alerts from my bank, along with a toll-free number to call to verify the transactions. In the heat of the moment, I dialed the number and started talking to my bank’s customer service representatives. Then it hit me: what if I were being phished? I told the person that I was going to call them back, using the number on the back of my card. Once I did, I found out I was talking to the right people after all, but still you can’t be too careful.

This heat-of-the-moment reaction is what the criminals count on, and how they prey on your heightened emotional state. In my case, I was well into my first call before I started thinking more carefully about the situation, so I could understand how phishing attacks can often work, even for experienced people.

To help cut down on these sorts of exploits, banks use a variety of risk-based or adaptive authentication technologies that monitor your transactions constantly, to try to figure out if it really is you doing them or someone else. In my case, the pattern of life didn’t fit, even though it was a transaction taking place only a few hundred miles away from where I lived. Those of you who travel internationally probably have come across this situation: if you forget to tell your bank you are traveling, your first purchase in a foreign country may be declined until you call them and authorize it. But now the granularity of what can be caught is much finer, which was good news for me.

These technologies can take several forms: some of them are part of identity management tools or multi-factor authentication tools, others come as part of regular features of cloud access security brokers. They aren’t inexpensive, and they take time to implement properly. In a story I wrote last month for CSOonline, I discuss what IT managers need to know to make the right purchasing decision.

In that article, I also talk about these tools and how they have matured over the past few years. As we move more of our online activity to mobiles and social networks, hackers are finding ways at leveraging our identity in new and sneaky ways. One-time passwords that are being sent to our phones can be more readily intercepted, using the knowledge that we broadcast on our social media. And to make matters worse, attackers are also getting better at conducting blended attacks that can cut across a website, a mobile phone app, voice phone calls, and legacy on-premises applications.

Of course, all the tech in the world doesn’t help if your bank can’t respond quickly when you uncover some fraudulent activity. Criminals specifically targeted a UK bank that was having issues with switching over its computer systems last month, knowing that customers would have a hard time getting through to its customer support call centers. The linked article documents how one customer waited on hold for more than four hours, watching while criminals took thousands of pounds out of his account. Other victims were robbed of five and six-figure sums after falling for phishing messages that asked them to input their login credentials.

Steve Ragan in a screencast below shows you the phishing techniques that were used in this particular situation.

The moral of the story: don’t panic when you get a potentially dire fraud alert message. Take a breath, take time to think it through. And call your bank when in doubt.


Finding the right escape room for your group

I am a bit slow to the whole escape room phenomenon, but it seems like a great idea to me. While I am not a computer gamer, I have run sites with that editorial content and know many professional gamers as a result. I am also a big Sudoku and crossword fan, having done those puzzles for more than a decade.

The idea, if you are still not tuned in, is to bring a few friends to a facility and try to escape from a locked room within an hour. You have to solve various puzzles. Actually, you have to find the clues and then figure out the puzzle, without a lot of guidance. If you haven’t ever done a room, you first have to be very observant, looking at what objects have been placed in the room, what information is written on the walls or displayed on various monitor screens, and what objects might lead you to other things. For those of you that don’t like solving puzzles, this is probably not something you are going to like. If you do like puzzles, or if you go to haunted houses every fall (or even build your own), this is probably something you have already checked out.

While I am not a computer gamer, I recognize that many years ago I spent weeks of my life trying to solve the puzzles of Myst. Back then, I said that “Myst starts out a total puzzle, and as you gain skills and understand the sequence of play involved, you get drawn into the universe of the game and lose track of real life and elapsed time.” You can say that about many modern computer games too. The problem with this is that you only have an hour to escape your particular room, and you don’t know how many puzzles you will have along your journey.

Given that there are thousands of rooms in cities all over the world, if you want to try one out the next hurdle is going to be to find one that suits your particular skills, experience, and group. Wouldn’t it be nice if someone reviewed rooms with some sort of consistency? Fortunately, there is a site that does called EscRoomAddict. I spoke to one of their editors, named Jeremie Wood. (You can see a sample of one review here.)

The site has teams of reviewers in LA, Chicago, New York, Kansas City, Denver and Toronto, which is where they began four years ago. They have reviewed more than 400 rooms in North America. There are other sites that have reviews, but not as well organized or as consistent in their evaluations as ERA, as they call themselves. The site doesn’t pay their reviewers, but usually the room operator comps the reviewers to do the room. Many of his reviewers have played 50 or more rooms during their tenure, and Wood himself has lost count but thinks he has been party to at least 180 room reviews.

He told me based on his experience that he doesn’t think the escape room craze has peaked yet, and there are still new rooms being built. One opportunity is to try to attract more corporate customers, who use the room as a team-building exercise. And part of that effort is what motivated the founders to start ERA, so that corporate customers could find the best rooms in a particular location.

The escape room landscape is also changing. “Many of the early operators have closed, mainly because the standards for the best experience keep going up.” You might think that the best rooms are the ones that take the most money to build, but that hasn’t been his observation. “I have seen great rooms that didn’t cost much, and lousy rooms that were very expensive,” he said. ”You don’t have to spend huge amounts of cash, but you do have to know what you are doing and design something that has really great puzzles and a great story.”

One of the reasons I like the ERA site is that it attempts to have consistent review metrics for all of its room reviews. The teams from the various cities met earlier this year here in St. Louis to try to iron out consistent style and to set up minimum requirements for their reviews. The reviewers also try to take into account a wide range of puzzle solving ability in their write-ups. Each room is done by at least three different people, who then collaborate on the review, and they usually agree on their evaluation.

Having been to so many rooms, Wood told me that the average Canadian rooms are smaller and more suitable for 4 to 6 people, whereas in the States, they can hold more participants. Also, in Canada, you usually book a room exclusively for your own group, even if it is smaller than the room capacity. In the US, your team is sharing the room with others if the demand is there.

If you have particular room experiences and want to share them with my readers, please post a comment here.

Why your networking future shouldn’t include NAT

This post is taken from a recent issue of the Internet Protocol Journal and reused with permission. It is written by Leroy Harvey, a data network architect.

The networking world seems to be losing sight that NAT is a crutch of sorts, a way of dealing with the primary problem of a lack of IPv4 addresses. An earlier article in IPJ stated that NAT provides a firewall function. I think NATs and firewalls are mutually exclusive, even if they are found on the same networking device. This is because NATs don’t by themselves provide any natural protection from the host on the other side of a protection point. The two can operate independently.

NAT does present real-world problems with a number of products, such as Microsoft AD Replication and IBM’s Virtual Tape Library. Passing through a NAT breaks the application’s intended communication model and requires compensating mechanisms.

We are asking the wrong question if we say, “should I deploy IPv6 now”. Someone once told me that IPv6 was here to stay. To my way of thinking, it has not arrived after 25 years.

Let’s look at the situation where we want to merge two large company networks together that both make extensive use of NAT. This becomes more complex than if the two networks were originally using a valid replacement for IPv4 and sadly, that protocol doesn’t exist. While I agree with the notion that the Internet can’t be completely stateless, this doesn’t justify using NAT as middleware. Justifying NAT for the sake of IPv4 life-support is nonsensical.

We should appreciate NAT for its role as a tactical compensating mechanism for IPv4 address depletion, not a a strategic future-proofing scalability mechanism for IPv4. Really what many are saying about NAT is just putting lipstick on the IPv4 pig. Unfortunately, in IT there is nothing more permanent than a temporary solution. Let us not fall victim to this easy psychological trap only because we seem to have collectively painted ourselves into a corner of sorts.


In my role as a journalist, I’ve been deluged with hundreds of pitches for GDPR-related stories, which went into effect last week. It didn’t help matters that on the first day the UK commissioner’s website was down for a couple of hours, an Austrian privacy advocate hit Facebook and Google with billions of euros in lawsuits and the privacy browser plug in Ghostery sent out emails about its change in policy, but inadvertently cc’d 500 user names in each batch of email.

In this episode of FIR B2B podcast (19 min.), I discuss the impact of GDPR with my partner Paul Gillin, who has seen his fair share of pitches as well. We discuss some of the best and worst PR pitches we received in the months running up to the launch of the General Data Privacy Regulation, and why a handful stood out.

SecurityIntelligence (IBM blog): Are ransomware attacks rising or falling?

There are conflicting reports over whether or not ransomware attacks are growing. Many organizations state (quite convincingly) that it’s the most popular malware form and that ransom-related attacks have been increasing at a rapid rate over the past year. However, other reports offer a more nuanced point of view.While the raw number of ransom-based attacks is increasing, the proportion of ransom-related attacks is dropping over the last part of 2017. Many businesses are not paying out the ransoms, motivating criminals to try other malware methods.

I compare the results and show how they differ in my latest blog post for IBM”s Security Intelligence blog.