Category Archives: App Dev tools

SaltStack: beyond application configuration management

Posted on by
1

When it comes to building online applications, you can build them with old tools and attitudes or with new methods that are purpose-built for solving today’s problems and infrastructures. Back in the days when mainframes still walked the earth, setting up a series of online applications used some very primitive tools. And while we have more integrated development environments that embrace SaaS apps running in the cloud, it is more of a half-hearted acceptance. Few tools really have what it takes for handling and automating online apps.

Today’s IT environments are in a constant state of flux and moving at an unprecedented velocity. The tools used to manage these environments weren’t designed for this level of complexity nor designed for rapid changes in resources. The modern data center requires juggling numerous open source repositories, handling multiple cloud providers, being able to rapidly scale up and down its resources, orchestrating changes and populating builds across multiple servers and services.

And matters are only going to become more complex. More non-digital businesses are moving into the cloud, creating new applications that make use of mobile devices that tie them closer to their customers, suppliers and partners. Digital-first vendors are adding features and integrating their websites with a variety of third parties that both increase their security risks and complicate their applications flow and logic. The old days of manual labor for handling these situations are looking more than ever like the days when we last made buggy whips.

Typical use cases

Salt was initially created to handle remote execution across complex application development environments, allowing its users to execute commands across thousands of servers concurrently and automatically. But today we need more from our toolsets than just the ability to run code remotely. Since it began in 2012, Salt has expanded its role to thrive on a mixed open and closed source environment that spans cloud and on-premises infrastructures. Here are some typical scenarios for its use:

  • A developer needs to schedule tasks that run in a particular sequence, waiting for a dependent server to reach a particular state before it can be launched. While this can be done manually, it can be tedious and error-prone and begs for more automated methods.
  • Or an IT manager needs to install a particular set of updates and patches to their environment. However, these must be done in a certain order and only when one is successfully installed can the next step be initiated. To add to this complexity, the IT department manages a mixed collection of Windows, Macs and Linux machines that carry particular pieces of their applications infrastructure. Again, this could be done manually, but not in a reasonable time when these patches have to be applied to a thousand different servers.
  • An application development manager needs to deliver the latest build of their software stack to their production environment, while ensuring that the code is secure and solid. Manual methods are inadequate to handle the velocity of coding changes and applications provisioning in any timely fashion.
  • An infrastructure engineer needs to set up a multi-tiered web and database application that will require a combination of servers, networks, storage and security devices. The complete collection spans multiple VMs, Docker containers and physical servers, all of which have separate and complex configurations where one misstep could mean a large amount of downtime and debugging.
  • A new security exploit is discovered that has massive implications across a variety of OS’s and system configurations. Security researchers recommend wholesale updates to be done as quickly as possible, to avoid any potential intrusions by hackers. Using “sneaker-net” or running from server stack to stack will take weeks to accomplish, not counting the time needed to verify the changes are made correctly.
  • An engineer wants to automatically enable auto-scaling features of their cloud provider to match the resources needed as demand rises and falls. While the major cloud vendors offer the ability to spin up and down VMs as needed, more coordination is needed to install the right series of application servers on the new VMs and to balance the overall loads appropriately. This is nearly impossible to accomplish manually.
  • Or an enterprise wants to migrate its entire cloud infrastructure from AWS to Azure, which involves moving hundreds of virtual servers in a particular order and under certain specifications for each VM. Doing this manually would involve weeks of work, and workers need automation to help with the migration.

Salt’s key features

In each of these cases, the old-school manual methods are inadequate for reasons of time, accuracy, security, or just the sheer effort involving coordinating expensive and highly-skilled IT staffers. That is where Salt comes into play. Here are some of its key features.

Salt’s event-driven automation tools make these tasks much easier to programmatically happen, without a lot of manual operator intervention

Salt also understands orchestration and how the sequencing of various steps has to occur.  Salt can handle the necessary conditional logic that control the various configuration and installation steps.

It also contains cloud controls that can manage public, private, and hybrid clouds. It can extract the infrastructure layer, spin up VMs under certain conditions and with certain configurations. This makes moving from one cloud provider to another easier and less error-prone.

Salt comes with sensors that react under certain conditions, such as the presence or absence of a particular application or detection of a particular OS version level.

As we said earlier, Salt originally was created for remote execution tasks. It deploys both push and pull architectures. This differs from many other configuration management tools which make use of one or the other methods. Salt has the flexibility to mix both kinds, making scheduling and message-connected events simple. It addition, it can handle both agent and agentless options, to give its automation processes the maximum level of flexibility and support to the widest collection of endpoint devices, servers and services.

Finally, to support all these automated methods, Salt has solid configuration management features that can detect and manage a wide variety of circumstances. All of its scripts are written in Python, making them more accessible to a wider collection of developers who have learned this language. Other tools have their own proprietary scripting tools that have steeper learning curves.

Salt is used by a wide variety of digital businesses to manage tens of thousands of VMs and physical servers, including LinkedIn and eBay. At the former, it is used to serve up massive amounts of data at very low latencies to improve usability. Salt enables ”us to quickly and dynamically provision caching layers for many of the services that make up our site,” according to that blog post. You should take a closer look at what they offer and how it can be deployed in your organization.

CSOonline: 4 open source red-team ATT&CK-based tools reviewed

Posted on by
Reply

In an article that I wrote last week for CSOonline, I described the use of a red team framework from Mitre called ATT&CK. in my post this week, I compare four free open source tools that leverage this framework and how they can be deployed to help expose your network vulnerabilities. The four tools are:

  • Endgame’s Red Team Automation (RTA),
  • Mitre’s own Caldera,
  • Red Canary’s Atomic Red, and
  • Uber’s Metta

Each have their good and bad points. You can read my review here.

HPE blog: The changing perception of open source in enterprise IT

Posted on by
Reply

Once upon a time, when someone in IT wanted to make use of open source software, it was usually an off-the-books project that didn’t require much in the way of management buy-in. Costs were minimal, projects often were smaller with a couple of people in a single department, and it was easy to grasp what a particular open source project provided. Back then, IT primarily used open source to save money and “do more with less,” letting the department forgo the cost of commercial software.

Times have certainly changed. Yes, software costs are still a factor, and while it is generally true that open source can save money, it isn’t the only reason nowadays to adopt it. While application deployment costs have risen, the direct software cost is a small part of the overall development budget, often dwarfed by infrastructure, scalability, and reliability measures.

As a result, today’s open source efforts aren’t anything like those in earlier days.

You can read the full story on HPE’s blog here.

What happened to the Web user interface?

Posted on by
1

More than 20 years ago, the Web was just getting started. People were experimenting with all kinds of web servers as publishing mechanisms and as user interfaces for various devices. Back then, I thought this was a neat idea: having a web interface was a great way to demonstrate a product across the Internet, unify the user experience across different browsers and end user platforms without having to develop separate programs for them, and perhaps simplify end user training too. It was the brave new world.

Back then, there were some dissenting voices. Having more Web UIs would ”set computer programming back 30 years and is about the worst technology I’ve laid eyes on,” said one UI consultant that I interviewed at the time. Another pointed out that the Windows graphical interface (which was just getting going back then) was far superior to anything the Web could produce in terms of interactive controls. That distinction has largely disappeared over the decades. And having the cloud to handle various tasks (think calendar synch or database queries) makes the Web UI superior to a local Windows app under certain circumstances.

I wrote about these issues for Computerworld in the summer of 1996. Back then, Netscape (remember them?) and Microsoft were duking it out over which company’s HTML extensions were going to become more popular (we know how that fight went down). At the time, I said, “having all software go to the Web UI might hasten to have an all-Windows world: since multi-platform apps can be supported by web servers, developers have moved away from Everything Else and concentrated on Everything Windows.” I don’t think that has come true, and let’s not forget about smartphone apps that have their own wicked interface with their own screen real estate limitations.

I asked my favorite UX consultant, Danielle Cooley, what she thought about my comments from 1996. “Things have changed dramatically, of course, both on the technology side and the design side,” she told me in a recent email. “Speaking as the user advocate, I would say consumers’ standards are much higher across the board then they were 21 years ago. Thanks to the user-centered approach taken by large organizations like Amazon, Apple, and Google, laypeople have less patience for digital products that force them to contort their thinking and behavior. Now, they have more and more access to tools that fit the way they already think and behave. Many organizations still suffer from serious UX immaturity. Lack of investment and integration here has resulted in the confusing and frustrating interfaces we’ve all come to hate. The fact that there are still SO MANY of these, 21 years after your Computerworld article, is telling and alarming.”

But the Web UI is here to stay, one way or another. Now at least we have responsive design, so at least smaller or larger screens can view appropriate webpages automatically. And hopefully, developers will finally learn what makes for a better UI experience.

Is Windows Continuum Worth Your Time?

Posted on by
Reply

When I was attending the Citrix Synergy show last week, much was made about the support of the Windows Continuum effort by Microsoft. This puts the Windows 10 functionality on a lot of different and non-traditional IT devices, such as the Surface Hub gigantic TV, Xbox consoles, and Windows Phones. If you look at the linked webpage above, you will see a lot of information about how you can use a Windows Phone as the basis for a new kind of docked workstation that has a real keyboard and screen attached.

When I spoke to Citrix SVP PJ Hough about this, he changed my thinking about Continuum. It isn’t all about the Windows Phone, but about the other stuff that is enabled here. Continuum is really about how you can essentially upgrade these devices to become smarter about their deployment and delivery of Windows apps themselves.

Naturally, Citrix has a vested interest here, because Receiver now supports Windows 10 S installations, which are devices that are part of the Continuum ecosystem. One of the issues for Win10 S is that it is a locked-down OS that only runs the applications delivered from Windows Store. This means if you have legacy Win32 apps on your older desktops, you were out of luck to run them before now. Having Receiver on 10 S gives you the best of both worlds: a more secure desktop that can still run your crusty older apps in a protected workspace.

Citrix Receiver — compatible with Windows 10 S — is built using the Microsoft Universal Windows Platform technology. This was introduced by Microsoft earlier this year and at this link you can find more information on how to build apps and learn from the samples that they have provided. Essentially, what Microsoft is trying to do is create a common core that app developers can use on a variety of other devices, including HoloLens and its Surface line of tablets and TVs.

But the real secret sauce of the universal platform is how it can be distributed using the Windows Store. Microsoft has learned from the Apple Store that app distribution is the real friction for getting apps to actually be used. Universal apps thus come with a built-in marketing bonus.

To make true use of Citrix Receiver, you of course will need XenApp and XenDesktop, running on XenServer or in a cloud-based infrastructure through Citrix Cloud to deliver the complete desktop experience. You can see the video of how this works here:

Lessons learned from building software at scale

Posted on by
1

So you have read The Lean Startup. Suffered through following several agile blogs (such as this one). You think you are ready to join the cool kids and have product scrums and stand-up meetings and all that other stuff. Now you need an implementation plan.

Maybe it is time to read this post by Paul Adams on the Intercon blog. He describes some of the lessons he and his development team have learned from building software and scaling it up as the company grows. I asked a few of my contacts at startup software firms what they thought of the post and there was mostly general agreement with his methodology.

Here are some of Adams’ main points to ponder.

Everyone has a different process, and the process itself changes as the company matures and grows. But his description is for their current team size of four product managers, four software designers, and 25 engineers. Like he says: “it’s not how we worked when we had a much smaller team, and it may not work when we have doubled our team.”

Create a culture where you can make small and incremental steps with lots of checkpoints, goals, and evaluations. “We always optimize for shipping the fastest, smallest, simplest thing that will get us closer to our objective and help us learn what works.” They have a weekly Friday afternoon beer-fueled demo to show how far they have gotten in their development for the week. Anyone can attend and provide comments.

Facetime is important. While a lot of folks can work remotely, they find productivity and collaboration increases when everyone is in the same room in a “pod.” Having run many remote teams, certainly local pods can be better but if you have the right managers, you can pull off remote teams too. It appears IBM is moving in this “local is better” mode lately.

Have small teams and make them strictly accountable. Adams has a series of accountability rules for when something goes wrong. Create these rules and teams and stick by them. “We never take a step without knowing the success measurement,” said one friend of mine, who agrees with much of what Adams says in his post. My friend also mentions when using small teams, “not all resources have a one-to-one relationship in terms of productivity; we find that we that the resources we use for prototyping new features can generally float between teams.”

Have a roadmap but keep things flexible and keep it transparent. “Everything in our roadmap is broken down by team objective, which is broken down into multiple projects, which in turn are broken down into individual releases,” said Adams. They use the Trello collaboration tool for this purpose, something that can either be a terrific asset or a major liability, depending on the buy-in from the rest of the team and how faithful they are to keeping it updated.

However, caution is advised: “The comprehensive approach that Adams describes would be entirely too much overhead for most startups,” says my friend. This might mean that you evaluate what it will take to produce the kind of detail that you really need. And this brings up one final point:

Don’t have too many tools, though. “Using software to build software is often slower than using whiteboards and Post-it notes. We use the minimum number of software tools to get the job done. When managing a product includes all of Google Docs, Trello, Github, Basecamp, Asana, Slack, Dropbox, and Confluence, then something is very wrong.”

Everyone is now a software company (again)

Posted on by
3

Several years ago I wrote, “everyone is in the software business. All of the interesting business operations are happening inside your company’s software.” Since then, this trend has intensified. Today I want to share with you three companies that should come under the software label. And while you may not think of these three as software vendors, all three run themselves like a typical software company.

The three are Tesla, Express Scripts, and the Washington Post. It is just mere happenstance that they also make cars, manage prescription benefits and publish a newspaper. Software lies at the heart of each company, as much as a Google or a Microsoft.

In my blog post from 2014, I talked about how the cloud, big data, creating online storefronts and improving the online customer experience is driving more companies to act like software vendors. That is still true today. But now there are several other things to look for that make Tesla et al. into software vendors:

  • Continuous updates. One of the distinguishing features of the Tesla car line is that they update themselves while they are parked in your garage. Most car companies can’t update their fleet as easily, or even ever. You have to bring them in for servicing, to make any changes to how they operate. Tesla’s dashboard is mostly contained inside a beautiful and huge touch LED screen: the days of dedicated dials are so over. These continuous updates are also the case for The Washington Post website, so they can stay competitive and current. The Post posts more total articles than the NYTimes with double the reporting staff of the DC-based paper. That shows how seriously they take their digital mission too.
  • These companies are driven by web analytics and traffic and engagement metrics. Just like Google or some other SaaS-based vendor, The Washington Post post-Bezos is obsessed with stats. Which articles are being read more? Can they get quicker load times, especially on mobile devices? Will readers pay more for this better performance? The Post will try out different news pegs for each piece to see how it performs, just like a SaaS vendor does A/B testing of its pages.
  • Digital products are the drivers of innovation. “There are no sacred cows [here, we] push experimentation,” said one of the Post digital editors. “It is basically, how fast do you move? Innovation thrives in companies where design is respected.” The same is true for Express Scripts. “We have over 10 petabytes of useful data from which we can gain insights and for which we can develop solutions,” said their former CIO in an article from several years ago.
  • Scaling up the operations is key. Tesla is making a very small number of cars at present. They are designing their factories to scale up, to where they can move into a bigger market. Like a typical SaaS vendor, they want to build in scale from the beginning. They built their own ERP system that shortens the feedback loop from customers to engineers and manages their entire operations, so they can make quick changes when something isn’t working. You don’t think of car companies being so nimble. The same is true for Express Scripts. They are in the business of managing your prescriptions, and understanding how people get their meds has become more of a big data problem. They can quickly figure out if a patient is following their prescription and predict the potential pill waste if they aren’t. The company has developed a collection of products that tie in an online customer portal to their call center and mobile apps.

I am sure you can come up with other companies that make normal stuff like cars and newspapers that you can apply some of these metrics to. The lessons learned from the software industry are slowly seeping into other businesses, particularly those businesses that want to fail fast and more quickly as their markets and customers change.

Why runtime application self-protection is critical for next gen security

Posted on by
Reply

raspToday most of us go about implementing security from the outside in. The common practice to define and then defend a perimeter isn’t viable any longer. With the added complexities of more mobile endpoints, agile development and more sophisticated malware, better protective methods are needed.

In this white paper I wrote for VASCO , I describe a method that is gaining traction by defending the actual apps themselves using runtime self-protection. RASP, as it is called, comes from a Gartner 2012 report, but is catching on with several vendors, including Arxan Technologies, HPE App Defender, Immun.io, Lookout App Security/Bluebox, Prevoty, Vasco Digipass for Apps, Veracode and Waratek.

RASP can be a solid defense and a way to isolate and neutralize a potential threat, so you can operate your business safely in these uncertain environments.

 

Fast Track Blog: There are Better Ways to Manage Data than Google Docs

Posted on by
Reply

Google Docs is a favorite way to build applications for lightweight data manipulation, reporting, and analytics as well as useful for building websites that can capture and display data. While it is a great tool to get started using an online all-purpose office suite, you should also know its limitations and when it is time to move on to something more industrial strength. In my post for Quickbase’ FastTrack blog, et’s look at what is missing and when you should move on.

Quickbase blog: Signs that you have outgrown Microsoft Access

Posted on by
1

Many of us started out with database software with something like Microsoft Access. It was part of the Office suite, fairly easy to get started and infinitely customizable. However, it might be time to look elsewhere for alternatives, especially for citizen developers who want to build more sophisticated online databases. Here are some ways to recognize the warning signs and to start thinking about its replacement.

First, Access was designed as a personal product, where one developer creates an entire application from scratch. If your need are more collaborative, or where you have a database where multiple people input information, Access isn’t the best solution and this is where having a SaaS-based app shines. Using an online product like QuickBase means you don’t have to worry about setting up a server and worrying about when more than one person is inputting records: this is handled automatically for you.

Second, when non-Windows computers are using your database it also might be time to switch. While there are now versions of Access for Macs and iOS, not everything that is developed for Windows versions works across platforms. It might make sense to build your app with something that natively speaks the Web, or that has the exact same look and interface across all versions.

Are you heavily involved in using Visual Basic? This was the underlying programming language for Access, and while it has a large ecosystem that Microsoft has done a great job cultivating, it might be costly to maintain and to hire the appropriate skilled staff to continue to build VB apps. Look at ways that you can build your apps without a lot of programming expertise, or that make use of Web-based forms and templates that can accomplish many of your tasks quicker, and with lower cost of ownership.

And when your users span the globe and are working on the same app, it also might be time to retire that Access project and find something more flexible and more comfortable working in a distributed environment.

When you can’t find your favorite command or function, thanks to yet another UI “improvement.” Tired of Microsoft re-arranging the menu ribbon yet again? It seems every major upgrade of Office comes with a new interface. Now Microsoft touts a “feature” that helps you find your favorite command.