Category: Technology

Chick-Fil-A…Runs Kubernetes…At Every Store

Lorin Olsen


How many of you thought that Chick-fil-A would have a tech blog? And how many of you thought that they would be clustering edge nodes at every store? When I read this article, I was surprised – and quite excited.

The basic use case is that every Chick-fil-A store needs to run certain basic management apps. These apps run at the edge but are connected to the central office. These apps include network and IT management stuff. But they also include some of the “mundane” back-office apps that keep a company going.

Routine stuff, right? But in the Chick-fil-A case, these apps/systems need to be remote and resilient. The hardware must be installed and maintained by non-technical (or semi-technical) employees (and/or contractors). If a node fails, the recovery must be as simple as unplugging the failed device and plugging in a replacement device. Similarly, the node enrollment, software distribution, and system recovery capabilities have to be automated – and flawless.

Here is where containers and Kubernetes enters the picture.

The secret to Chick-Fil-A’s success is the recipe that they use to assemble all of the parts into a yummy solution. The servers (i.e., Intel NUC devices) power up, download the relevant software, and join the local cluster. The most exciting part of this solution is its dependence upon commodity components and open source software to build a resilient platform for the company’s “secret sauce” (i.e., their proprietary apps).

The next time you go into a Chick-fil-A, remember that they are using leading-edge tech to ensure that you get the sandwich that you so desperately want to eat.

View at Medium.com

Default Passwords = Bad; Continuous Testing = Good

Lorin Olsen


Well, the verdict is in. The drone documents found on the dark web were drone maintenance documents. These documents were found behind a Netgear router whose FTP (file transfer protocol) password had not been changed.

This is a simple mistake. You might even say that this was a “rookie” mistake. Nevertheless, I am stunned that this kind of mistake would be made on a program that had already been granted its authority to operate (ATO). But the fact that this has happened proves that continuous vulnerability testing and compliance monitoring are keys to ensuring the ongoing (and safe) operations of a program.

And if this is true for the U.S. Department of Defense, then it is also true for each of us. So here is my simple question: have you changed default passwords on every system that you access?

Learn From Drone Documents Found on the Dark Web

Lorin Olsen

Today, the Wall Street Journal reported that secret data about combat drones had been stolen and had been made available on the “dark web”. This revelation should not be surprising. In a world where every document and every conversation can be digitized, there is ample opportunity for data to fall into unexpected hands.

Is this a problem with the “dark web” itself? No, not really. Yes, the dark web is inhabited by denizens. But it is also inhabited by those seeking relief from oppressive political regimes. The real problem here is that either secure systems have been breached or someone within the “military-industrial complex” has released sensitive data to an unauthorized recipient.

I am sure that an inspector general is already investigating. In the meantime, there are lessons to be learned – and applied – for your personal assets:

  1. Know your data. While you should protect everything, you should be able to say what data is truly valuable.
  2. Protect your valuable data. Have  layers of security. This should include strong (and unique) passwords, multi-factor authentication, encrypted “data at rest”, and also encrypted communications for valuable data.
  3. Review your protection plans on a regular basis. Perform threat simulations wherever possible. This is not something that should be done just by governments and corporations. You should do this for your own data – lest you be awoken to the sad truth that you have been hacked.
  4. Review all access attempts to determine if you have been breached. This means that you should check access logs (if possible) to see if they match what you actually did. For example, check last login times on tools like Facebook and Twitter. But this also means using tools like “Have I Been Pwned” so that you know whether your credentials have been compromised. You might even want to use tools from credit sources (like Experian).
  5. Always have a remediation plan if your data is compromised. This should include contacting service providers (especially banks), changing passwords, etc.

You may not have military-grade secrets to protect. But with a little investment of time, you can be craftier than the slower antelopes.

If Vigilance Is Required At Home…

Lorin Olsen
… then how much more important is it at work?
It is well said that the price of freedom is eternal vigilance. Similarly the price of personal freedom must be paid on a recurring basis. For me, activity during the week focuses upon work. And updating of security at home is almost always deferred until the weekend change window – when my wife (i.e., the CAB chairperson) can accept a more protracted outage.
So the change was scheduled for last night. And what were the contents of the change? Security updates were the sole focus.
Last month, the Talos team (at Cisco) issued a warning about an old threat (i.e., VPNFilter) that had returned from the dead – in a much more virulent form. Talos (and the FBI) recommended immediate reboots of home routers. I did this the same day of the warning. But after Talos (and the FBI) repeated their warnings about VPNFilter, I determined that it was time to rebuild the router from scratch following a factory reset. So once my wife disconnected from her “work” network, I started the changes. And it went reasonably well.
 
Since I coupled the change with a complete renumbering of the IP address space at home, the time before service restoration was longer than it would otherwise have been. In fact, the total rebuild of the router – and the assignment of new IP addresses across the network – took about two hours. After that window, normal services were successfully restored. But it took another two hours to clean up a few items – including the rebuilding of my Home Assistant hub. So the total change window lasted approximately four hours. At the end of the change window, we had a completely rebuilt home network.
 
When I got up this morning, I realized that it was also time to further secure my browser. My posture was immeasurably better than most of my neighbors. I browse via a VPN. I use uBlock Origin and Pi-hole to block ads. I use Privacy Badger for another layer of browser protection. But “good enough” is not good enough for me. So I decided to deploy uMatrix as an additional means of both understanding all network interactions and controlling those interactions.
 
For those not familiar with uMatrix (which is pronounced “micro matrix”), think of it as the next step beyond the NoScript tool. With uMatrix, you see a matrix of external sites and access types used when you load pages from any site (or domain). And you can allow access on either a temporary or a permanent basis. Once you get past the first shock of seeing all of the cross-site and cross-domain activity, you realize that uMatrix does provide you with incredibly granular control over how pages are rendered in your browser.
 
The first thing that I realized when I started to dig deeper was that securing my browsing experience almost always results in a “broken” user experience. This was not a new revelation. When I first used NoScript, I had to whitelist a whole lot of sites – or live with reduced functionality. So the process of evaluating sites and functions was both expected and welcomed.
 
The first sites that I decided to validate were those associated with security-related podcasts. And as expected, every podcast was accompanied by necessary changes to enable streaming. The most ironic thing that I saw was just how much cross-site activity was required to even listen to security podcasts. But knowing the precise elements that were needed by a page allowed me to open just those elements that were truly required. Basically, uMatrix provided me with fine-grained access control. And it also reminded me that “free” almost always means trading function/feature access against limited access to me (and my data) by advertising agencies/networks.
 
Once I dealt with the security podcasts, I wanted to see just how pernicious Facebook access was. Currently, I do not use any Facebook “apps”. Instead, I use a simple browser. I run their browser pages inside of a “container” that limits data leakage. Nevertheless, I still expected some additional cross-site activity. What I saw was positively astonishing. Over two-hundred elements requiring cross-domain access were requested. And that was after ad blocking was done by my Pi-hole and by uBlock Origin. Am I surprised? No, not really. But the scope of what remained – even after ad blocking – was positively astonishing.
 
So what are the key takeaways from yesterday and today?
 
  1. Change control is always needed – even at home. Of course, the discipline that you follow at home will depend upon the willingness of family members. But this is no different than how things function at the office. Build your processes to meet your stakeholders’ and customers’ needs. Please remember that there are differences between the needs of both groups. At home, you and your spouse are the stakeholders while your kids (and guests) are the customers. As the stakeholders, you need to make the choices about how much security is too much security. And I guarantee that whatever you decide, your kids will probably disagree with you. 😉
  2. There is no such thing as secure enough. You can always do more in order to be even more secure. And if you do nothing, you will just lose ground over time. To stay secure, you need to always do more.
  3. Always remember that “free” just means that the price may not be immediately discernible or quantifiable. Use tools that help you discern the heretofore indiscernible. I do recommend uMatrix. But other tools can be used.

The work of ensuring security is never complete. Your home is not safe just because you have a door lock. You need to lock it. And then you need to realize that your windows are a threat vector. In the same way, information security is not just about having an ISP-provided router and a password on you primary system. But whether you are totally insecure or currently “state-of-the-art” in your practices, there is always more that you can do. So take the next steps to further secure your home. Then remember, your workplace is no different than your home. It requires constant tending – by both the security professionals and by every employee.

“The Dark Web”: New Bogeyman…of Madison Avenue

Lorin Olsen

Every conflict needs a villain. This is true for Thanos, the Mad Titan (i.e., the protagonist of the latest “Avengers” movie). It is true for worldwide safety and security (e.g., terrorism in general and weapons of mass destruction in particular). It is also apparently true for online security services.

While doing my casual morning browsing of news sites, I ran across an ad for “dark web” scanning (linked below). I am not necessarily recommending the services offered by Experian. I am sure that it is a fine, general-purpose service. But I did want to highlight the use of fear and uncertainty as a motivation. Today, the “dark web” is the undeniable ‘big bad’ for online users. We are now told that it isn’t trusted companies (who abuse your identity for their revenue). It apparently isn’t the NSA (who collects everything about you in order to “protect” you). Listen carefully: according to Experian, it is the ‘dark web’ that seeks to hurt you.

Please don’t misunderstand my subtle (and not-so-subtle) prodding. The ‘dark web’ does provide a hideout for those who wish to lurk. At the same time, it provides a sanctuary for those escaping tyrannical pursuit (by hostile governments or hostile corporations). The ‘dark web’ is not – in an of itself – something to be feared. Rather, it is something to be understood.

At its foundation, the ‘dark web’ is a non-indexed part of the Internet whose content is obscured via encryption. So if you desire to be anonymous (and untraceable) while on the Internet, then you are a potential user of the dark web. And if you want to host content that is neither indexed (by Google) nor unencrypted, then you are seeking some of the attributes of the dark web.

Yes, Experian (and other companies) are offering you a “detector” that will let you know whether key pieces of your identity have been compromised by known individuals, groups, or sites that are identified as part of the “dark web”. Of course, they cannot tell you if some unknown individual, group, or site has your PII data. Unfortunately, it is the unknown threat that should concern you.

So here is a novel thought: assume that anyone can access the information that you move across the Internet. If you assume that everything is possible to compromise, then you will take the right steps to protect essential data that must move across the Internet. Don’t let someone else do the hard work for you. You must decide what is important to you. And you must decide which steps are appropriate and which are too onerous. For some folks, remembering to lock their back door is an onerous task – until they learn that their neighbors experienced a break in. Then, all of a sudden, locking the doors is not too onerous. So assume that your neighbors have been ransacked. And assume that your nosy neighbor wants more than just a smile in return. Be charitable. Be gracious. And be prepared.

And if you want to check out some free resources, then consider https://haveibeenpwned.com/.

http://bit.ly/2L5Uxny

DNS Rebinding Attacks: “Lions, and tigers, and bears – oh my!”

Lorin Olsen

What’s a new day without a new attack vector being published? Yesterday, Google, Roku, and Sonos all announced that they will be updating their home devices in order to address DNS rebinding attacks.

So what is a DNS rebinding attack? According to BleepingComputer.com,

The purpose of a DNS rebinding attack is to make a device bind to a malicious DNS server and then make the device access unintended domains. DNS rebinding attacks are usually used to compromise devices and use them as relay points inside an internal network. A typical DNS rebinding attack usually goes through the following stages:

1) Attacker sets up a custom DNS server for a malicious domain.

2) Attacker fools victim into accessing a link for this malicious domain (this can be done via phishing, IM spam, XSS, or by hiding a link to the malicious domain on a malicious site or inside ads delivered on legitimate sites).

3) The user’s browser makes a query for that domain’s DNS settings.

4) The malicious DNS server responds, and the browser caches an address like XX.XX.XX.XX.

5) Because the attacker has configured the DNS TTL setting inside the initial response to be one second, after one second, the user’s browser makes another DNS request for the same domain, as the previous entry has expired and it needs a new IP address for the malicious domain.

6) The attacker’s malicious DNS setting responds with a malicious IP address, such as YY.YY.YY.YY, usually for a domain inside the device’s private network.

7) Attacker repeatedly uses the malicious DNS server to access more and more of these IPs on the private network for various purposes (data collection, initiating malicious actions, etc.).

In short, an attacker can hijack your DNS queries and provide invalid (and malicious) responses.

So what should you do in response?

Actually, that is a tough question. This truly affects home users. It is not nearly as big a threat for enterprise administrators. Yes, every enterprise should mitigate this vulnerability through appropriate maintenance. And yes, if you place products/services within your customers’ home, then this is a current issue for you. But even if you are not involved today, it is important to note that as more devices in the home are becoming Internet-aware, this problem will become larger.

  • As a customer, update your client software as soon as updates are made available by the vendors. This is a default answer for most things. But in this case, it applies as well. Fundamentally, the issue is with the client software. So the client software is where the fix must be applied. Google, Roku, and Sonos have already committed to bringing forward appropriate fixes.
  • Press other product vendors to provide updates to their software. This includes: Amazon, Netgear, TP-Link, Phillips, Ikea (Tradfri), Blink, GE, and many others. As a professional, this is probably not your call to resolve. After all, do you really want to get involved in the dealings between a customer and another premise-device provider? Usually, I’d recommend keeping your nose out of other people’s business. But in this case, this is a matter of domestic hygiene. Your products and services will never work optimally if the entire home ecosystem is “polluted”. Of course, the biggest reason to be involved is to provide more mass in order to affect the “gravitational” effect upon these vendors. As an advocate for your customers, encourage your peers to “do the right thing”.
  • As a homeowner, I would recommend running your own DNS, if you can. Maintain its currency to ensure that its software does not become the next attack vector. Unfortunately, this step won’t resolve the current problem. But it will resolve many other problems – especially problems imposed by lax ISP maintenance procedures. If you can’t run your own DNS, then use a logical (or physical) “proxy” for your DNS queries. This will resolve many of these issues. For example, your SmartThings hub can deal with the internet-based DNS services for your devices. But whatever technical steps you may take, please be a counselor and advocate for your customers. At the same time, maybe it’s time for your company to provide a DNS appliance solution. Maybe this isn’t just “table stakes” for the OEM router providers and the ISP’s. Maybe your company can economically provide a cool product that bundles DNS, ad blocking, and proxy services.
  • If you use your own DNS, then use DNSSEC. This won’t be the short-term solution. Indeed, most IoT clients won’t have the processing power to provide authentication and encryption to a secure DNS infrastructure. But if you can bake this into your products, then do so – soon. Please, and thank you.

In the final analysis, this problem is an “edge” problem. So all solutions must occur at the edge. But if you are a service provider, then you have an obligation to your customers to act as a trusted advisor. Help them to be successful and they will help you to be successful.

https://www.bleepingcomputer.com/news/security/google-roku-sonos-to-fix-dns-rebinding-attack-vector/

Get Ranked to Become More Secure

Lorin Olsen
I’ve been in the business world for a few years. And in the past two decades, the forced ranking of employees has been used by most HR departments. These ranking systems have generated both great advantages and equally great disadvantage. But the motivation for implementing such competitive systems is quite clear: as humans, most of us are driven to compete. So it is theorized that this imperative can be channeled to “inspire” maximum performance while on the job.
 
We want to be the “best” in whatever we do. This includes having the best house (or car), maintaining the best yard, encouraging the best students (or student/athletes), or being the “best” member of a great team. These kinds of systems inspire us to be the best that we can be. Such reward-based systems are nothing new in technology either. For a generation, game designers have built reward systems into their products. It is no longer just about beating the “big bad”. It is also about wearing the best armor or having the coolest spaceship. And social media systems have often devolved into follower counting or “influence” ratings.
 
So how can such comparison and esteem systems result in a stronger security posture?
 
The folks at LastPass (which is owned by LogMeIn) have been using a “security challenge” program to motivate people to be more secure than they have ever been. While such a system does not work for everyone, it has always worked for me. As a result of this system, I remained dissatisfied with being in the top ten percent of LastPass users. The test inspired me to work hard in order to join the top one percent of users. And this week, it inspired me to implement any and all recommended areas of improvement.
 
I’m not certain whether the aforementioned example speaks to the power of motivation systems or to a fundamental facet of my personal psyche. But for the sake of this article, I’ll assume the former while considering the latter at some point in the future. After cleaning up (and locking down) all of my credentials, I decided to turn my focus towards household vulnerabilities. And my tool of choice to evaluate vulnerabilities is Nessus (http://www.tenable.com).
 
I’ll probably write a follow-up article about my findings – and my subsequent actions. In the meantime, I will tell you that the very first thing which I started to do after seeing the most recent results was to triage the important vulnerabilities. I looked at the items that Tenable noted as most important. I then researched and worked towards remediation of all of the highlighted vulnerabilities. Bottom line: I was motivated to be better than my nearest neighbors. This “better than the Jones’s” compulsion is driven by my fundamental view that to be a survivor, one cannot be the slowest antelope in the herd. Consequently, I am using an incentive-based system (and some fear-based motivation) to further strengthen my security posture.
 
In the final analysis, I am convinced that harnessing ego rewards and highlighting real risks (i.e., letting people know of the possible punishments for not addressing vulnerabilities) are a winning strategy – if you have a company with employees like myself.
 
http://smallbusiness.chron.com/employee-motivation-reward-systems-15978.html

Trading Privacy for a Little Convenience

Lorin Olsen
Benjamin Franklin once wrote, “Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety.” The quote (and its source) is often disputed (see https://www.npr.org/2015/03/02/390245038/ben-franklins-famous-liberty-safety-quote-lost-its-context-in-21st-century). But it is clear that modern privacy advocates see this quote as a proof text for the shortsightedness of exchanging your privacy for your security. Indeed, I too have used this quote as a rallying cry. But in candor, my use of this quote is more of an “appeal to authority” rhetorical argument rather than a reasoned defense of unfettered freedom.
 
But how should we respond to HART (the Homeland Advanced Recognition Technology project)? DHS is building a massive repository of identity information. This is, ostensibly, for ensuring our security. From the Electronic Freedom Foundation (at https://www.eff.org/deeplinks/2018/06/hart-homeland-securitys-massive-new-database-will-include-face-recognition-dna-and),
 

DHS’s plans for future data collection and use should make us all very worried. For example, despite pushback from EFF, Georgetown, ACLU, and others, DHS believes it’s legally authorized to collect and retain face data from millions of U.S. citizens traveling internationally. However, as Georgetown’s Center on Privacy and Technology notes, Congress has never authorized face scans of American citizens.
 
Despite this, DHS plans to roll out its face recognition program to every international flight in the country within the next four years. DHS has stated “the only way for an individual to ensure he or she is not subject to collection of biometric information when traveling internationally is to refrain from traveling.”

On its face, this is repulsive. And for most Americans, this kind of assault on our freedom and our right to privacy is unthinkable. But the federal government apparently hoped that this effort would gain little public attention.

But while we chafe over such obvious governmental incursions, why do we embrace the same incursions when they come from a private company? Most Apple users applauded the availability of facial recognition as part of the new Face ID feature. And I daresay that Android users would welcome the very same technology, if they knew that it already existed on their phones.

So what’s the problem with a company doing this?

There is little problem if you trust the company and if you read your grant of license. I daresay that miwe do trust companies and we don’t read license agreements. Of course, it should be the other way around. If we read the grant of license, then we would realize that most companies will use whatever they can leverage to increase profits for their owners/shareholders. And if we give away our rights (as well as personally identifiable information), then we are worse than those who gave away freedom for security. We’re doing it to save a few seconds of login time.

The Real Cloud Computing Transformation

Lorin Olsen


I recently gave the keynote address at a Kansas City symposium on cloud computing. So when I saw this article, my interest was piqued. The author believes that cloud computing is devouring itself. Um, alright. I guess I can buy the premise. But I think the author may be too absorbed with the technical layers of the infrastructure rather than focusing on the real transformation.
From my vantage point (as a consumer and as a technologist), I see this wave of transformation as being more about business and less about technology. Maybe even more important is the fact that tech-centric companies are moving from a capex-centric budgeting model to an opex-centric budgeting model.
What does this imply? First, it means that businesses are seeing IT as fundamental to their future. But while it is fundamental, it is no longer a strategic differentiator. Indeed, most companies are frustrated after spending thousands of dollars (or millions or billions) on IT capital without any real strategic benefit. The cloud allows a company to abandon capex investments in favor of opex outlays. This means that companies can be nimble – assuming that the cloud vendors are nimble enough to meet their customers’ needs.
But the larger and more fundamental implication of this capex/opex migration is that businesses are moving to an even more short-term focus. With a capex focus, companies were thinking about tax advantages and long-term viability. With an opex focus, they are looking at short-term return to the owners/shareholders. This does ensure that a company is competitive as long as it is agile.
But are we abandoning the future by focusing even more acutely on the short term? I am not certain. It does mean that on a micro-level, individual companies are trusting their future to their partners. On a macro-level, we may not be losing future investment streams to other countries. But we are pushing capital investment into the hands of a smaller number of companies that will be managing public or private clouds on behalf of other companies.
If this is indeed what is happening, then the savvy investor will need to look at hosting providers within the tech sector as these companies will be the real capital aggregation points. I’m looking at HP, Rackspace and Amazon being some of the biggest players in this space. And I would throw Salesforce.com into the mix as a software aggregation point.