Archives for posts with tag: vulnerability

For all the time that I have been managing veterinary hospitals I have also had oversight responsibility for the computers and technological systems (I.T.) employed at those hospitals. Coming from a technical background in the entertainment lighting world, this just seemed to be a natural extension of my existing skill sets – the things that make me good (hopefully) at what I do.

One of the things that I have always been passionate about is data security. I’ve written articles and had them published on the subject. I made sure my hospitals had proper backup procedures, good anti-virus protection, updates ran regularly, and great firewalls.  I’d always felt that there is always an element of carelessness, or lack of understanding of the risks, when hearing about those that have fallen foul of hardware failures or ransomware. Nothing in my years of experience did anything to dissuade me from this impression.

Until one of my hospitals was hacked.

The story starts first thing on a Sunday morning when I get a call from the office manager on site saying that they are unable to get their practice management software up and running. I try logging in remotely and get nowhere so I make the five-minute drive to take a look. What meets me on the screen of the server is a message that runs my blood cold and leads to feeling of despair sinking into my stomach.

“Your System has been hacked. All of your data has been encrypted. To release your, data payment must be made via Bitcoin….”

We have backups, we have a replication server, we’ll be fine.

We were not.

The ransomware attack had been possible due to one the connections that we used to allow doctors to write up their medical records from home. In addition, there had been an old user with administrator rights that had somehow been overlooked and led to the hacker being able to access the server. The height of irony was that we were in the process of moving over to a more secure system to allow remote access when this attack took place. If we had been a couple of weeks further along this attack would not have been possible as it unfolded.

Our server was encrypted, our replication server was encrypted, our daily incremental backup drive was encrypted, our weekly full backup drive was encrypted, and several workstations were also encrypted. We had no internet, no practice management software, and so no access to medical records, schedule, email, or files.

Paying was not an option on general principles.

Our last hope was offsite cloud backup.

This backup had been fully protected and within 24 hours we were able to have remote access to this so we could access schedule and records.

We tried for three days to download the massive database onto a drive to allow us to restore the server. After three days of failure, in part cause by file size, an inopportune Windows automatic update, and network / computer stability issues, our cloud backup vendor arranged for a physical drive to be sent to us. Once the drive arrived (at 8PM at night several days later) the instructions were unclear as to how to access what had been sent to us and the cloud backup provider did not have anyone on staff late at night who knew how the drive had been prepared. We finally restored functionality to most of the hospital on the seventh day after the attack.

Lessons learned

Our I.T. vendor had, for the most part, been great. They understood the position we were in and I, in turn, protected them from the owners and staff who were rightly upset and frustrated. I had several moments of frustration myself , particularly when it came to getting a physical drive from the cloud backup vendor which turned into a comedy of errors. But both sides were able to work on the problem and maintain a professional atmosphere. We’ve had a long relationship with our I.T. support vendor and they have been very good to us in turn. They understood our need to go with other suppliers for things such as phone systems and servers but were still being prepared to help support those items and the overall health of our networks. Without that long term relationship, and atmosphere of mutual trust, things could have been very difficult indeed.

We used our barely functioning network to try and download a huge amount of data. We should have done this offsite, at one of our other locations. We should have also immediately requested a physical drive to be sent to us. I offered multiple times to get on plane and courier the drive personally, however, this was turned down but did add to the pressure on the cloud backup company to get their act together.

Try to be calm. After all was said the done the total loss of business for the week that we were unable to either take care of that week or squeeze into the following week was estimated at 4% – well within the normal variation from week to week. Not even close to the amount to bother our insurance company with. Clients will understand. Deal with what you can, improvise, and communicate as much as possible with everyone.

The major lesson that I learned, however, was one of humility. Anything can he hacked. All it takes is time and a willingness to spend that time. There was, in the heat of the moment, a number of times when the blame game reared its head. I made the decision to not allow that from anyone, feeling that if there was any blame it was the wrong time to even talk about it. What I ultimately realized that what is important was not in preventing a hack, but our resilience if the face of that attack. It is not a matter of if, but when. We lost no data – I consider that a great victory. We lost little to no business – I also consider that a victory. We also came out the other side of the ransomware attack with a much stronger awareness and agreement on the importance of cyber security.

Humility is not an excuse, or a reason to not try everything possible to prevent issues. But it helps with the realization that all systems are vulnerable. That the very things that make I.T. systems so great and useful, are also the things that can lead to vulnerabilities.

Humility is directly related to resilience. When bad things happen what is important is that we can recover from them as quickly and easily as possible- not to pretend that there are no bad things or that we are immune to them.

 

The Culture Code

It is easy to dismiss “The Culture Code, The Secrets of Highly Successful Groups” by Daniel Coyle within the first few pages as I very nearly did.

This, however, would be a mistake.

There are two initial problems. The first is in the choice of companies, or organizations, that are used as case studies. In the time since the book was written, and even since its publication in January of 2018, two of these heavily featured companies have undergone significant cultural upheaval and it is hard not to see those case studies through the prism of hindsight. Pixar lost John Lasseter due to revelations in the wake of the #meetoo scandal. And Zappos, to add to the woes mentioned in the book regarding the Downtown Project, lost 18% of its workforce, including a significant proportion of management, due to its all or nothing adoption of Holacracy. To be fair to both companies, they both seem to have survived these events and continue to grow; but it does make the reader question the book from the start.

In addition, it is hard to shake the impression from the initial introduction and chapters, that The Culture Code and its talk of “belonging cues” is more about hacking interpersonal relationships and the manipulation of people through our actions and specific phraseology. Which just feels wrong.

This, however, is not the case.

What the Culture Code has unpicked is the remarkable reasons why teams of people work well together, and why they don’t work. We presume teams of skilled individuals will produce skilled results. And we are wrong as Mr. Coyle points out. Belonging cues, which can take the form of active listening, light touching, showing people where they fit into an organization, the closeness of employees’ desks, and the language we use, creates a continuous sense of safety. Even just simple “thank yous” from managers, and them picking up trash, can signal that “we are all in this together” and that they serve the group.

As with most culture research, The Culture Code repeatedly emphasizes that great cultures start at the top. One of the ways to create a safe space for the group is for leaders to be vulnerable. Being vulnerable is a significant belonging cue. Vulnerability sparks cooperation and trust, and asking for help as a manager, or leader, sends a clear signal that you have vulnerabilities. Interestingly, vulnerability can be contagious with the obvious benefits to the group. Difficult and painful interactions can actually help create a more bonded team through shared vulnerability.

While creating a sense of safety and vulnerability in the group makes for a better team, Mr. Coyle turns to storytelling to give that team focus. Groups that have successful cultures repeatedly and consistently, often to the point of redundancy, tell their story. Simple beacons, such as slogans, phrases, or imagery, focus attention to the shared goal. “High purpose environments are filled with vivid signals” the Culture Code reveals referring to Pixar having images of Woody and Buzz Lightyear in their buildings or the Seals having a piece of the World Trade Center in their lobby.

“Build a language to build behavior.”

Do we really need to tell nurses and other staff that a particular surgery is better for the patent, and that they should speak up if they see a mistake, even by a doctor, being made? The answer the Culture Code gives us is a resounding yes.

“The value of signals is not in the information but that they orientate the team to the task and to one another. What seems like repetition is in fact navigation.”
The Culture is that most unique of books. A book arranged and filled with great ideas and real-world examples of those ideas in action. Impeccably researched, the march of time notwithstanding, and well written, The Culture Code is a leadership book about daily interactions and grand visions. It is a management book showing the pitfalls and routes to success.

I’m better for having read it, and I have no doubt that it will be a book I return to and recommend to other managers.

%d bloggers like this: