Protecting Against Ransomware

[Richard]

Dave Rice wrote:

Such “embedded” devices or even separate control PCs are widespread in the defence and postal sectors too. But as I mentioned earlier they are thoroughly isolated and air gapped. The problem that I have seen with my own eyes is that XP isn’t just in use on such systems, it’s still in widespread use on clinicians PCs too. Speculation. Just getting the patches out on (maybe) unaffected PCs (this malware can lie dormant for weeks) will I suspect be a manual affair as you can’t risk putting a machine back on the network. Problem is if they’ve disabled the USB and DVD drives by policy or software you may not be able to do even that (as you have to have it on the network to change the policy). I think I’d just be rebuilding the lot and be done with it. The problem I have seen with line of business software isn’t so much that it won’t work on a particular o/s, but that it will only work in a particular browser. And of course it costs money to put that right. The other thing with obsolete o/ses is that you may not be able to run up to date apps and those apps will have vulnerabilities too. Patch XP and you still have holes elsewhere. EDIT – Ed MS did warn of dire consequences. Each patch comes with an Affected Software and Vulnerability Severity Rating by o/s. I used to put these into a matrix for review by the Security Dept to decide if an accelerated roll out was required for a particular patch. In cases like this Security would have been on to us as MS would have pre warned them. In the case of MS17-010 it’s Critical / Remote Code Execution pretty much across the board link, enough to have alarm bells ringing.

Dave, I admit that it was some years ago and before widespread use of networking anyway, but I wrote data capture programs to take printer output, (actually from non-PC devices) so data could be processed downstream. I guess that is as close to air gapped as it needed to be. Though this was not full air-gapping since there was a one way output only electronic connection to a receiver. In some cases one device acted as a collector for more than one source.Printed output can be scanned into downstream machines  to achieve suitable separation.

I wrote terminal emulation packages that could pretend to be a manual operator working via MMI ports to execute commands into a system to extract or input data, it was orders of magnitude faster than a human operator!  This required some careful system management due to the rapid pace of inputs. As the data source was the only verified one in the business unit it ensured that systems were, for once consistent – yippee!

High value, but not depreciated devices with hard or impossible to upgrade OS systems can only be managed safely if there is a desire to achieve such a ‘breakthrough’.

I understand your comment on patching out of date machines, possible already nobbled machines. Re-imaging is ideal – probably a lot faster and ideally with a modern OS. This is perhaps the only way to achieve the required relief  – if suitable images exist or can be created and if the hardware can support the image. Too many ifs to be a certain option.

With hind sight browser dependant business (non)systems were a terrible idea.

Which will cause the bigger business hit, ditching such rubbish now, or losing their network including the browser dependant crap at anytime?

I was surprised to see that even patched Windows 10 machines still had SMBv1 made available. Is there any advantage in not removing it as an active option? I have unticked it on my machine and that of my wife, so far without an apparent issue. At least one advisory suggested disabling it in this way.

I understand the existence of XP PCs varies by trust.

 

[Ed P]

Richard if I go back nearly 30 years ALL our process control computers were completely air-gapped from the outside world.

Unfortunately (or should I say fortunately) times have moved on, driven by productivity and convenience. Just before this debacle I had to go to my local hospital for scheduled dental work that required a CT scan to show the details of my sinus cavity, and its relationship to the roots of a tooth. The radiography department was a five minute walk from the dental department, but when it was all over the radiographer just pressed a button to wing it all over to the dental surgeon. He did not have to wait for or use a hospital porter, and neither did the dental surgeon. What would have consumed an hour of my time in the old days took maybe 15minutes at the most. I would estimate that through the day the surgeon gets an extra hour of productive time, and saves a couple of hours in hospital porter time.

As it happens I know that this hospital was completely unaffected by the exploit so the embedded device was either patched or attached to a fully patched server. (Visible PCs are all Windows 7).

[edit] There is a fair chance that the CT scanner uses an embedded Linux device rather than Windows and Samba would be the normal interface medium.

[Tippon]

Dave Rice wrote:

Ed MS did warn of dire consequences. Each patch comes with an Affected Software and Vulnerability Severity Rating by o/s. I used to put these into a matrix for review by the Security Dept to decide if an accelerated roll out was required for a particular patch. In cases like this Security would have been on to us as MS would have pre warned them. In the case of MS17-010 it’s Critical / Remote Code Execution pretty much across the board link, enough to have alarm bells ringing.

Would the individual NHS trusts have received the warnings if they were no longer paying for XP support though? It’s understandable that MS would think that XP was gone, as an NHS trust would be expected to keep its software up to date, so the warning may not have even been sent to them. I’m obviously not blaming MS here, just whichever muppet at some level in the food chain who decided to penny pinch by cutting support for in use systems  :wacko:

[Dave Rice]

This is publicly available information issued in advance of the patches, all you need is a (free) Microsoft Account. Then go here – use IE or Edge – and set your preferences with options to receive email notifications such as this:

The Comprehensive Updates version serves as an incremental supplement to Microsoft’s Security Notification Service. It provides advance notification of upcoming security bulletins and timely notification of any minor changes to previously released Microsoft Security Bulletins as well as notification of new or revised Security Advisories. These notifications are written for IT professionals and contain in-depth technical information.

Anyone doing dedicated IT Support, whether in house or not, will know of this service. Larger organisations will have a more in depth relationship with MS which was the case where I was. In the normal course of events we (the outsourced Technical Dept) compiled the information for discussion with the customers Security Dept. in advance of Patch Tuesday and dealt with the roll out process, Change Control etc. In cases such as this Security would pre-empt that by saying we want MSxxxxx rolled out to an accelerated timetable and may attend Change Control meetings with us so there was no doubt in peoples minds that this was a priority (Change Control can by nature be very conservative).

There is no excuse whatsoever for any IT Dept to say they don’t know what patches are coming up and what they are for and I doubt that is where the problem lies.

A quick explanation of Change Control. If you wish to change specific parts of the infrastructure or over a certain percentage of the estate you must get permission from the Change Control board. You need to present your implementation plan in a highly formal way and also include the plan for roll back in the case of problems arising. Before you even attend the board you need to get sign off of those plans by certain high level technical and production representatives. I’ve spent many a happy hour before cut off chasing some very senior people. At the board you present your plan which is then discussed and you either get the go ahead or are told to have a rethink because they don’t like X, Y or Z. I used to hate it as you get some very senior (career threatening level) types attending or listening in and sometimes you have to be robust in defending your corner.

And you thought it was all button pressing ?

[Ed P]

“And you thought it was all button pressing ?!

It used to be pretty much that, except that a lot more info was given up front, whether you wanted it or not. Times have changed!

[Richard]

Ed P wrote:

Richard if I go back nearly 30 years ALL our process control computers were completely air-gapped from the outside world. Unfortunately (or should I say fortunately) times have moved on, driven by productivity and convenience. Just before this debacle I had to go to my local hospital for scheduled dental work that required a CT scan to show the details of my sinus cavity, and its relationship to the roots of a tooth. The radiography department was a five minute walk from the dental department, but when it was all over the radiographer just pressed a button to wing it all over to the dental surgeon. He did not have to wait for or use a hospital porter, and neither did the dental surgeon. What would have consumed an hour of my time in the old days took maybe 15minutes at the most. I would estimate that through the day the surgeon gets an extra hour of productive time, and saves a couple of hours in hospital porter time. As it happens I know that this hospital was completely unaffected by the exploit so the embedded device was either patched or attached to a fully patched server. (Visible PCs are all Windows 7). [edit] There is a fair chance that the CT scanner uses an embedded Linux device rather than Windows and Samba would be the normal interface medium.

I am aware of the vast improvement that has taken place over the past few years. When I had back trouble back in 2002 I was told to take some pills and go away. In 2012 by which time I was not able to walk unaided I had MRI scans and, not only were they available to the on site staff, I was seen at a hospital 20 miles away who had full access to the results. Within 24 hours of an operation I could walk again. I was given a copy of the MRI results for a later back scan which preceded a second spinal operation. I am converted to making information available What I was saying was that 30 years go networking issues, i.e. no network, could be by passed. such abilities still could avoid the issue of not networking vital but fragile hardware/software.

I believe that most of my trust’s hardware is (a) suitably modern and (b)  tended by a switched on IT department. They were pre-emptive on Friday and I suspect have been busy ever since.

PS, every possible source of information has even sent me data on this issue and I am not signed up for any special treatment. Loads of the messages had links into further details.

PPS, it is time to black block list all software crapware vendors who mandate only one browser ever be used with their crapware until such time as they make their offering agnostic. As for insisting on the use of IE6 or something else a dinosaur sat on or used when it was at school, bankrupt the stupid XXXXs with extreme prejudice. There was an offering a little while ago that could ape obsolete software while running on secure, usable hardware/software, was it Browsium and is it still available? A quick check suggests it is still offered and its web site suggests it would have been of some interest to those stuck with crapware.

Edited to correct a very unfortunate typo and add in a few missing items.

[Ed P]

I accept that there are always crude ways of air-gapping such as using a USB stick or even printing.

Forgetting for the moment the horrendous control problems with sticks, if you go back to manually transporting all the data you lose a lot of productivity. Mandating air-gapping everything deemed critical brings its own (I think bigger) issues.

There are of course ways of accomplishing pseudo-airgapping (for example a crude method could be interposing a secure Linux box with rigorous rules on file transfers). However these ‘solutions’ cost money and add complexity. Better I think to address the root cause and get the NHS funding and targets sorted out.

As said earlier asking a Hospital Administrator to choose between drugs and PC upgrades was an impossibly hard requirement.

(It has always been one of my biggest bitches about the UK Government/Civil Service, they do not seem to understand the differences between expense, depreciation and capital.)

[JayCeeDee]

Edited to correct a very unfortunate typo ……………………. :wacko: :yahoo:  

 

I was going to ask if it was a really lower back problem!!

[Richard]

Ed P wrote:

I accept that there are always crude ways of air-gapping such as using a USB stick or even printing. Forgetting for the moment the horrendous control problems with sticks, if you go back to manually transporting all the data you lose a lot of productivity. Mandating air-gapping everything deemed critical brings its own (I think bigger) issues. There are of course ways of accomplishing pseudo-airgapping (for example a crude method could be interposing a secure Linux box with rigorous rules on file transfers). However these ‘solutions’ cost money and add complexity. Better I think to address the root cause and get the NHS funding and targets sorted out. As said earlier asking a Hospital Administrator to choose between drugs and PC upgrades was an impossibly hard requirement. (It has always been one of my biggest bitches about the UK Government/Civil Service, they do not seem to understand the differences between expense, depreciation and capital.)

In about 24 months I see a whole new set of headlines as the gravy train now being stocked with fresh fuel derails and the gravy turns out to be something else.

Can anyone tall me why SMBv1 is still ticked to be available in Windows 10?

[Ed P]

“PPS, it is time to black block list all software crapware vendors who mandate only one browser ever be used with their crapware until such time as they make their offering agnostic. As for insisting on the use of IE6 or something else a dinosaur sat on or used when it was at school, bankrupt the stupid XXXXs with extreme prejudice. There was an offering a little while ago that could ape obsolete software while running on secure, usable hardware/software, was it Browsium and is it still available? A quick check suggests it is still offered and its web site suggests it would have been of some interest to those stuck with crapware.”

Dave could give a definitive response (if he is allowed), but IIRC nearly all IE6 requirements in the UK were as a result of Civil Service failures to spend the money to update XP era software that was written to Civil Service specs  but  hard-coded with OLE controls. i.e. it was caused by clueless Mandarins reporting to clueless Ministers failing to grasp that software like hardware depreciates over time.

[Dave Rice]

Legacy compatibility. Legacy is the thorn in the side of Microsoft. MS tried ditching legacy with W8 for ARM, look where that ended up.

There is a Microsoft blog from 2015 “The Deprecation of SMB1 – You should be planning to get rid of this old SMB dialect”.

[Richard]

I removed SMBv1 yesterday for both of the machines in the office, others will follow when the portables are next woken up.A check of the WHS found no trace, I hope that is a good sign?

Perhaps IE6 was a passable idea in when, in 2006 or before? However, the world has moved on so it is clearly time for action on that thorn and I really hope MS show some armour covered action to break that and the SMBv1 issue giving a cut off date in the very near future.

I gave the issue of e.g. scanners, MRI, CT, and a number of other highly expensive pieces of hardware, etc. that were, in 2010 using XP since then much water has flowed under the bridge and asking around most of that era hardware has been replaced. The philosophy of not even notifying IT of such items, let alone getting them patched may well still apply, cultural changes are almost impossible to achieve. The device I spoke about had no patches applied in 6 years of use and IT were not even ware of its status, the culture rode again.

Our GP reported no issues this morning and confirmed that the bag of nails they had been using (running mandated software) had been replaced. The new system is less than a year old with an upgraded support package. I have to trust that package included update management…

[Ed P]

Actually Richard if you search on CT scanners it is very hard to find out exactly what embedded operating system they are using.  I do not think you can blame any technician for not knowing what is in the box any more than you can blame a PC user for not knowing they have Linux in their router. IF (deliberate emphasis) there is the unlikely event that Windows CE was used in a CT scanner, no-one would know.

I would bet that Siemens use their OS and that Toshiba use a Linux variant, but it is anyone’s guess what GE use. However you can be pretty sure that they all use smb as a generic interface, and I would bet big money that until a year or so ago it was the generic smb1 in their firmware. I would further guess that firmware upgrades (if only for medical insurance reasons) are under the strict control of the CT provider and their maintenance agreements.

My additional guess is that any PC equipment (none are shown in any used CT equipment purchasing list)  is wired alongside the CT scanner and used as a comms device to interface to the hospital network. Just whose inventory that appears under is anyone’s guess but I’ll place a small bet that it is not office systems. I’ll also place a small bet that it is very low down in the pecking list for an upgrade.

All speculation I’ll admit but based on known facts and reasonable extrapolations. Bottom line, do not blame the hospital techs. If you must blame anyone then blame Hunt and his inflexible targets.

[Ed P]

Update I managed to grab the workstation service manual for a Philips CT scanner. I have no idea what OS it runs from this documentation. I would defy any IT Manager to risk putting his paws on what appears to be a fully integrated server, DASD and Process Control computer whose main access is direct to the Hospital’s database system. I could probably manage to break (as in destroy) something in a system like this but I would not risk trying to do anything constructive such as patching it. The Philips machine is to all extents and purposes a black box that happens to contain a computer. I stand by my earlier speculation that any firmware updates are OEM only.

 

[Richard]

Ed P wrote:

Bottom line, do not blame the hospital techs. If you must blame anyone then blame Hunt and his inflexible targets.

Ah yes blame culture, Hunt is a good target after all in 2014 and 2015 instructions were issued by him to remove XP and bring systems up to date.

I  introduced the fact that windows drove some equipment,. In the case to which I specifically referred, it was not  MRI or CT scanner devices but equally expensive kit – that also had a 10 year amortisation period. The fact that it used unpatched windows and was not software maintained was clear to anyone who looked. The hospital IT staff did not know it was even there until they were told of its existence and its parlous state. Have you no experience of the NHS culture? I have and it can be a dangerous issue when things like this happen always protect the guilty at all cost.

I wonder why your hospital and mine as well, along with our GP’s system were updated in time? Could they have  followed Hunt’s inflexible targets?

It is not always the case that PCs are shown on manifests, they can be treated as part and parcel of the installation supplied by the system integrator, sometimes the chassis is truly built in.

I have been guilty in the past of slipping in terminals to avoid the internal IT police, (in another country, a long and partially political issue). However, they were completely stand alone and had no interface to any other systems. So risks were non existent. Networking was not an option or needed.

[Ed P]

It is easy for a politician to speak out of both sides of his mouth. Politicians are well practised at buck passing or giving orders while simultaneously saying ‘No Extra Money’.

My Hospital Trust invested wisely and avoided the IT problems but was placed in ‘special measures’ for ignoring budget constraints. Maybe yours was as well Richard!

Anyway to turn to less contentious items and get the taste of Hunt out of my mouth; one piece of good news was that some Brit probably accidentally saved the world megabucks in productivity by stopping the Ransomware worm’s propagation dead in its tracks (at least for a time). Link to hero and his story – this could easily get Slashdotted as I think the individual only has a limited bandwidth.

[Ed P]

If you cannot reach the link above it has all been repeated in Ars.

[Dave Rice]

Going back a few posts to the IE6 issue, I couldn’t possibly comment as I don’t know the definitive answer. But I wouldn’t be at all surprised if the speculation was true.

EDIT just seen Jeremey Hunt trying to totally divert attention away from himself. Blaming Corbyn etc. for voting against the latest RIPA laws and lots of totally irrelevant party political bollocks. Where are the people who can ask the proper questions of these jerks? Even the BBC Breakfast red sofa lot did a better job this morning on the Govt security bod put in front of them.

[Ed P]

I’m not sure that the Beeb has any people who are real technical experts, even their ‘Security Expert’, Rory Cellan Jones only does a half-a*sed bit of analysis – for example he concentrates on XP in this report and fails to mention Windows 8 and Server 2003 – the latter probably being the most important vehicle for propagating the worm.

He also failed to ask Hunt about this, but we can put together the real picture from reports from locations such as the York Trust where 70% of their systems are OK but 30% are down. Either these are all XP and Hunt is lying through his teeth or they have been brought down due to the failure of a much smaller number of critical servers.

[Ed P]

Just in case the Government repeats this morning’s untruth about there being no new variants of the malware there were at least two when they issued their report. link

One was rapidly killed by buying its new hard coded domain, the other worked but failed to encrypt due to a bug.

For the life of me I cannot understand why the Government would want to make us forget about this malware — hang on, wait a moment it could not have anything to do with NHS funding surely!

[Author]

Posts