[clug] [OT] Hoax 'rm -rf' post on serverfault.

Sat Apr 16 02:44:03 UTC 2016

There are multiple interesting points in this:
 - authors not checking sources [multiple media reports]
 - deliberate trolling is alive and well, for many reasons, on technical forums
 - the guy was being a smart-arse, didn’t supply running code, expected people to pickup a deliberate and subtle omission.
 - lots of “Halt and Catch Fire” class faults still exist in production systems. Defending ourselves against "own goals" is still important.
 - ‘rm -rf’ destroying the system is a fault of the security & protection model, not a limitation of the command that needs ‘fixing’.

Hope this is useful to some and entertaining to others.

cheers
steve

> On 16 Apr 2016, at 7:30 AM, Trev wrote:
> 
> http://www.techeye.net/business/man-deletes-his-entire-company
> 
> Regards,
> Trev

Hilarious!!! Feel for the guy, know that sinking feeling when the machine room starts to play “Sounds of Silence”.

I’ve Done Stupid Things as well. Some people really do need Adult Supervision.
Including me, protecting me from myself :)

1500 customers. He was doing well.

In 2011, “Distribute.IT” lost sites for 4,800 customers and all its DNS registrations (>5000). It also had its backups spinning on-line, and wiped.
RAID-5 doesn’t protect you from ‘fat fingering’ (as in ‘I fat-fingered… and deleted everything’)

I haven’t heard of this particular way of executing ‘rm -rf’.
The most usual way is:

intend to type:
	 ‘rm -rf /thing/*’
but actually type:
	‘rm -rf /thing/ *’ [note the space]

That piece goes _straight_ to the pool room!!!

Server fault have a special page dedicated to “Monday morning mistake”.
<http://serverfault.com/questions/587102/monday-morning-mistake-sudo-rm-rf-no-preserve-root>

In Ye Olde Olden Days, you _had_ to have tape backups.
Disks were so small and expensive, keeping a duplicate on-line wasn’t possible.
All your backups were ‘off-line’. Saved a lot of arses, that.

BTW, on that server-fault page, it now starts with:

"This is a hoax"
<http://serverfault.com/questions/769357/recovering-from-a-rm-rf>

[hoax story page, in Italian: 
<http://www.repubblica.it/tecnologia/2016/04/15/news/cancella_l_azienda_per_sbaglio_la_disavventura_tecnologica_di_marco_marsala-137693154/>]

> '' The command that I mentioned in the article is harmless but it seems that almost no one has noticed '', responds candidly Marsala when we ask what is the specific technique that makes the lethal string of code. And he concludes: '' With the inaccuracies that are there in the online comments I write a book '’. 

Other sites have picked up and run the story, pre-hoax warning, too:
<http://www.independent.co.uk/life-style/gadgets-and-tech/news/man-accidentally-deletes-his-entire-company-with-one-line-of-bad-code-a6984256.html>

Meanwhile, story was noted on slashdot, with the poster noting that that as written, the command wouldn’t delete anything much and ‘something feels weird’ about the story. Well spotted!

<https://developers.slashdot.org/story/16/04/14/1542246/man-deletes-his-entire-company-with-one-line-of-bad-code>

This is the command he typed in his ‘question’.
	rm -rf {foo}/{bar}

I read it as others did:
	rm -rf ${foo}/${bar}		# making those strings _shell variables_ ${foo} and ${bar}

Was the guy just trolling, collecting stories for ‘Dummies Horror Stories’ or "just being a dick”?
Don’t know, don’t care. Really good story, though.
[I’m not sure he’ll ever get any help on-line again. has he thought that through?]

If you want a real, monumental cock-up, Microsoft executed best ever turn & tuck with pike and dump in 2009.
Search for "T-Mobile Sidekick outage”. This came from a “Project Pink”. 
<https://en.wikipedia.org/wiki/2009_Sidekick_data_loss>

This later piece is a very believable report on the root cause:
	a management directive to cancel a backup underway before a major firmware change. Oops

[have to wade through a long opener to find the paras about that]
<http://www.roughlydrafted.com/2009/10/15/microsofts-pinkdanger-backup-problem-blamed-on-roz-ho/>

Contemporary report of the outage on ZDnet. Microsoft blames others.
<http://www.zdnet.com/article/microsoft-recovers-most-if-not-all-sidekick-users-data/>

Remind you of Telstra and its many outages, like multiple recent mobile network problems??

I learnt something new:

‘Trash CLI’ is a command line interface to GUI trashcan used by KDE, GNOME & XFCE (freedesktop.org)
Writer doesn’t recommend aliasing ‘rm’ to it, parameters are slightly different (no -R)

Moves files to a ‘trashcan’, allowing recovery. Has to be a security nightmare waiting in there :)
<https://github.com/andreafrancia/trash-cli>

Here’s an entertaining read. The author asks ‘What’s the moral of the story?’
In today’s world, for me it’s OS/X’s “System Integrity Protection” feature:
	it’s not a _perfect_ sandbox, but it’s a great step towards one. The system protects itself
	In the Linux world, the NSA-written “Security Enhanced Linux” (SELinux) uses ‘capabilities’ to defeat ‘super-user’ omnipotence.

<http://www.ee.ryerson.ca/~elf/hack/recovery.html>

--
Steve Jenkin, IT Systems and Design 
0412 786 915 (+61 412 786 915)
PO Box 48, Kippax ACT 2615, AUSTRALIA

mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin