I was away over the Easter long weekend and didn’t get to see the whole ‘amazon fail’ thing happen. Actually, I didn’t even really find out about it until today.

The story goes that sometime on Saturday Amazon suddenly stopped returning pro-lesbian/gay/bi/transgender (LGBT) material in search results – in some cases returning anti-LGBT material instead.

I’ll leave the morality of why this was a bad thing alone – other far more eloquent folks have written about it already – and instead offer my view of just how this can happen.

Some people have commented that they simply cannot believe this was a mistake, and that it must have been a deliberate act by someone at Amazon. Others have said that they cannot believe there isn’t more checks and balances against this sort of thing happening.

I don’t have any special knowledge of the specifics of this situation, but I still believe that it’s within the realm of possibility that this could have been a completely unintentional side-effect of another change.

Here’s a hypothetical situation which could explain what happened.

Let’s suppose that someone doing database administration on the product management side of things for Amazon has the ability to make direct, or semi-direct changes to the data in the database. 

Let’s also suppose that in order to prevent human error, the system that Amazon Employees use has a delay of (say) one hour before changes are sent to the actual production database.  That continual one-hour buffer would have some sort of checks to see if there were unusual behaviour – such as updates to more than some reasonable number of records at any one time.

It’s not unlikely that Amazon have a rather complex categorisation system that allows products to be placed in any number of categories, and that categories can belong to other categories. 

From here it’s but a hop-skip-and-blunder into someone updating a series of categories to ensure they’re marked as adult. A command to mark any category with ‘lesbian’ and ’sex’ in the title as Adult might seem fairly reasonable if you aren’t careful.  If this is one small series of category updates, it may not trigger any alerts, even though tends of of products are now categorised as Adult by association to those categories. (See Data Normalization Side note below)

Whenever a human is involved in something, there is always a chance for something to go wrong in entirely unexpected ways. The sign of intelligence though is learning from your mistakes and ensuring you do what you can to prevent it from re-occurring.

In Amazon’s case, this might be ensuring that they add a check to see if there is a major difference in the number of Adult products. But no matter how many checks and reviews you have – People will still manage to break things in new and creative ways.

Further Reading:

• Gizmodo: Amazon Deems LGBT Books Too ‘Adult’ for Search, Best Seller Lists
• MediaMemo: Amazon Apologizes for “Ham-fisted Cataloging Error”
• Gawker: Why Amazon can’t just call gay blacklist a ‘glitch’
• Clay Shirky: The Failure of #amazonfail

Side  note: Data Normalization

Because of the nature of computer databases, it’s encouraged (and very efficient) to group common information together, and reduce duplication of data by keeping only a reference to related information.

An example of this might be in a genealogy system – for any particular person, you have information about who the parents are. Instead of storing the Firstname, Lastname, and Date of Birth of each parent – I would instead store the unique identifier for each of the parent’s record.  This act of storing just the unique identifier is part of Normalisation of data. 

When it comes time to display that information about a person’s parents – you look up the record for each parent and retrieve any information needed then. 
This means you also have only one location to update information about any one person – for instance if a person died, you could put the location and date of death on that person’s record. When I then needed information about people who have living parents – I can cross-reference those tables.

Blog’s back up… on a new server too.

Things are moving around a fair bit.

One of the newer traditions at work is that a bunch of us go for pub lunch together on a Friday – have a few beers, and catch up on what’s been going on. A nice way to spend an hour before the weekend.  Our normal haunting grounds are The Dolphin or Trinity Bar, after we gave up on The Porterhouse’s extremely variable (and more often than not just plain bad) pub food.

We heard that The Beresford Hotel had re-opened after a major refurbishment, and was getting rave reviews – so, it was decided we’d try it out.  Our first clue that it wasn’t the best idea should have been that we couldn’t get in for a Friday lunch until some time after January.  But hey – it’s popular, and it’s getting good reviews, and they can get us in on a Thursday – close enough is good enough, right?

The second clue should’ve been when we arrived – it’s a *very* nice place, not so much a pub restraunt as a full service five star italian restraunt that just happens to be attached to a pub. 

Anyhow, we booked for eight people,  called back to adjust to nine, then when the group of us turned up there were ten. A pause from the Maitre’d and they set up another table to accomodate us. Then a few more people turned up – over the course of five or so minutes we ended up with 12, but since we were on one of the benches, we just squished up and the staff set some more places and pulled in another chair.

At first, when there were ten, we were given the A-La-Carte menu – a very long menu with a dazzling array of dishes to be had.  When the Maitre’ D noticed the 12th person arrive, (just as the rest of us had decided what we’d like) our menus were taken back, and switched to a more limited (banquet?) menu.  Of course, as we all joked as the menus were swapped – almost none of the things we’d decided on were on the more limited menu.

Then we noticed the fine print – minimum of two courses: $60. We enquired if we could just have a single course – after all, this was lunch, right? We had to get back to work in under an hour.  The Maitre’ D’s somewhat snarky comment was “Well, that’s new”. But back to the a-la-carte menu we go, more decisions again, and we all decided on a mains.  

Of course we didn’t realise that most of the ‘mains’ weren’t what you’d call mains in Australia – they’re going with the traditional Italian catering of a single named item. So, we’d be expected to order perhaps an Entrée, Salad and a Mains – at the very least.

Perhaps this wouldn’t have been a problem had this been explained by the wait-staff, and had mains not been priced at $30-$60 (Roast Pork Belly: $32, Prime Ribeye: $45 [approx 300g], Crab Tortellini (3): ~$45), with each of the additional courses costing usually $15 – $30 each. 

When the food arrived, I must say that it was exquisitely presented – what you’d expect from a five star restaurant, and obviously not a pub restaurant.

It was obviously as we were presented our meals that we realised why the Maitre’ D made the snarky comment, and that an order of ‘Slow Roasted Pork Belly with Peaches’ or ‘Prime Ribeye’ was just that – with little more than decoration on the plate.

I might have been more forgiving if perhaps the pork belly was particularly great – unfortunately mine was somewhat dry and fairly much devoid of any flavour.

I wasn’t the only one a little disappointed – I know at least one person went and got something else to eat on the way back to work. But some of the group did particularly enjoy theirs – the Tagiatelli and Gnocchi was apparently quite good.

So, all up I think there’s equal parts failure on both ends – it was our fault for not realising it’s not a pub restaurant (despite it being a restaurant in a pub) plus having more people than expected turn up. It was also, perhaps, their fault for not providing particularly good information about their style of service up-front. Our question about could we just order a single course should’ve triggered that information to be provided – at the very least.

Would I go back? Perhaps – but I doubt it. At the very least it would have to be for a dinner, and when I was feeling particularly rich. I wouldn’t budget any less than $120 a head for dinner, excluding any drinks. As it was, it came out to roughly $56/head for us.

A solid ‘meh’ and a half out of five.

Unless you’ve been living under a rock, you’ll surely have heard something about Senator Stephen Conroy’s plan for Cyber Safety. 

There are many parts to the Senator’s plan – but the part  that is causing the most debate is the proposed ‘Clean Feed’ system.  That is: All ISPs in Australia would be required to filter all internet access of certain types of content.

The argument put forth by the Senator and his supporters is that because certain content is illegal, Australians should therefore not be able to access it, and it should be completely filtered out, with no option to opt-out.

While the the original concept of protecting children from potentially harmful material has some merit, the actual plan under consideration has significant issues – not the least of which is how the actual plan would be implemented.

Senator Conroy (AAP: Alan Porritt)

Put simply, what is being asked for can NOT be delivered. Any system which was put in place would need to either completely cut off almost all access to the internet, or would be trivially bypassed with little or no effort on the part of internet-users.

The Senator’s plan calls for ISPs to provide a Clean Feed for all Web traffic. What most people don’t realise The Web is simply one small portion of the type of traffic flowing over the internet.  The plan that is being asked for would only be able to filter Web Traffic.  Other systems such as the type of private networking used in many businesses, and Peer to Peer networks would completely bypass the filter.

Even if we completely cut off all other non-Web traffic (which as I mention below, would cost businesses massive amounts of money), that still would not ensure we have a Clean Feed.  Why? Because of this little thing called Encryption.

Without getting into the technical details of how encryption works, when you enter (say) a shopping site it will ask you to provide certain private information like a credit card number or a password. That information you enter will be encrypted by your computer, and sent to the site. Many would ask “Who cares about encryption?” Well, just stick with me for a minute. 

Encryption is a good thing – it ensures that when you view your online banking site and pay some bills that you don’t have someone slip in a command to transfer some money to someone else. 

Because of the nature of this type of two-party Encryption (also known as Public Key Encryption or PKE), it is not possible for a filtering system to check to see if what you’re accessing is illegal. All details of what you’re accessing is secured against all others viewing it. 

“Ah-hah!” some might say “Just make it so that the government can also decrypt any traffic. You can trust us!”.  Let’s assume for a moment that the Government was completely trustworthy and that every employee responsible for the filtering plan at every ISP was also trustworthy. The very act of adding this sort of back-door has been proven time and time again to be exploitable by not just the Government and authorised users but also any determined hacker.

If you don’t believe me you can go ask the US Government and Pentagon. It was not too many years ago they were starting down a similar path to require all cryptography products to have a back-door that could be used by the Government to view the encrypted content. It failed because the systems could not be made secure and now the US Government mandates the use of PKE for all critical systems that need to access confidential information.  The situation for ‘back door’ encryption systems has only gotten worse since then. Computing power has increased hundreds or thousands of times and now a cheap laptop computer is able to apply astounding amounts of computing power towards breaking encryption.

“So, why does it matter that there’s this encryption stuff?”  you may ask. Well because there is no way for any ISP-level filter to determine just what is being requested when communications are encrypted.

An encrypted connection could be simply someone shopping on eBay for something perfectly innocent like a birthday present.
It could be someone accessing their work’s network from home and preparing a report for a client.
It could be someone making a video call to a see the kids while they’re away on business.
It could also just as easily be someone accessing something illegal. There is no way for a filter at an ISP

Image supplied by enimal

To begin to effectively filter the internet you would need to ban encryption.
If you ban encryption you immediately kill any online commerce, as the ability to transfer information in private like credit card numbers, would be banned.  
Try telling eBay and the hundreds of online shopping sites that they need to shut up shop.
Try telling the major banks that they need to get rid of their online banking sites, and have customers visit a branch or call in for service.
Try telling businesses that while they’re welcome to have multiple offices – any communication between them needs to be unencrypted over public networks.

Banning encryption would mean that any kid with a laptop and a spare minute or two can see all confidential files and information as it is accessed.

Once you make the obvious decision to allow encryption to remain, all ability to effectively filter the internet goes out the window. The only  thing you can do is slow down the speed at which people access the internet, and for what reason?

So, Senator Conroy and all other members of Parliament – please abandon this ineffective and ultimately useless Clean Feed proposal.

Instead, focus on supporting Mums and Dads to educate their children about appropriate and inappropriate use of the internet. Providing support to parents on how to install and use filters on their own computers* would also be a good thing. 

In fact, this home filtering thing is such a good idea that it’s already being done. The Federal government’s NetAlert program is already offering commercial filtering software at no-cost to all Australians.  Never heard of it? Not surprising.  The next logical step from NetAlert is to promote the education of parents on how to install and use these filters we’ve been provided.

* Home filtering, for the record, is the only place a filter can see what content is being accessed, regardless of encryption.  That’s because it gets to see the content before encryption, and after decryption.

Please visit nocleanfeed.com to see how you can voice your opposition to the Clean Feed proposal to those in government.

Image Sources:

#1: http://www.watchingcw.com/cats-and-kittens/ via Google Image Search
#2: http://www.abc.net.au/news/stories/2007/12/31/2129471.htm AAP: Alan Porritt
#3: http://www.sxc.hu/photo/913770 by forwardcom
#4: http://www.sxc.hu/photo/902697 by enimal
#5: http://www.sxc.hu/photo/97150 by wagg66

Wondering why your tweets are getting cut off when they’re apparently less than 140 characters? Well read on…

The reason behind it is a purely technical thing – and mainly due to Twitter’s implemetation of their character counting.

A simple explanation is that some characters are actually ‘worth’ more than others. For instance angle brackets (< and >) are worth four characters and ampersands (&) are worth five characters.  It gets worse when people use characters that arn’t normally on a US-English keyboard – For example that little ♥  is worth EIGHT characters. A little love goes a long way, huh?

If you want to understand the reason for this, you need to know a little about how HTML and other markup languages work.

In the Hypertext Markup Language (amongst others), there are several ‘reserved’ characters (entities) – these are ones used to actually write HTML. The most commonly recognised ones are angle brackets.

If you actually want to use an angle bracket in text – you need to encode it. The way that is done is by using ampersands and short codes or numbers, and a semi-colon.

So, “<” becomes “&lt;” (“lt” being short for less-than). “>” becomes “&gt;”. If you want an ampersand? Well, that’s also a reserved entity – given that it’s used to denote the encoding of other entities – so your “&” actually becomes “&amp;”.

Things get even more complicated when we talk about characters that arn’t typically on a US keyboard. 

Twitter could fix this in one of several ways – the easiest for them would be to alter the character count on the website to show the entity-encoded count, which is what the server uses.

I hope this was helpful to all those folks who I see scratching their heads over the suddenly shorter messages!

It seems that there’s a large number of IT/Digital Media Recruitment companies that are getting a presence on social networks. The unfortunate thing is that, of the ones I’ve heard from, almost none of them actually get the point of social networks.

Here’s a big fricken hint for you guys that DONT get it: Plastering everyone in your target audience with invites, friend requests, etc is a quick road to people’s shitlists.

The point of me being on social networks isn’t so that you guys have a convenient way to find and recruit me, it’s so that I can keep in touch with my friends and other folks I know.

Here’s a few indicators that you’re doing it wrong:

#1 – Following way more people than are following you (eg on Twitter)

This fine person is a great example:
# Following 1,690
# Followers 190

#2 – Being anonymous and using a Company alias instead of your real name.

Seriously, use your real name. Use a company profile only for company specific messages.

#3 – Sending Friend Requests to people you have no association with, and no friends in common.

If you open up my Facebook profile and see that we have zero contacts in common, and you’ve never actually contacted me in any other way – then that’s probably a pretty good indication that you should NOT be clicking that ‘Add to Friends’ link.

That’s all I can think of for the moment.

This was my second year at TechEd, and I was really looking forward to it. There were a lot of great sessions from folks I’d heard from last year – Steve Riley, Neil Roodyn, and Joel Pobar, amongst many others.

This year’s event was held at the Darling Harbour Convention Centre – taking up hall 5, plus both the Parkside and Bayside conference centre. A truly huge area with something like fifteen concurrent ‘main-line’ sessions, a big hands-on-labs area that saw heavy use, a well utilised Exhibitor hall with another two smaller presentation areas and another two ‘open area’ chalk and talk short presentation areas.

At times I was torn between two or even three events – and I only got to do one of the ten or so hands-on-labs events that I’d wanted to do.

In terms of technical content, Andrew Coates and the rest of the team that went into organising speakers and the presentations deserve a medal.  Seriously great stuff there.

The only thing that I’d like to see changed in future TechEd events is to have shorter talks – Say 45-50 minutes. The current 1h 15 minute format is incredibly difficult for all but the absolute best speakers to fill in a useful manner. But even still, sitting on those seats for that length of time gets really uncomfortable. 

I’d like to also like to thank Michael Kordahi for restructuring the Ask the Experts night – the public Q&A thing was a good icebreaker.

Unfortunately, a series of things combined to make me really disappointed with the event this year. Whilst they’re not the technical education side of things, they were supporting the event as a whole.

My first gripe is about the absolute outright failure of the internet access for the vast majority of the event. At a major event like TechEd, people have come to expect internet access. It’s not cheap to do this, but when you are sponsored by companies like HP and Cisco – you’d expect them to be showcasing some latest and greatest networking products here.  For some reason something went wrong, and despite having solid wifi signal in most of the conference centre – I just could                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          n’t get a DHCP lease.

Sure, internet access didn’t kill the event – but it meant it was harder to keep in contact with friends and colleagues who were also at the event, and also meant I couldn’t get the information the presenters talked about and begin playing with it immediately.

The second gripe sounds like I’m a spoilt brat, but I’m going to say it anyway: The catering this year left a lot to be desired.  Whilst last year’s gourmet catering was fantastic, this year’s was pretty abysmal. Especially so when you realise it’s about 500 meters from Haymarket/China Town where there’s amazing food for what I’m fairly sure is probably equivalent in cost at commercial scales.

The catering issues extended to having no vegetarian option for all practical purposes, with the exception of lunch on the last day. Drinks were, as far as I know, entirely non-existent on Thursday – with even  the water fountains in some locations going without being refilled (at least part of the day).  Pity those who forgot, or didn’t have a water bottle to fill.

My third, and final gripe is about the Closing Party. At a technical conference, people tend to have backpacks. It’s not unreasonable to expect that they might fill these backpacks with things like laptops and cameras.  You could also read this as ‘thousands of dollars of equipment’.

This is especially true of technical conferences where you happen to be giving everyone at the conference a backpack when signing up.  So when I’m told that I must hand my backpack over to someone, or else I can’t go in – my first reaction is to question the sanity of the person responsible for coming up with that decision.

My second reaction is a well known Cartman saying – Screw you guys, I’m going home. There is no way that I’ll hand over my laptop and camera gear to a bunch of people whom, whilst I’m sure are well meaning and nice people, have all care but no responsibility over what happens to said equipment from that point on.

As for what was on *in* the closing party? I’ve only heard second hand reports from others – so I’ll leave that for them to comment on.