Posts archived in Gadgets

Recommendation for Dev & Workstation Laptop: Lenovo W520

Where to get: eBay (recommended), or Lenovo Website.

Where did I get mine? eBay.
Why eBay? About 40% the price of the same model on the Lenovo Australia site, including shipping.
What eBay Seller? bunnypan

What specific configuration?

This one: http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=140531673940

Or, summarised:
– Core i7 2720QM 2.2Ghz (Quad Core w/ Hyperthreading). Turbo mode = 3.3Ghz NB: This is a “Sandybridge” CPU – latest, fastest as of May 2011
– 8GB RAM (2x 4GB) NB: 2 free slots for adding more ram. See below
– 1920×1080 Matte Screen. NB: IPS Screen. Best colour quality.
– 500GB 7200 RPM SATA HDD.
– nVidia Quadro 1000M w/ 2GB Video Memory, and Optimus.
– 3 year (1 year battery) RTB Warranty.
– Intel Centrino Advanced-N 6205 2×2 Wifi card. NB: Can add 3G Modem

What did it cost you? USD$1598 + USD$149 shipping to Australia.
With the prevailing exchange rate – this worked out as just over AUD$1600 on my card, including conversion fees.

What else did you do to it?
Upgraded RAM – two more 4GB DDR3 DIMMS for $55 each. ($110 total)
Replaced HDD – OCZ Vertex 2 250GB SSD. (~$499)
Added a ThinkPad Mini Dock Plus Series 3 (170W) Model 433830U – USD$240 + shipping.

Why the RAM?
Why not, it was only $110 more, and more ram is almost always useful.

It’s been years since people started pointing out how Facebook and other sites are encouraging bad security habits. Yet Facebook still continues to encourage handing over really private credentials that should never be shared. 

Obviously it’s working, two of the folks I’m friends with on Facebook have used it recently.

failfailfail

This little gem I found in the sidebar of my profile.

Everywhere I go on Facebook there’s a prompt asking me to hand over passwords for email and other services.

image 

People have their online identities (not to mention anything important: like internet banking services) associated with these email addresses – why on earth would anyone willingly hand those details over?

Oh, that’s right, there’s a little blue padlock and a nice reassuring “Facebook won’t save your password” message. We should all rest easy knowing the bright sparks at Facebook have our best intentions at heart. Because no-one has ever accidentally turned on data logging. Especially not anyone working for a big reputable and trustworthy totally-not-evil company.

Carry on then.

I’d been trying to do a little reporting in RavenDB and needed to do a group several items.

The secret is that you can group into a new anonymous type, which I found out about on David Klein’s blog posting a few years ago.

So, my Reduce statement looks something like:

Reduce = results => from result in results
                    group result by
                        new
                            {
                                result.Date,
                                result.Application
                            }
                    into g
                    select
                        new
                            {
                                g.Key.Date,
                                g.Key.Application,
                                Count = g.Sum(x => x.Count)
                            };

I was surprised it worked so easily once I knew the secret ingredient.

RavenDB is a document database, and unlike a relational database (like say Microsoft SQL Server) – you can’t do your usual group-by type queries quite as simply.

Consider this scenario: I have a series of documents showing when users have performed certain actions within an application.
I want to produce a unique count of users, and a count of Application starts.

public class StatisticEntry
    {
        public DateTime Timestamp { get; set; }
        public string UserId { get; set; }
        public string Action { get; set; }
    }

 

My documents will look something like:

{ "Timestamp": "2010-01-01", "UserId": "Able", "Action": "Open"}
{ "Timestamp": "2010-01-01", "UserId": "Barry", "Action": "Open"}
{ "Timestamp": "2010-01-02", "UserId": "Barry", "Action": "Close"}
{ "Timestamp": "2010-01-03", "UserId": "Barry", "Action": "Open"}
{ "Timestamp": "2010-01-03", "UserId": "Charlie", "Action": "Open"} 

In SQL Server, I would do something like:

SELECT UserID, count(Timestamp) as Count
FROM StatisticEntries
WHERE Action=”Open”
GROUP BY UserID

This would give me:

Row # UserID Count
1 Able 1
2 Barry 2
3 Charlie 1


For RavenDB instead I need to use a Map-Reduce query. The basics of this are that the first part (the Map) runs over the documents, and finds the matches. The reduce then takes the found matches, and summarises (reduces) the result set.

To make this magic happen, we need to know the results we want out at the end, and define this as a class:

public class StatisticResult
{
    public string UserId { get; set; }
    public int Count { get; set; }
}

We then create our index like so, adding a reference to Raven.Abstractions.dll

public class StatisticsUniqueUsers : AbstractIndexCreationTask<StatisticsEntry, StatisticResult>
{
    public StatisticsUniqueVisitors()
    { 

        Map = docs => from doc in docs
                      where doc.Action == "Open"
                      select new
                                 {
                                     UserId = doc.UserId,
                                     Count = 1
                                 };
        Reduce = results => from result in results
                            group result by result.UserId
                            into g
                            select new
                                       {
                                           UserId = g.Key,
                                           Count = g.Sum(x => x.Count)
                                       };
    }
}

You then feed this class to RavenDB using this bit of magic (This took me a while to find):

Raven.Client.Indexes.IndexCreation.CreateIndexes(typeof(StatisticsUniqueUsers).Assembly, store);


NB: This will result in scanning this entire assembly for indexes. All of them will be created at that time. If any are invalid, you’ll get exceptions.

The CreateIndexes method should be called when your application starts – but be aware that you may get stale results temporarily while RavenDB produces the index.

So, Let me break the index class we created above:

The Map gives us a list of matching documents, with the intermediate results being:

{ "UserId": "Able", "Count": 1}
{ "UserId": "Barry", "Count": 1}
{ "UserId": "Barry", "Count": 1}
{ "UserId": "Charlie", "Count": 1}

The Reduce then summarises these values and gives us the summarised values:

{ "UserId": "Able", "Count": 1}
{ "UserId": "Barry", "Count": 2}
{ "UserId": "Charlie", "Count": 1}

…so, we then have this index, which gives us a per-user count. But we just wanted unique users and total Opens?  Quite right.. It’s a little bit of a round trip, but we get there in the end.

We get the results by performing two queries:

int uniqueUsers = _documentSession.Query<StatisticsEntry, StatisticsUniqueUsers>().Count();
int totalOpens = _documentSession.Query<StatisticsEntry>().Where(s=>s.Action=="Open").Count();

The biggest ‘wtf’ for me was how to actually load the indexes into RavenDB – since there’s no apparent functions to do so on the IDocumentStore interface.

0 comments

Leaving Massive

For the last three years (give or take a month or so) I’ve been working with the great folks at Massive.
Time has come for me to move on and find other great opportunities.

If you know of any interesting .NET Jobs opening up in Sydney (or Melbourne), please let me know.
I’ll be available to start from about the 16th of November. Update: I’m no longer looking for work, thanks.

Things I’m currently interested in:

  • Playing with cool gadgets: Android & Windows Phone 7
  • Multithreading & Parallel Processing in .NET 3.5 and 4.0. (ParallelFx, Rx, etc)
  • IPTV / Streaming Media Delivery
  • ASP.NET MVC

Things I’ve been working with most recently:

  • ASP.NET Web Apps (.NET 3.5/WebForms)
  • RESTful WCF Services
  • Building Custom Serialisation/Deserialistion Adaptors & APIs for third party services.
  • Build Management

To all the folks at Massive: It’s been a great ride; Working with some of the smartest folks I know. Thanks for all the fish.

(Originally Posted by me on AuTechHeads, April 27th, 2010, and preserved here)

Why Cloud Computing Services have huge stumbling blocks to their adoption for the projects I work on.

One of the projects I’m working on has a need to switch to an service bus / message queue system.

We’re after something that’s fairly light-weight. Ideally something we can package into our existing distribution and manage the configuration as part of our existing application’s configuration.
We also need some level of reliability – we’re not expecting clients to go yanking servers out of the rack, but if we send a message, we want to know that it’s going to be delivered.

A few people have suggested Cloud based queue systems as a potential solution. Amazon SQS, Azure AppFabric, and Linxter have all been mentioned a few times.

Unfortunately, no cloud solution is going to pass even a preliminary inspection.

When you use a cloud based architecture, you get to offload some of the responsibility of ensuring the solution is up and running. But at the same time you take a big dependency on the security and stability of not just that cloud provider’s infrastructure – but the entire route from your client to the provider.

If some script kiddie decides that today’s the day he’s going to packet Facebook, and your cloud provider happens to be on the wrong side of a congested router, well – your solution better work fine without it’s message queue.

Any time you add another provider, it complicates the solution. Even if the configuration of the service is as simple as adding a DLL, and two lines of code.

The complexity comes not from configuration files or deployment instructions – but in numerous other peripheral things. Just getting someone to hand over a corporate credit card to pay for the damn thing, even if it IS only pocket change is in and of itself a huge ordeal.  If it’s an ongoing cost, then it’s often nigh on impossible.
Lets not even get into the things you need to do to get firewall ports opened to be able to communicate out, the need for complying with the EU Data Protection Directive, or any concerns a client has about what data is passed over the internet.

Unless there’s a particularly good fit for a client, I doubt we’ll be looking much at cloud based services in the near future.

(Originally posted by me on AuTechHeads, April 26th 2010, and preserved here)

Why a fancy resume is useless if you have no enthusiasm for technology.

One of the things rarely discussed in guides on how to get a job in IT is enthusiasm for technology. I am of the opinion that first and foremost, you need to be a technology geek if you want to work in IT.

Don’t confuse being a technology geek with being the stereotypical pimply-faced, pale skinned, greasy haired dweeb. I talk simply of people who have an innate understanding of some area of IT. The kind of person that hears about some new thing and gets a little (or lot) excited.

Personally, I can’t understand why would anyone choose a job in IT if they didn’t like tech.
It would be like me choosing a career in marketing or interior design. Areas in which I really have zero interest.

Yet when interviewing candidates for software development positions, I find far too many of these people. They tend to express no particular interest in any part of software development in particular, or technology in general.
My only conclusion is that they are in IT because it’s a reasonably well paying job.

The not-so-funny thing about it is that these people have fantastic resumes, with years of experience across all sorts of tehnologies. Often they’ve worked for big name companies.

Perhaps these people work well in development teams where there’s a large corporate structure that treats developers like cogs. But for smaller development shops – you really need your wits about you.
Situations inevitably arise where nobody on the team has experience with some particular technology.

A technology geek will be able to solve the problem (either on their own, or with the assistance of the team) and draw on their own experiences gained while reading about or experimenting with other related technology.

That, in my opinion is why you must be a technology geek to work in IT.

So, readers: What is your opinion – is there a role for non-tech-geek folks in IT?

Atlassian already have some documentation on how to integrate IIS and JIRA.

Unfortunately it requires installing some ISAPI components, and a whole lot of fiddling around.

I wanted to see if I could get Application Request Routing to do the same job. Turns out, yes, you can – here’s how.

1. Make sure JIRA is installed and working on your server.

Let’s say that it’s at http://example.com:8080/

I want to access JIRA via: http://jira.example.com/ – but IIS7 is already using port 80 on that server.

2. Alter your conf/server.xml file in JIRA.

Find the /Server/Service/Connector element, and add two attributes:

proxyName=”jira.example.com”

proxyPort=”80″

The Connector element should now look something like

 <Connector port="8080" enableLookups="false" proxyName="jira.example.com" proxyPort="80">

3. Restart the JIRA Service.

4. Install, if you havn’t already, Application Request Routing 2.0, along with URL Rewriting 2.0

5. Enable Proxying on ARR:

  • From the IIS7 Console, click on {ServerName}.
  • Open Application Request Routing.
  • From the Actions pane on the right hand side, Select  ‘Server Proxy Settings’
  • Check ‘Enable Proxy’
  • Set HTTP Version to ‘HTTP/1.1′

6. Add a new site ‘jira.example.com’, with bindings for http://jira.example.com

7. Add a new URL Rewrite Rule for jira.example.com

  • From the IIS7 Console, click on jira.example.com
  • Open URL Rewrite
  • From the Actions pane on the right hand side, select ‘Add Rules’
  • Choose ‘Blank Rule’
  • Set Match Rule to:
  • Requested URL Matches the Pattern
  • Using Regular Expressions
  • Pattern: (.*)
  • Ignore Case: checked
  • Set Action to:
  • Action Type: Rewrite
  • Rewrite URL: http://example.com:8080/{R:1}
  • Append query string: checked
  • Stop processing of subsequent rules: checked

8. Now, with any luck – you should be able to access JIRA via http://jira.example.com  - if not, something isn’t set correctly.

Setting up Fisheye is almost as simple.

Say Fisheye is set up on http://example.com:8060 and I want to access it via http://fisheye.example.com

Repeat steps 4-8 above, substituting ‘fisheye’ for ‘jira’, and then verify you can access fisheye from http://fisheye.example.com

If you’re also doing .NET Development, or have .cs/.aspx/.asmx files in your repository, then you’ll also need to do the following.

Edit the web.config for fisheye.jira.com

Add the following to just before </system.webServer>

<handlers>
 <remove name="WebServiceHandlerFactory-ISAPI-2.0-64" />
 <remove name="WebServiceHandlerFactory-ISAPI-2.0" />
 <remove name="PageHandlerFactory-ISAPI-2.0" />
 <remove name="PageHandlerFactory-ISAPI-2.0-64" />
 <remove name="PageHandlerFactory-Integrated" />
 <remove name="WebServiceHandlerFactory-Integrated" />
 <remove name="SimpleHandlerFactory-ISAPI-2.0-64" />
 <remove name="SimpleHandlerFactory-ISAPI-2.0" />
 <remove name="SimpleHandlerFactory-Integrated" />
 <remove name="CGI-exe" />
 <remove name="ISAPI-dll" />
</handlers>
<staticContent>
 <mimeMap fileExtension=".cs" mimeType="text/plain" />
</staticContent>
<security>
 <requestFiltering>
  <fileExtensions>
   <remove fileExtension=".config" />
   <remove fileExtension=".csproj" />
   <remove fileExtension=".cs" />
   <add fileExtension=".cs" allowed="true" />
   <add fileExtension=".csproj" allowed="true" />
   <add fileExtension=".config" allowed="true" />
  </fileExtensions>
 </requestFiltering>
</security>

If there are any additional filetypes that are in your Fisheye repository that generate 404 errors when navigating, then add them to the fileExtensions section. First as a ‘Remove’, and then an ‘Add’ with allowed=true. You’ll also need to probably add a mimeMap entry too.

Thanks to @OhCrap for the pointers on enabling .cs serving with IIS7.

A few months ago for some unknown reason the Output pane in Visual Studio stopped displaying output from my application.

I’d get the build notices, exceptions, and thread/process exit information but any calls to Debug or Console to output information wouldn’t display.

tickprogramoutputIt turns out that you can de-select “Program Output” – and somehow it’d become deselected. Even now it still turns itself off, apparently by random.

Right clicking in the Output pane should let you re-select that value.

Edit: This post was linked from Stack Overflow – you might want to check back there for more/better discussion about the topic.

Recently Jeff Attwood wrote about how they are using ELMAH to get more information about the types of errors occurring in Stack Overflow.

Effectively ELMAH is designed as a ‘drop in’ fault capturing system for ASP.NET. It works really well there, and for many situations you can get along just fine without even needing to recompile your application (it does need some editing of the web.config though).

I wanted a way to capture more detail about the faults occurring in our dev and production environments, especially when working with WCF – since a lot of error detail tends to be hidden, or is difficult to reproduce.

Dropping in ELMAH into a WCF application will by default mean you miss the vast majority of errors – WCF swallows the error, and doesn’t let it get back up to ASP.NET.

There’s two ways you can go about fixing this:
Side Note: If you’re not hosting WCF in ASP.NET, then Option 2 may not be directly possible for you without some modification.

#1 – Wrap everything in try/catch blocks (if you didn’t already) and sprinkle this line around everywhere:

Elmah.ErrorSignal.FromCurrentContext().Raise(YourExceptionHere);

#2 Add a HttpHandler, and Decorate your Service(s) with an Error Handling attribute.

I borrowed the ServiceErrorBehaviourAttribute code from somewhere else, and I can’t find the source of it at the moment. Effectively this was so I could manipulate the HTTP Status Codes going back to the client when there was an error. It just so happens that this is a great way of capturing Exceptions and sending them to ELMAH at the same time.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.ServiceModel;
using System.ServiceModel.Dispatcher;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.Collections.ObjectModel;
using System.Net;
using System.Web;
using System.IO;
using Elmah;
namespace YourApplication
{
	/// <summary>
	/// Your handler to actually tell ELMAH about the problem.
	/// </summary>
    public class HttpErrorHandler : IErrorHandler
    {
        public bool HandleError(Exception error)
        {
            return false;
        }

        public void ProvideFault(Exception error, MessageVersion version, ref Message fault)
        {
            if (error != null ) // Notify ELMAH of the exception.
            {
                Elmah.ErrorSignal.FromCurrentContext().Raise(error);
            }
        }
    }
	/// <summary>
	/// So we can decorate Services with the [ServiceErrorBehaviour(typeof(HttpErrorHandler))]
	/// ...and errors reported to ELMAH
	/// </summary>
	public class ServiceErrorBehaviourAttribute : Attribute, IServiceBehavior
    {
        Type errorHandlerType;

        public ServiceErrorBehaviourAttribute(Type errorHandlerType)
        {
            this.errorHandlerType = errorHandlerType;
        }

        public void Validate(ServiceDescription description, ServiceHostBase serviceHostBase)
        {
        }

        public void AddBindingParameters(ServiceDescription description, ServiceHostBase serviceHostBase, Collection endpoints, BindingParameterCollection parameters)
        {
        }

        public void ApplyDispatchBehavior(ServiceDescription description, ServiceHostBase serviceHostBase)
        {
            IErrorHandler errorHandler;
            errorHandler = (IErrorHandler)Activator.CreateInstance(errorHandlerType);
            foreach (ChannelDispatcherBase channelDispatcherBase in serviceHostBase.ChannelDispatchers)
            {
                ChannelDispatcher channelDispatcher = channelDispatcherBase as ChannelDispatcher;
                channelDispatcher.ErrorHandlers.Add(errorHandler);
            }
        }
    }
}

Once you’ve added that, then it’s just a matter of decorating your Service like so:

    [ServiceContract(Namespace = "http://example.com/api/v1.0/")]
    [ServiceErrorBehaviour(typeof(HttpErrorHandler))]
    public class MyServiceService
    {
      // ...
    }

…and then making sure ELMAH is added as a reference, and adding it’s entries to your web.config.

Then you’ll be getting a whole stack of errors you otherwise may not have seen.

It’s also possible to log exceptions from higher up the chain (eg Databases, Files, etc) by using the line of code from Option 1.

Issues

Whilst ELMAH is great for capturing information about the request, I havn’t yet found any way to capture the original HTTP Request – this would be the ultimate goal for me.

It’s also not particularly easy to capture additional information (such as database records, or objects in cache, etc) without rolling your own copy of ELMAH.

All in all though – for a few minutes work, it’s one additional way to capture errors that your existing code may not be able to.

Yes, ELMAH even captures errors (in most situations) when your WCF services can’t start up (eg your fubar’ed some attributes).

Hope that helps.

NB: Use this code at your own risk, don’t blame me if it brings down your multi-million-dollar-per-hour application and causes you to go bankrupt.