Living and working in the Internet age

Many VMs keep it neat

2012-01-06T21:34:00.000Z

I've been working a lot on Virtual Machines recently so I can keep all the code and settings for each project I'm working on separate. When I need a desktop dev environment or a web server I just create a new VM!

It sounds overkill to start a new machine for each project but here are several reasons why creating one is a great idea.

Separation - When you have a whole machine for each project it's impossible for one set of tools to get in the way of the other. You can keep the config right for just that project.
Backup - I snapshot each machine before I do something big. The ability to go back to a point in time in one click is incredibly powerful if you use it regularly. I also run an rsync script regularly to copy the VMs to another drive, so you are not relying on one HDD. You just copy one file and you have the whole machine backed up.
Keep your "host" clean and running fast - When you install many apps on one machine they generally slowly get clogged up and confusing. I use OSX as my host and run mostly linux and Win7 in the VMs.
Software versions - I recently needed 2 different versions of the same software for different tasks. Creating a new machine made it so easy.

I started off using VMware Fusion, as I used to work with VMware ESX and was always impressed with it. However I've recently been using VirtualBox more and more. It's free and I haven't found anything I can't do with it yet.

A few tips:

Run dropbox on the machines and keep any files you want to move between machines on there. It's the perfect backed up, versioned shared directory.

You need lots of Ram. I have a Mac Pro with 2 dual core processors which can cope fine with 3 VMs running simultaneously. RAM you can't really do without. I use 10GB which always leaves plenty for the Mac too.

Finding lost attributes in Magento

2011-08-17T13:32:00.000+01:00

We have recently launched our new site www.thistribe.com, a new eCommerce site selling high end equipment to the Military and other members of the security industry.

Using Magento we have been able to very rapidly develop a cutting edge site with the vast array of modules that have been written for it. We already have a good network of developers ready for future projects. with it's recent acquisition by e-bay it looks like it is going to be the e-commerce platform to be reckoned with, so we made a good choice.

Having not worked with Magento before it has been a steep learning curve, but it's very powerful. The database looks incredibly complicated as it uses and EAV pattern. This allows you to extend it easily without schema changes. For example the way the site uses attributes and attribute sets for products is very powerful, all down to EAV.

The Magento database Schema (V1.3)

Since the site went live we have been continually improving things and adding features. One of these tasks was adding a Google Shopping feed. We are using the module from Wyomind This product will generate all sorts of different format feeds from your products in Magento. You can manually edit the templates from within the admin site and even test the output before you generate the files. Pierre, the creator has also been incredibly responsive and helpful in adding features, I can't recommend it highly enough.

We have used several different attributes for product size as obviously all products don't have the same sizing attributes. As each product only uses one of these we just put them all into the template, only the one that has a value will output the tag:

<g:size>{size_head_sealskinz}</g:size>

<g:size>{size_gloves}</g:size>

<g:size>{size_head}</g:size>

<g:size>{size_head_truspec}</g:size>

<g:size>{size_pants}</g:size>

<g:size>{size_jacket}</g:size>

<g:size>{size_shoe}</g:size>

<g:size>{size_socks}</g:size>

This works fine, for most products! We found a few that very strangely had 2 sizes, event stranger was that some hat products had 2 sizes, the correct one for a hat, and another for a jacket.

Obviously something was not right. It all looked fine in the Magento admin site, so it was time to look into the database itself.

We spent a while looking for the relevant records, it's made a little trickier by the fact that all the attributes are stored in attribute sets.

The important tables are:

catalog_product_entity

This allows you to find the entity_id from your SKU.

catalog_product_entity_int

For attributes with multiple options this table will have rows for the entity_id you just found.

eav_attribute_option_value

This is where all the options in an attribute set are. So you look for your text of the attribute in here.

You can see a full list of the actions and SQL used here.

Magento is a very powerful but complex product. It takes time to understand the underlying structure but it's time well spent when you run into problems like this.

iCloud is using Amazon and Azure for storage

2011-06-13T17:40:00.006+01:00

Breaking news, very interestingly Apple's new iCould service seems to be using both Amazon and Azure as backends for it's services. This article on InfiniteApple shows that URL's to content are indeed Amazon and Azure CDN links .

I guess this makes sense although I'm sure some Apple users won't like it, such is their hatred of Microsoft, the idea of making money and relying on the arch enemy might be too much! It will be interesting to see if this is just a Beta thing or whether it will make it into the final release.

It makes sense in that Apple doesn't have the experience of this sort of storage service. It's much more likely to succeed if they use tried and tested services. It does make you wonder what they are storing in their nice new shiny data centers though.

iCloud is the new Mac glue and it looks sticky

2011-06-07T11:45:00.015+01:00

Apple has recently released details of it's new iCloud product. It has been slow to come to the market with a full Cloud offering (it has built new data centers to launch this one) and is significantly behind other players in this field. Google is nearly completely Cloud based, it's upcoming foray into client OS - Chrome will just be away of running a browser, but even Microsoft, a traditional thick client animal, as is Apple, has made more ventures into this space with its Office Web Apps and SkyDrive services.

It seems Apple has a different vision for the Cloud, as the glue that holds its devices together. The focus seems much less on Cloud storage or as an application platform but rather as simply a system to sync application data and devices.

This should not come as a surprise of course as Apple is all about client hardware and software. With the iPhone and iPad now they are assuming that every Mac convert will have at least 2 Apple products, and these need to be backed up and in sync, a logical conclusion.

It's about time, this has been missing, with mobileMe as the only similar offering previously from Apple. Even Steve Jobs in his recent keynote announcing iCloud made fun of it and admitted it wasn't the finest product they had ever released!

iCloud has clearly been well thought out and is deeply app integrated, your apps will use it's services for their individual requirements. App Store and iTunes are major beneficiaries of this, and will keep a history of your purchases and allow you to download them to all your devices. As well as the obvious apps like iCal and Adress book backing up their data, iWork will sync all it's docs as well. iPhones will back up their config and data, including music which should be an end to plugging your iPhone in via USB.

You get the picture, this is the real benefit of iCloud, not only do you have all your data on every device, but if you loose a device you just sign in on a new one and all your data and settings are downloaded instantly over WiFi. In typical Apple fashion it will be largely self configuring, I can believe that.

I think they may have got it really right with their new Photostream service. This basically ensures that all your recent photos are available on every device, irrespective of which device you took them on. This is great for sharing photos as without this sort of service photos end up either not shared or being emailed around. So this looks like a well thought out solution.
Photostream stores your images for 30 days, all devices store your last 1000 images while all desktop and laptop computers will keep the entire history as long as you add them to albums. Nice.

How do you get it? Well it's all going to miraculously appear when you upgrade you iPhone and iPad to iOS5 and your Macs to Lion. Lion will be available through the App Store in July, they have moved away from a physical media route which is great. They are clearly trying to get everyone to upgrade, Lion will be a $30 upgrade, iCould is free and comes with 5GB of storage space. I'm sure there will be an option to upgrade your storage although this hasn't been announced yet.

It's not all white fluffy clouds however, there are some dark ones too. With the retirement of MobileMe we may actually lose some of Apples online services, it's not clear yet. There is a lot of talk on the Net about service like webmail, iWeb hosting, iPhoto web galleries and iDisk. As yet there seems to be no news from Apple on these.

They have announced that you will still be able to use me.com email adresses so it seems strange that they would remove the webmail functionality. I hope that these new Cloud based services will not be a closed shop when it comes to integration with other services. A beta API has been released so hopefully this shows a commitment to opening all the new services up.

It's going to be an interesting autumn for Apple users, I wonder if they will be able to lure me back to iPhone....

Microsoft urges you to abandon Internet Explorer

2011-06-06T18:46:00.003+01:00

Well version 6 of the browser anyway!

It's been the bain of every HTML designer for years, so it's good to see Microsoft publicly encouraging people to upgrade. I wonder how many will stay with IE, it seems that growing numbers are turning to Firefox and Chrome to replace it.

You can see how they are doing at http://www.theie6countdown.com/default.aspx

China and India lead the way in remaining IE6 users. I'm guessing this is directly related to the number of users using Windows Update. Draw your own conclusions from that.

I'm looking forward to not having to worry about supporting IE6 when writing web sites, Google have already announced that they will not be supporting it for their browser based apps after August 1st. They claim that the desired support for HTML5 has led to this decision.

Happy days!

Google post on blog about account security

2011-06-02T16:37:00.003+01:00

Google today has revealed that a cyber attack aimed at top US officials, military personnel and journalists originated from Jinan in China. Although Google said it's systems were not breached, some passwords were fraudulently obtained in this way. Attacks of this sort are on the increase with the hackers aiming to get sensitive information by getting the user to enter their password into a fake website designed to look like the one you are expecting. Known as spear phishing, the technique has reached epidemic proportions.

In my recent post I encouraged you to use the free 2 factor authentication Google provides, yesterday Google published this on their official blog. 2 Factor authentication and a strong password are the best defence against someone hacking your account. Of course you need to make sure that you are entering your details into the correct website, simple!

If you use Gmail...

2011-05-28T19:48:00.008+01:00

I use this feature in Gmail so much I forgot it wasn't standard! It's a simple and free feature in the Labs. There are lots of cool things in Labs, check them out, just click the little wheel icon in the top right hand side of your browser then select Labs.

Just click to enable the simple but awesome "Send and Archive" feature. What does it do I hear you ask? Well, (obviously) it simply archives a conversation after sending a mail.... This means you can keep you inbox tidy and easy to manage with the least effort. If someone replies in your conversation then the conversation appears back in your inbox.

If you aren't using the Archive feature of Gmail, shame on you :)

Sunday morning, another cup of coffee.

Free 2 factor authentication for Google accounts, use it!

2011-05-16T15:28:00.005+01:00

At the last company I ran, Bmycharity.com, we needed a 2 factor authentication system to pass the stringent PCI certification requirements as we were processing lots of credit card information. It cost a lot of money and you needed to carry yet another device around with you.

It was good though, it really worked, but what is it?

Simply put it works by requiring 2 pieces of information from you when you log in, something you know and something you have. Your password you know, hopefully, you then generate a code from the security device (the something you have) and log in. The code you generate only works once, so even if someone sees you enter the code, or somehow records it, it is of no use to them after you logged in, neat huh?

Without something like this all that a hacker needs is your password and they might as well be you. You should of course have a strong password policy enforced on your system but then what tends to happen is that people write passwords down because they are complicated, back to square one.

The good news is that if you use Google there is now a free (of course) solution provided by the mighty G. 2 factor authentication is now available on all Google accounts after initially rolling it out to Apps customers.

This really is a great addition to your account and I strongly urge you to use it. The more information you store in Google the more attractive it is to hackers, do you really want anyone with an internet connection to be one (maybe simple, maybe written in your wallet) password away from being able to use your Checkout account?

It gets better; you don't need to buy or carry round another electronic device, in fact you probably have it already - your phone. If you have a BlackBerry, iPhone or Android powered phone you simply install the App on your phone which generates the one time access codes.

There are a couple of nice touches too, if you use the same PC all the time and you are happy that it's secure, you can check a box at login and it won't prompt you for another code for 30 days. For apps and services that don't know how to prompt you for a verification code, like deploying App Engine code for instance, then you can generate an application specific strong password from your account page.

I've been using it for about a month now and have no complaints, although I've had to find my phone a few times to generate a new code, the frustration soon wears off knowing my account is secure and hey, I know where my phone is, all good.

There is a getting started guide here, go and do it now and you will sleep better tonight.

Royal Wedding site sensibly heads for the clouds

2011-05-07T08:33:00.003+01:00

If you want a good example of when to use a scalable, distributed Cloud service like GAE for hosting a website then the Royal Wedding site is certainly a good example.

The site is very different from most sites in many ways, especially from a technical standpoint (sorry no more mentions of dresses, kissing or Pippa Middleton from here on).
For a start it is relatively short lived, in all honesty it's not going to get many hits over the next year, especially with the choice of domain name. Accenture, who created the site saw this and also that they were going to get a huge (unknown) spike on the day.

This from the Google App Engine blog:

"The Official Royal Wedding web site generated over 2,000 requests per second, serving 15 million page views from 5.6 million visitors on the day of the wedding. Despite this load, the web site ran smoothly, serving off Google’s shared platform without disturbing the other 200,000+ apps, which collectively serve over 1.5 billion views per day."

So GAE was an easy choice, scale up effortlessly to handle a truly worldwide demand spike but then scale back down afterwards. No infrastructure investment or management panic. It obviously helps if you have some Googlers watching your App as well!

Impressive stuff. As I've posted on the blog, it would be very interesting to have a debrief from someone at Google so we can see things like how many instances did the system start to cope with this spike?

When building sites on GAE it's reassuring to see real world examples like this that work without problems.

http://www.officialroyalwedding2011.org/

Google moves Apps towards business.

2011-05-06T09:41:00.004+01:00

It looks like the mighty Google machine is slowly turning Google Apps into an offering aimed more directly at businesses.

I've just received an email for one of the many accounts I manage saying the following:

"As of May 10, any organisation that signs up for a new account will be required to use the paid Google Apps for Business product in order to create more than 10 users."

So it looks like the original limit of 200 users (and then the ability to ask for more) is being further eroded. It certainly seemed a very generous offer and looks like all existing domains using the free version will still be able to add more users. More recent domains I've set up are limited to 100 users so it looks like this will be the final offering, pushing people to head over to the Business version once they go past 10 users.

Keeping all your pictures online and safe

2011-01-20T15:44:00.005Z

With digital cameras recording larger and larger file sizes the question of where to store all your images and movies while making sure they are safe is a question that keeps coming up.

Every time I download a set from a camera I want them to be available to me and people I share them with, while making sure that they are not at the mercy of a disk failure or losing a laptop. Now we don't print pictures as much there is always the danger of losing precious memories.

There are obviously lots of options for software to manage your images on your PC and then upload the to an online service, with an ever growing library I am looking for a service where I'm not going to run out of space or incur large charges.

I have now settled on Google's Picasa Web Albums for the online storage and sharing. A large part of the reason for this was because the extra storage options seemed such good value. 80GB for $20 is hard to beat. You can go up to 16TB for $4096 a year! If I was starting up an online agency then I might well look at this as a storage platform as there are no traffic fees. Full details of the storage costs are here. Of course this also comes with all the usual Google goodness in terms of backup, once you have uploaded to their storage you can assume your memories are safe.

I've also started using the excellent Picasa for Mac application to make batch uploading of large numbers of files a breeze. This way you can set it off uploading a whole album overnight if you aren't on a super fast connection.

I do keep some of the images on my local machines but this is mostly just being lazy or if I want to send some directly to people. They just slow down backups and are not really necessary as the Picasa web site is great for viewing and now even editing after Google acquired Picnik.

The only annoyance is that at the time of writing you can't add extra storage to your Google Apps account. You still get the 1GB of free storage but you can't buy more as you can with a standard account. I'm sure this will change in the future but for now it's a bit of a limitation for Apps users.

Windows 7 admin shares

2010-12-14T15:56:00.003Z

I hadn't realised up to now that Windows 7 admin shares are disabled by default.

Amazingly you seem to need to make a registry change as well as enabling sharing.. Good article here.

Suppose it makes sense but hope you can turn that on in GP or in some other way if you have large domains that need it!

Sitefinity, the evolution of a CMS

2010-11-16T10:39:00.003Z

Last week Telerik announced what the Release Candidate of Sitefinity 4 would look like. This must have been a big week for the Sitefinity team on all levels. The announcement included the pricing and feature matrix for the product which has been radically changed for the major version release. Sitefinity 4.0 will be a complete rewrite, using the latest technologies including the in house Telerik ORM data access layer.

The webinar and information release provoked a lively discussion regarding both the features and the pricing of different levels on the Sitefinity forums.
Martin Kirov, the Sitefinity Business Development Manager, also explained the rationale behind the new product versions on the company blog.

It seemed that quite a few people (including myself) were initially worried about several aspects of the new version.

Particularly, many people had been looking forward to the analytics, granular permissions and form builder features, not realising these would not be available in the standard edition.
Vassil Terziev (Telerik CEO) who very diligently commented on many of the posts on the forum, has announced that some of these features will now be extended down the range, particularly to the Standard edition which I believe was a gracious gesture, acknowledging the respect for the Sitefinity community that Telerik have. There are not many CEOs of companies that size who will either take the time to comment on individual blog posts, or concede very quickly to any well put argument for a modification. Bravo Vassil!

Many users are very worried that the new feature limiting the number of concurrent users will prevent them from using the new product. This feature restricts the number of users that can simultaneously be logged into the admin part of the product depending on the version. I initially thought this would be a deal breaker for me as for a current project just starting I need lots of front end users to be able to be logged in at once. However it seems this restriction is only people logging in to the admin section of the site, not users authenticated through code on your site. Panic over for me at least.

There has been a wide spectrum of comments and this wide range of views come naturally because people are using Sitefinity for a whole range of clients and uses, from Scout groups to major multinationals. Obviously these groups use the product to vastly varying degrees and one price tag was clearly not sufficient. What Telerik have set out to do is scale the product so you pay as you use it. This is a difficult but excellent aim, not just because it leads to a wider usage but also allows the cost of the product to increase proportionally with the usage - fair I think.

Lino Tadros commented on the Falafel.com blog making the case that Sitefinity was under priced before and must move forwards into the larger market. In his brave (and humorous) defence of the pricing terms he cites paying $250,000 per CPU for a CMS system 15 years ago. I don't think this can specifically be used as an argument for increased prices though Lino! Moore's law makes comparisons at that range a historical curiosity even without the .com bubble craziness added in.

Sitefinity is changing: Telerik has put a lot of resources behind the development of 4.0 (it is apparently the largest department at the company which employs around 300 people in total). It has evolved to start to become a serious contender for the large scale CMS and Application development platform. The amount of work done on the API and extensibility shows clearly that Telerik are not only interested in the product for serving news and blog pages but as a platform for building complex sites on.

The problem is that they have had to radically change the pricing and feature matrix to allow for the increasing capability of 4.0 as well as the high development costs.
This has to happen, it is Sitefinitys coming of age, it is stepping up to compete with the big boys with version 4. As Lino points out, being too cheap when trying to compete in that space is a distinct disadvantage!
However, as Vassil states in answer to a post of mine on the forum, at the moment the feature set is limited and it's hard to see the difference between the versions. We have no modules for cloud storage or the whole application, have not seen the load balancing feature yet, no eCommerce modules and several of the modules included in 3.7 will not be available in the initial release of 4.0.

However, I believe Telerik had to bite the bullet early even if it would annoy some people in the early stages. If they don't do it pre launch then it will be hard or even impossible to do it cleanly later. As soon as people are selling it and developing on it this would be a nightmare and cause much more disruption.

Telerik have obviously tried very hard to make this difficult step as cleanly as possible without causing too much disruption, fully knowing that it would cause a fuss.
They are asking for partners and developers to trust them, that if we buy in and support it now, they will turn 4.0 into the leading CMS/Application Platform.

I believe them and will continue to use and develop on Sitefinity, I look forward to an exciting 2011, to Sitefinity and beyond! :)

Azure Storage Plugin for Wordpress

2010-10-26T15:40:00.003+01:00

It's good to see from several articles on Wordpress that people are starting to think about hosting it in the Cloud. I notice that there is a Plugin that allows you to use Azure Storage (and therefore the CDN) on Azure. This is fantastic as it not only backs up all your content, but also allows you to reduce the load on your server instance, great stuff.

If anyone has any experience of using it in a production environment I would love to hear from you, I will be looking at Wordpress running on Azure shortly.

Vosao, finally a promising CMS on Google App Engine

2010-10-25T08:04:00.009+01:00

I have been testing and evaluating several large CMS systems recently including Sitefinity and Orchard, specifically systems that can be deployed to the Cloud for speed, low startup costs and scalability. Most of my focus has been on large commercial products running on Azure or Amazon EC2.
Once you have added up the cost of the software, then the per hour costs of the hosting these products are fine for business but not attractive or suitable for personal projects.

Over the weekend I started looking at options for my sister, who wants to host some content for some small projects and was looking at the options. Here is the basic brief:

A few pages (around 10)
Low traffic sites but projects that need a permanent Internet presence.
A CMS so it's easy to add new pages and edit content.
Complete control over layout, she wants to design from scratch in Dreamweaver.
Some framework for adding tools, such as sending mails or creating search indexes.

She was looking at options on shared hosting platforms that although don't cost much don't really promise much either. She wants more control than you get with most blogging platforms or site builders such as Google Sites.

Enter Google App Engine:

I have been playing with GAE for a while. For me it's an awesome product, partly because of the power of the platform you are getting, the scaling ability, data store, monitoring but most of all because you get a generous free quota every day. If you hit that quota you can enable billing and pay respectable rates for what you use but for low traffic sites or testing things it's completely free. The fact that it is now being rolled out as a business product is great news, we have even been promised a hosted SQL service in the near future as well, very exciting.

What it lacked though was a simple CMS (but not too simple) that could be easily deployed so users could get on with creating content rather than building an app from scratch.
Finally we now have this with the appearance of Vosao. Vosao is an Open Source project, hosted on Google Code and already supported by an active community. They even offer to install it for you! It provides all that you need to get a few pages on the web for free while having absolute control over the layout.

It is very easy to deploy to GAE, there are instructions on the site. It already has all the basics you need including users/groups, nested pages, fully controllable templates, SEO friendly URLs and more advanced concepts like a plug-in framework capability and structures. The Blog items on the home page use these to separate the markup from user content, it's very simple but elegant. There is a form builder too which looks great.

I think this is going to be a great CMS, it's not yet in it's final release and there is lots more to come by the sound of it.

If you are interested, head on over to the Google Group and support it, I'm certainly going to!

Move to the cloud but don't forget your laptop

2010-10-11T19:44:00.002+01:00

After working on several cloud storage systems over the last few months it is easy to become complacent that all you important data is stored in multiple locations around the world and it will be hard to lose it. But while this should largely be true for any server data it's important to remember your client!

Losing a laptop disk is not as bad as it used to be because so much of the data is now stored online. I've been using Google Apps for all my personal email, calendars, some docs, pictures etc for some time now, so obviously this is all safe. I don't even use a local mail client now, just log in using a browser and get all that constantly updating UI goodness.

I was originally using iDisk to sync other files between both my macs and also to back them up. Recently I have switched to Dropbox as it's free (first 2GB) and is just a fantastic product. It just works and instantly copies your files to any machines you have connected as well as pushing the data to Amazon S3, safe.

So assuming most of your data is already backed up to the net losing a laptop disk is really not that much of a pain in terms of data loss. The only problems are that you will have to reinstall your OS, (OS X in the case of my laptop), install all your apps and settings and log into your online providers and wait for everything to re appear. As I found when a friends laptop died recently, this is still a major pain and takes a long time to get working again.

So I started looking at whole machine backup solutions for a Mac environment. I got a pleasant surprise when I looked at Time Machine, which is included in OS X.

I knew Time Machine essentially makes differential backups of your whole computer, it also provides a typically Apple user interface to look through your backups and recover files you have deleted or lost.
Time Machine also allows you to do a full machine restore in the event of a disk failure which is the bit I'm really interested in.

In my experience, when you set users up with a local backup solution that involves plugging in an external disk it just doesn't happen, they forget or do't have the time to do it, so no backups. Like in a corporate environment, backup needs to happen without the user having to do anything, or even be aware of it. I looked at buying a Time Capsule from Apple which is essentially a NAS device on your wireless network. They are very expensive though and I already have several wireless routers, I only want the storage.

I have a Mac Pro workstation that I use for work and also for streaming music etc. After a bit of reading it looked like Time Machine on a laptop could use a network drive to back up to which was a great discovery. So I bought a 1TB extra disk for the Pro and installed it.
To use time Machine over the network you simply map a network drive from your laptop to the workstation and then point Time Machine at it. It really works too, when you come home and fire up your laptop on the home network Time Machine will connect and backup over the wire without prompting. I have set it up for 2 laptops to the same workstation now without a problem.

Great stuff, it you haven't used Time Machine before then you need to see it, you can click backwards through backups and select files to restore, all with moving stars, very Apple!

Time Machine UI, cosmic!

This is great, especially as Apps like iPhoto and others are Time Machine aware so you can browse your photos in backups.

However, I started looking at this for a complete backup so how does it work if you haven't got a running system?
Luckily you can restore over the network when booting from the install CD, here is how to do it:

Boot from the installation disk.
Connect to the network, either with a cable or by joining your wireless network.
Run the Terminal from the Utilities menu.
Map a network drive to the workstation.
Select "Restore System From Backup" from the Utilities menu.
Select the network drive in the list of available backups.
Select the backup date and time you want to restore from and off you go.

Here is how you map a drive to your backup share from the console.

 mkdir /Volumes/TimeMachine   
 mount -t afp afp://[username]:[password]@[computer]/TimeMachineLaptops /Volumes/TimeMachine

This provides a really neat way to backup everything on your Mac without the cost of a Time Capsule. It protects you against loss of files through accidental deletion but it also protects you against a disk melting down.

The small things in life.

2010-08-27T09:52:00.001+01:00

I've just noticed that Gmail is now allowing you to link images in your signature block. Good one Google!

I've helped scores of companies and individuals migrate their domains and email to Google and it's one of the only things that people had on their wish list. I never get any requests for help on the core functionality of the product, just the finer points like this.

The editor box now has an images button and you get the following window:

So you still need to have your image hosted on the web somewhere but this is fine with the huge array of products out there and it's not trying to embed the image in the email, better out than in :)

It's this sort of constant evolution of UI and sometimes infrastructure as well that makes Gmail the killer app that it is.

All you need to know about everything

2010-08-26T16:39:00.001+01:00

"A human being is a part of the whole called by us universe, a part limited in time and space. He experiences himself, his thoughts and feeling as something separated from the rest, a kind of optical delusion of his consciousness. This delusion is a kind of prison for us, restricting us to our personal desires and to affection for a few persons nearest to us. Our task must be to free ourselves from this prison by widening our circle of compassion to embrace all living creatures and the whole of nature in its beauty."
— Albert Einstein

Personally I think the man was a genius :)
More quotes here.

Azure needs distributed caching built in.

2010-08-19T08:26:00.000+01:00

I'm looking at migrating a large scale web app to Azure at the moment.

The idea is to use SQL Azure for the main structure of the DB. For the time being a single DB will be used although in the future we may look at sharding to really allow it to scale out. Hopefully we will get some development on this from the Azure team so it doesn't have to be done from scratch.

The hope is that by not storing bit content, images or videos in the DB itself but instead storing them on Blob Store and using the CDN to speed up delivery that the DB will not be overworked.

Of course a sensible caching solution is also needed to make sure that as traffic spikes the bottleneck of the single DB does not become limiting.
There will be several Worker Roles deployed as part of the project to groom queues including tasks like sending emails, resizing images and generating reports. It seems sensible to make these roles run a distributed cache as well so that we can keep the pressure off the DB. As the Web Role count is scaled up we will add Worker Roles too to cope with the increased load, so if we ran a distributed cache on these as well the cache would also scale in size. Of course we could add dedicated caching Roles as well if needed.

I was hoping that by now the Windows AppFabric Caching would have made the jump to Azure but it seems that this is still in the pipeline. Hopefully when it arrives it will be able to be run in a Worker Role alongside our other tasks.

In the interim it seems that a common option is to run Memcached in a worker role
http://code.msdn.microsoft.com/winazurememcached/Release/ProjectReleases.aspx?ReleaseId=3551

IT seems that Microsoft Consulting have also been thinking about it in this blog post.
http://blogs.msdn.com/b/mcsuksoldev/archive/2010/02/04/9958494.aspx
So this is good news!
Any news on when we can expect this Microsoft?

New Azure functionality that would increase app migration to the cloud

2010-08-18T16:34:00.001+01:00

Over the past week I have been playing with migrating a couple of ASP.NET apps to Azure.

The first thing I do is make sure that the DB being used its compatible with the Azure feature set. If this step doesn't work then there is little point persisting with the investigation.

The next problem with apps that have a suitable database is that the App_Data folder, so heavily used in many .NET apps and rightly so as content here is not served by IIS, it's the place for all your..well...er app data! In Azure the App_Data folder is read only though.

Microsoft recommend running a minimum of 2 Web Roles in any deployment. In fact the SLA does not take effect unless you do. The Azure system will make sure these 2 roles are not hosted on the same hardware, so giving you hardware redundancy in the event of a host going down, great stuff..
As soon as you are running 2 instances of your app, writing to the App_Data folder becomes pointless anyway as the next user request may be routed to the other instance and the change would not be visible.

So this is understandable but what it does is immediately make all apps that write to this folder unsuitable for migration to Azure without significant coding changes.
So you decide that you want to make your app to work on Azure. What to do?, you have to rewrite it to not write anything to the local storage but to store everything elsewhere.

Steve Marx has created his own Web Role from a Worker Role and cleverly used it to serve web content by using Hosted Web Core internally. He got it doing 2 cool things, running a site using content from a Azure Drive, and also using Blob Storage to host the actual content of the site and sync it to all running instances.

Steve himself is keen to point out that this is an interesting exercise but that use in a production environment is not recommend. As he points out, you then lose all the built in functionality of the Web Role and have to manage IIS yourself etc.

However it's close to what I think we need in Azure, the ability to have a read/write area, probably App_Data that is persistent but can cope with many instances, essentially a cross between the Azure Drive example (which should be read /write already) and the Sync from Blob example but maybe just for the App_Data folder.

In this way, when one Web Role changes something in App_Data, the Role itself syncs this to Azure Blob Storage (so is persistent and can be remotely managed) and raises an event that makes the other Web Roles update their folders. They could cache the content locally for speed and only overwrite when it changes.

While this is probably possible by modifying the HWC examples Steve has created and creating a virtual directory, mapping App_Data to it (I'm not sure this is possible), and creating a 2 way sync between Blob Storage and other instances, I think this would be a good feature to build into Azure itself. With it Microsoft would instantly increase the potential number of existing apps that can be migrated to Azure without modification.

Sitefinity 4 Beta running on Azure!

2010-08-13T15:36:00.002+01:00

With the recent release of the long awaited Sitefinity 4 Beta I decided to see how close it was to working on Azure.

It's been on the feature list for version 4 for some time and although it's not on the list for the Beta I was keen to see how close it was, especially in terms of the DB. I had previously tested the 3.7 version against SQL Azure and found that it didn't adhere to the SQL Azure feature set.

So the first task was to see if the database can be easily moved to SQL Azure. If it can't then there is no point looking at migrating the application itself at this point.

I created a new Sitefinity site using the Project Manager and let it create a new DB on the local instance of SQL 2008.

I decided to use the SQL Azure migration wizard, a free tool available at http://sqlazuremw.codeplex.com/
This great tool allows you to script analyse and apply data to and from Azure.

When you run the tool you are taken through a wizard that generates a script from a datasource that you can then apply to another. Simple, but it also checks that the features of the datasource you are scripting are compatible with the feature set of SQL Azure. Although SQL Azure is essentially a high availability 2008 SQL instance, because of the architecture of the system some features on standard 2008 are not available. For example, different file-groups are not supported, as you have no say in where Azure is storing your data this is an irrelevant feature, it's all taken care of.

I was really impressed that even in the Beta the team at Sitefinity have made the DB 100% compliant with the current Azure SQL feature set.

Another cool feature of this tool is that it also copies the data content of the tables at the same time. It does this by using BCP to insert the data once it has created the table. So it's a tool you can use for deploying Sitefinity projects as well (Needless to say it can also be used for backing up a database from Azure to another location).

Once you have a generated script it's simply a case of pointing the tool to your SQL Azure instance and letting it go. The tool sets out the format of the database name and the user name clearly, remember to trim off the "tcp:" that Azure puts on the connection string, you need that in apps you write but not here.

Also remember to allow the IP address of your machine to access your Azure SQL account in the firewall settings tab of the Azure portal or you obviously won't be able to connect.

Once the tool completes running the script you will have a copy of the Sitefinity database in the cloud.

You can now test this from a local Sitefinity 4 Beta site by simply using the connection string generated by the Azure Portal and copying it into the DataConfig.config file in the project. It won't be very quick as it's going out over the net every time you hit the database but it's good for testing and proof of concept.

In the next post I will show how easily the Sitefinity 4 Beta site can be deployed to a Azure as well.

Here is a basic video of going through the process from start to finish.

Sitefinity 4.0 Beta API look

2010-08-10T01:57:00.001+01:00

Just watched this great screencast from Lino Tadros from Falafel.

Shows both the standard and fluent APIs that are the core of 4.0.
http://tv.falafel.com/default/10-08-08/Sitefinity_4_0_APIs_vs_Fluent_APIs.aspx

One interesting point is that the API is now authenticated so you will still need to authenticate the session before you can make API calls to objects that have permissions.

Blogger Stats hidden away

2010-08-05T15:56:00.002+01:00

After setting up my Blog today I've been looking at putting Google Analytics in there (I try and put it in anything).

Having put the Analytics tracking code in a simple HTML/Javascript widget I am yet to see it collecting data. So a further search finds that Blogger has it's own stats module as outlined below here.

This looks great although very hard to find. They also mention that if you do put in GA as I did then the stats don't match due to different collection methods, I'm looking forward to checking once my Analytics starts recording.

New Blogger page

2010-08-05T10:32:00.005+01:00

After a long time of playing with different options for collecting together all my various postings I have finally settled on Blogger over the other options.

Even just looking at the options within Google it's hard to guess where they are focusing their efforts. Sites is a powerful tool although I believe it is not really mature enough yet to be used in anger, more on that in another post.

So I will try and collect everything here and we will see how it goes, both products seem to have pros and cons, I may end up using both!

Blogs not yet available in Google Apps accounts

2010-08-05T09:47:00.003+01:00

I took the bait and merged my Google Apps domain with my "proper" Google account of the same name, can't see many changes but now I need to change the email on the original account!

Luckily I didn't log out of it and a page refresh revealed the very cryptic new address.

Update: Looks like not much has changed for now, hopefully more products will become available shortly, I realise it's an early adopter program.

UPDATE.

Blogs are now available and although you can't move a blog from one account to another, you can add your Google Apps account as an author so you can edit your blog from your new account.