Thursday, December 27, 2007

2008: Some thoughts on going virtual

One of the goals for me in 2008 is to bring my production environment over to a virtual machine. 

That should give the following benefits:
1. Complete installations can be backed up, so if a future update fails, it is easy to step back in time. (Less downtime)
2. It is easy to make multiple backups, with multiple states of the installation.
3. It is easy to take you complete production environment with you.
4. More flexible for setting up new software without hurting your existing software. (Testing new technologies etc.)

The benefits are clear, more secure, less downtime in case of an event, and more flexible.
First question that comes to mind thinking about going virtual is:

How to set up a VM environment as efficient as possible?

The answer to this question is, of course, specific for each situation. Because I have never done this, I will share with you how I think about settings things up. If you have any tips, please step in!

I use, the two major IDE's in the market today, for my development, namely CodeGear RAD Studio and Microsoft Visual Studio. The latter is for .NET development only, and mostly ASP.NET development.
So I think about setting up two seperate VM's: (They will be based on Windows XP pro)

1. Delphi VM
- Delphi 7 (for older projects)
- BDS 2006 (.NET personality for Winforms .NET 1.1 development)
- RAD Studio 2007
- Delphi third party components
- Visual SourceSafe for version control

2. Visual Studio VM
- Visual Studio 2005 (Winforms/ASP.NET developement)
- Visual Studio 2008 (future...)
- Third party components
- Visual SourceSafe for version control

On each VM I completely install all necessary software and components. After that I will backup those VM's as the basic production environment setups. With each future update (IDE, components), I will update those basic environments also, keeping them up-to-date. If an update fails there is no problem, just use the last backed up clean environment.

The source code and SourceSafe
On the hosting machine, my laptop by default, I will install all other necessary applications, like Microsoft SQL Server, Office etcetera.
I will keep my application source codes located on the hosting machine, which I can reach in my VM through shared folders. Doing this, I can backup them as usual. With Visual SourceSafe in VM, I can check out files on my laptop, and alter them within the VM.

Well just some thoughts about setting up a development VM.

Happy new year!!


Fishous said...

I've been looking at doing this same thing.

One thing that might be an option for you, is using something like Acronis True Image Echo to make an image of your current system(s) and then restoring that disk image to the virtual machine.

True Image Echo can do that with an add on they have. I haven't tried it yet.

Someone said...

You're on the right track!

I currently only VMs for testing as my machine is a few years old and it would be a bit slower for normal work.

A few hints:

- Use the option to split the virtual disk files into 2 GB files: doesn't affect the VM but it's easier to backup multiple 2 GB files than a single 10 GB file.

- Use multiple partitions, at least 2: 1 for system, 2nd one for Swap file, temporary files, etc. This will allow you to keep the "main" partition smaller and easier to backup/restore.

- Make a "base" VM with nothing but windows/updates/pre-reqs and configure it as needed (Temp/Swap/Mounted Host Drives/Display Resolution/SFC cache removal/etc/etc).

- Clone that base VM into your 2 "main" VMs and install the rest that's specific of those VMs.

- Use the "Revert to Snapshot" option (or "Ask me"): makes shutting down to a "clean" state much easier and faster.

- Don't forget you can have disks be independent of snapshots, so you could have your source files on one such virtual disk and still power down to a "safe" state without needing to fetch the sources again next time, while at the same time ensuring your "environment" stays the same each and every run.

Fernando Madruga

Jay Faubion said...

That's good advice from Fernando. Where I work we still support a large Delphi 6 project, and are slowly/painfully migrating to BDS2007. At the same time I write Windows services and ASP web services in Visual Studio 2005. So, I keep all three of these in a single VM. It's a large one, but it is so convenient to be able to back everything up in a single copy operation.
I can't begin to say how many times - before VMs - I trashed my whole development environment and had to rebuild from scratch. Those days are a thing of the past now. VMs are the only way to go for me. They are literally "throw-away" computers that you can copy, move and abuse at will.

Roland said...

That is indeed a fast way to 'clone' an existing PC though I prefered to install a clean virtual PC.

Thanks for the tips!
The virtual disks can grow fast I noticed. Snapshot is a function from VMWare? Currently using Virtual PC, I think that has only a 'manual' snapshotting feature. VPC has an Undo Disks function, which looks like the same.

GVerdile said...

I did the same thing with my new laptop.
Only suggestion is separate Delphi installations. Maybe it's only an error during installation but my Intraweb update for BDS2006 crashes previous installations on delphi 7.
Now I have a VM for Delphi 7, a VM for Delphi 2006 and a VM for RadStudio 2007.

xsintill said...

I haven't used it yet but there is also something like thinstall which let's you install an application into an image. an image looks like an application but has all the installed components plus registry keys virtualized in the image itself.
After we are done with the new release of our software we might have a look at this ourselves I hope it really is at good as it sounds.
You can have a look at if you're interested.

Rick Carter said...

You might want to consider VirtualBox rather than VirtualPC, especially if you might have use for a Linux guest or host at some point. It's free for personal use, and even for many commercial uses, and I think it would be free for what you intend to do.

Delphi Coder said...

Use Linux as your base OS for hosting your VM's if possible. Linux allows you to select your File system which can have a big impact on performance.

I use JFS for the VM files themselves on a separate VM partition, which has demonstrated superior performance with large files. I use ReiserFS for the Linux system partition, which has proven superior performance for small files.

My distro of choice is Mandriva. I have installed complex systems like Oracle 9.2 & 10G without a hitch on Mandrake/Mandriva, while having nothing but problems on RedHat or - heaven forbid - Debian.

I ran vmWare VM's on Mandrake/Mandriva for 3 years and it worked great. I used to code in Kylix 3 and D7. I had a complete Linux Server test environment and Windows client environment on 1 machine. Worked great!

I even ran multiple VM's, one for D7, the other for VS2005, on a Dell laptop. I found that VS'05, running in a VM hosted on Mandriva, was actually outperforming VS'05 running on pure windows.

Allocate all disk space needed for you VM up front. It's harder to grow your VM if you need to but it really improves performance - especially if you run on Linux.

Also there is a setting in VMWare to fit all virtual machine memory into host RAM. I always select this as Linux puts everything into RAM first, then swaps only if it runs out. So it's a lot faster that way. Believe me, using VM's will make you keenly aware of the need for as much performance as possible.

I currently work in an all Windows environment and find the VM's run much slower, in spite of all of the tweaking that I have done.

Make your VM partitions SCSI, not IDE. SCSI is known to work better in VMWare, and consistently shows better performance.

Finally Acronis True Image v7 (and above?) will allow you to image your host and restore to a VM. I have done this multiple times. Caveat: XP - if that's your base OS - will notice the difference in HW and will require you to reregister. There is a hack/work around for this available on the net- you'll have to look it up.

I wrote a performance paper for my current employer that condenses some of this info. If you'd like I can email it to you. Let me know via the blog. Also VMWare has some performance white papers on their site that I have found useful.


David Keith

Anonymous said...

D7 images are small enough to back up to DVD, but 2007 or later images are so huge that you're just better off getting a removable HD drive cage with a few extra cheap HDs for backup purposes.

My current 2007 environment is 13gb, and unless I want to make images that span DVD disks. Harddrives are cheap now, and their slight expense over dvds more than make up for their immediate accessibility.

RItsaert Hornstra said...

I've just returned to non virtual for daily use. I often ran into performance problems but also practical problems when you need to interface with hardware (like addin cards and USB devices).

One hint: place your source files in the virtual machine and sync them using for example a version control app. When I started I left the source trees on a local harddisk and the performance was very sluggish.

Lars Fosdal said...

I run VMware Workstation (VMW) as well, and have for some time. I don't know how I managed without it. I also tried VirtualPC, but found VMW to be better suited to my needs, and there is a plethora of pre-configured VMs available for download - ranging from mini Linux distro's to preconfigured full fledged web servers.

There are a couple of pitfalls around using Windows XP on VMs. License rules require one XP license per running VM. Vista is more liberal, allowing you to run 4 VMs in addition to your regular Vista install.

For source control, I would strongly suggest using Subversion / TortoiseSVN instead of VSS, and having your core repository outside the VMs. VSS is a disaster waiting to happen. It is not a question of if, but a question of when something will go wrong with the VSS repository.

Roland said...

Thanks for all the input. It makes this post a interesting read.