The .NET Framework's garbage collector manages the allocation and release of memory for your application. Each time you use the newoperator to create an object, the runtime allocates memory for the object from the managed heap. As long as address space is available in the managed heap, the runtime continues to allocate space for new objects. However, memory is not infinite. Eventually the garbage collector must perform a collection in order to free some memory. The garbage collector's optimizing engine determines the best time to perform a collection, based upon the allocations being made. When the garbage collector performs a collection, it checks for objects in the managed heap that are no longer being used by the application and performs the necessary operations to reclaim their memory.
This text (taken from the microsoft website itself) states that the garbage collector will only start to clean up memory when it is absolutely needed. However, this will also mean that allocated memory that is not used for some time will be written to your virtual memory, slowing down the whole process. Other languages (as the age-old c++ for instance) requires the developer to clean up the mess (s)he leaves behind after using an object. This will ensure that the memory is cleaned up as soon as possible. If you run a fair number of applications (be honest, which system runs only what is absolutely nessecary? To ensure all the lights and keys on my laptop work, I need at least 10 little applications to run), you will need to swap memory more often.
Besides that, if you use unmanaged code, you still need to clean up yourself. Even more, If you create an object, dispose of it and create the same object again, the garbage-collector will not allow you to 'reuse' the same memory. If the Garbage Collector would re-init the object after you dispose it and allow the 'new'-keyword to hook into that object, then it might be on to something.
Conclusion: The Garbage Collector is nice for those new developers, but I seriously doubt it will lead to a better world.