I've also been playing around with XNA - very neat stuff there. I was able to throw together a 3D "hello world" app amazingly fast (a simple heightmap).
Unfortunately there is one very big problem with it - distributing a game. As far as I can tell there are 3 ways of doing so:
1) Published on Xbox Live Arcade
This is pretty awesome and could potentially be a great revenue generator, but the process of getting it greenlighted by Microsoft seems pretty daunting for a independent game studio, or even worse a "hobbyist" programmer team or individual.
2) Xbox 360 via content creators club
Nice that you can play it on a 360, but your potential audience is rather small. Though I can understand why Microsoft wouldn't want wide distribution of these games to the general 360 population, and do not blame them at all for requiring a content creators subscription to be able to play them.
3) PC
Absolutely horrible, the end user has to have The .NET Framework 2.0 Redistributable, the XNA Framework Redistributable corresponding to the version your game was built against, and a number of files from the DirectX 9.0c Redistributable installed.
A few years from now when Vista has become more widespread the .Net Framework requirement shouldn't be as much of an issue, but expecting a user to have the other 2 installed is a bit much. If you're making a commercial game your customers are not going to be happy about having to download all that extra stuff - they just want a 1 click installer (and if you've made a casual game only a few megs in size the total amount potentially needed to download jumps in size exponentially). Chances are people will download a demo of the game, then when they realize everything else they have to do just to play it will probably not even bother with it.
As it stands now I can think of 3 good reasons to use XNA - Rapid prototyping of a game concept (which I think is an excellent use of it). Having a publishing deal for live arcade (which I would think twice about if I already had an established codebase in another language from previous projects, or didn't have anyone with extensive C# experience on the team). And finally, if you are just learning programming and want to make games. From what i've seen so far C# is a much better language for a beginner (compared to C++ anyways), and XNA seems fairly easy to use.